Recent Posts (page 21 / 71)

Moving On

Today was my last day at Aurora. Some of you I’ve worked with for 15+ years across 3 ½ companies, others only for a few months at one company. Regardless, to all of you, a big thank you. You’ve all helped me learn and grow. As a developer, an architect, a mentor, and as a person.

Even though it’s been just over a year, I look back at all that’s been accomplished and it’s truly impressive. Aurora merged two companies’ people, technology, and infrastructure. It’s gone public. It’s pulling loads for partners. It’s showing the world that they can move the goods and manage the fleets.

Although I’m moving on, I have complete confidence that they will successfully bring not just the Aurora Driver, but the entire ecosystem, to market, safely, quickly, and broadly.

Thank you again, and I’ll miss you.

Exit Criteria

Over my career I’ve written plenty of documents. Requirements docs, design docs, project specifications, white papers, and vision docs. They all have a few things in common. A definition of the current state, the problem with the current state, how the thing being written about will solve the problem (goals), and what issues it won’t address (non-goals). After that the different docs have different focuses and levels of detail appropriate for the role the doc was supposed to fill.

Over the weekend I ran across another section that should probably be in many of those docs. The exit criteria. Not the exit plan, which is a good idea, and says what you’ll do if/when you need to do something else. The exit criteria aren’t how you’ll change, it’s the set of questions/markers that tell you when to change.

Because one of the hardest things to do is to stop doing something that has been working well for a long time. After all, if it ain’t broke, don’t fix it. But what if it is broken, and you don’t recognize it, or don’t want to admit it? It’s also the sunk cost fallacy. It used to work. All I need to do is make one little tweak and all will be well. Besides, I built it, so it must be good.

This is especially true at the Process/Policy level. When I started at Uber there were 14 cultural values (14 is way too many, but that’s a separate issue). They worked really well for Uber at the beginning. They kept people working together, working quickly, and let the company grow much faster than outsiders expected. Then, over time, people started to use them not to make better decisions for the company, but to make themselves look better, sometimes at the expense of others. They took those values and used them to get what they wanted. Toe-Stepping and Be Yourself went from sharing diverse viewpoints to find the most effective to the loudest voice in the room wins. Let Builders Build went from don’t be blocked to promotion-based development. There are lots of other examples

But the values didn’t change. Instead, we started layering process on top of process to try and reduce the impact of weaponized values. Re-education and posters on the wall. An ever-changing promotion process. Locking the beer taps before 6:00 PM. Those things helped a little. Or at least helped reduce the expression of the problem, but they didn’t fix the problem. Eventually things got so bad that we got a new CEO and new values.

It’s by no means the only reason, but the fact that we didn’t revisit the values certainly contributed to the problem. And one of the big reasons they weren’t revisited was that everyone looked at the values themselves instead of the impact those values had. After all, who wouldn’t want to be themselves at work?

I don’t know exactly how to word it, but if, in the detailed description of each value it said something like “When this value starts being used to <XXX> we will reexamine ALL the values and ensure that they still represent and are being expressed as the values we want for Uber” things might have turned out a little differently.

This applies to documents, not just values. When you’re writing a design doc for a system you design it for a certain environment, with some allowed variation. Be explicit about that. If you’re designing a system to handle 100 QPS then it should probably be designed to handle 120-150 QPS just in case. And there should be a note in the doc that says “This design is no longer valid when QPS > 80 for longer than XX” so you know that you need to revisit the problem.

When you come across a system that is outside of its design parameters, make sure you understand how the current situation differs from them, and react accordingly

Reading Roadmaps

Roadmaps are important. Having an idea about how to get where you"re going is pretty important to getting there. You know what’s more important than a roadmap though? A destination. Because if you don’t have a destination, you don’t know if your roadmap is going to get you to the right place. But even with a destination, you need to know how to read (or write) a roadmap.

Inevitably though, you end up presenting them to someone. When you do, or even (especially?) when someone presents one to you, remember Sam Higham’s advice:

• This is a plan, not a promise
• New ideas are still welcome
• We don’t have all the answers
• Not everything will work
• We will update you as we learn more

There’s a lot to unpack there. Of course, there are the specifics. Things can (and will change). It won’t go exactly the way we think it will. No one has done exactly this before. If there is something that hasn’t been thought of, bring it up. We don’t know everything. We don’t even know all the questions to ask. We’re all learning and will continue to learn as we execute on the roadmap. The part that should be a promise though, is the promise to honestly provide updates as things progress.

Then there are the intangibles. It’s saying “Yes, we made a plan, but I"m not sure.” It’s an admission of imperfection. It’s an admission that we don’t know everything. It’s an admission we might be wrong. It’s a statement that we"re willing to change. As the person providing the roadmap it takes courage to make that kind of statement. And as the person who’s being shown the roadmap, it takes courage to accept the ambiguity and uncertainty inherent in it.

That’s why having a sea chart can be better than a roadmap. A sea chart embraces the uncertainty in the plan. You know where you are, you know what the known issues are, and you know where you expect to be along the way. Once you are on the way, you update the sea chart with your position and any other info discovered. Then you do some more planning, chart the best known course from where you are to where you want to be, and continue.

Kinds of Costs

Costs are costs, right? You pays your money and you takes your choice. Simple, right? Not really. There are all kinds of costs. Up front, back end, hidden, and delayed for example. All of those are relevant to development. Besides those, there are two major ones to be aware of because they often turn into biases

• Opportunity Cost: The time & money you spend on X means you can’t do Y. So choose wisely.
• Sunk Cost: The time money already spent on X is gone. Is Y cheaper than continuing X?

Opportunity cost bias often gets expressed as one of 2 ways. The first is fear of making the wrong choice. Instead of thinking about what you should do and devoting enough resources on X to actually get it done, choose not to decide and peanut butter the resources around. You spread them thinly over multiple choices, and instead of finishing something you end up not finishing anything. Just like you want to [minimize work in progress] /posts/2021/09/09/). You have the time to get things done. You want to focus your resources.

The second is assuming facts not in evidence. Things like assuming you know exactly what the result of the different choices would be. That choice X will have a 100% return, while choice Y will have a 1000% return. It might happen, but what are the odds of each. You can’t use the potential upside without considering the probabilities. Not just ROI, but things like time, or how others will perceive it. You might assume X will take 2 days and Y 2 weeks, but that doesn’t mean it will happen. You might like bright colors and choose to paint a salmon-pink accent wall in your kitchen, but that doesn’t mean your prospective buyers will like it. There’s a reason why your real estate agent stages your house without doing anything unusual.

Sunk cost bias also come from fear. Fear that you’ve wasted your time/money/resources. No one wants to waste things, so you get that feeling you have that you’re almost done. That solving one more little problem will make everything work. After all, you’ve already invested (sunk) a bunch of resources into it. A little more to make it work makes sense. And if it’s a little more then you’re right. But if you let the fear (or pride of ownership) take control then you underestimate the cost of continuing and overestimate the cost of trying something new. Otherwise known as throwing good money after bad.

The thing is, both costs are real. You need to take them into account. The trick is to make a good decision. One based on facts, not biases.

Cupid

In honor of Valentine’s Day, a new acronym. SOLID has been around for a while, and there are lots of good things about SOLID. Applying the SOLID principles (almost) always leads to better code. But, as @tastapod wrote, SOLID is about principles, and with principles, like rules, you’re either in compliance with the principles or not. Now you just have to decide how compliant you want to be. The answer to that of course, is it depends. Strict, absolute compliance can lead to over-abstracted code that might work well but is effectively opaque to someone new to the codebase. Ignoring SOLID can lead to a big ball of mud.

Enter CUPID. A set of properties for your code, not a set of rules to follow. The more your code exhibits these properties, the more readable, the more understandable, to another reader it is. So what is CUPID?

Composable: Each function/library/package has clear boundaries and works well with other functions/libraries/packages

Unix Philosophy: Do one thing and do it well

Predictable: Does what you expect. The principle of least surprise

Idiomatic: Feels natural. Don’t be too clever

Domain based: The solution domain models the problem domain in language and structure

CUPID is a set of properties that your code should have, not a set of principles to write it by. And the choice of those 5 properties was driven by a set of meta-properties. Each of the CUPID properties needed to be practical, human, and layered.

It’s those 3 meta-properties that really differentiate CUPID from SOLID. Instead of being about the code itself, CUPID is about how the code feels to the people who have to write, read, and maintain it. But don’t take my word for it. Read what Dan has to say about it when he introduced it.

Because, as Martin Fowler said,

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”

More Perspective

​​I’ve talked about perspective before. Your point of view is a huge component of how you look at a situation. Let’s say you’re a fish. And let’s say you needed to navigate from Portland, Maine to Portland, Oregon. A map would be a good thing. It would really help you get from Portland to Portland. But for a fish the map wouldn’t look like most of the maps you’re used to seeing. It would look a lot more like this:

Lots of ocean and a bit of coastline. (Yes, there’s too much land showing in Africa and major rivers aren’t shown, but you get the idea). Because that’s what’s important to fish. They don’t know, or care, what’s happening in Kansas or the central Asia. As far as they’re concerned those places might as well be on the moon. They have an impact on the world, but when getting from Portland to Portland they just don’t matter.

Working on software can be a lot like that. As the developer you care a lot about the internals. How the pieces fit together. How they communicate. How a change in one place impacts another. Your customers, on the other hand, couldn’t care less. If it’s an application they care about how many button clicks it takes them to do their task. If it’s a library they care about how good the documentation is and how consistent the API is. In general, customers want you to do the thing that makes their lives easier. The only time they care about making your life easier is when it helps them get what they want.

There are lots of ways that this impacts us, but one of the most important is how we communicate with our customers. When communicating with customers the key is to speak in their language. To talk about the benefits they get. How their life will be easier/better. Not that run time is ½ or that memory required is lower, but that they’ll be able to run twice as many things in the same time and it will cost them less. That’s customer benefit. Not “We’ve re-built the system as a set of microservices to reduce hard coupling”, but “Our new microservice architecture lets makes the system more resilient to network and hardware issues and lets us respond to customer needs faster”. Which one do you think customers want more?

So next time you need to explain something to someone and you draw a picture, make sure it’s from their perspective.

Beware Of Just

This comes up in lots of different contexts

but it boils down to “I know you’re busy, but here’s a simple and obvious (according to my cocktail party understanding) solution to the problem I’m having. I need it working tomorrow.” 

After all, how hard can it be to write a script to automate it? You already know how to do it. Response times for your web query are slow. Just add a cache. It’s simple.

It’s simple on the surface, but when you dig into it there are a lot of details. Lots of fiddly bits that need to be just right, or you end up in worse shape than you were before. The first implementation of the script hides error messages, or worse, ignores errors and continues on, doing the wrong thing. The cache you just turned on gives inconsistent results depending on when you ask and where you ask from. All of these things are correctable, and when you get it working you are in better shape, but it’s not as simple as “Just cache it”.

I can be guilty of this myself. I used to say to people, "While you’re in there, can you make this other change?" While it might be the right time, it’s never as simple as that. Neither I, nor anyone else should expect it to be, yet we keep doing it.

So Just Remember, when you hear someone say “Just ….”, it’s never as easy as it sounds.

Perspective 2

Consider this

"Everyone has this problem so clearly we can tolerate it"

Vs

"Everyone has this problem so clearly we should solve it"

Start with the exact same situation and come up with the opposite result. How is that possible? Is one right and one wrong?

As usual, my answer is, It Depends. It depends on what the problem is. It depends on the impact of the problem. It depends on your goal. It depends on your capabilities. It depends on what you mean by tolerate and solve. But most of all, it depends on the perspective you look at it from.

The problem, whatever it is, is how the current state differs from a desired state. Your perspective is what determines how you see it. Do you see the problem as an impediment to everyone’s progress that you should just minimize and move on, or do you see it as an opportunity to improve things? An opportunity to give yourself a competitive advantage by removing the impediment.

Solving the problem is better. So, we should be solving every problem we see, immediately, right? Unfortunately, no. You can’t solve every problem. You really can’t solve every problem at once. And partial solutions are often worse than the original problem, so you don’t want to start something you’re not going to finish. Which means you need to take the time to understand the problem and make an informed decision about which problem(s) to solve and which to tolerate (for now).

Which brings us back to what it depends on. Does the problem take 5 minutes, once a week, or 10 seconds 25 times a day for 100 people? That’s big difference to its impact. Will it take a few hours to fix or a few months? The ratio of impact to cost is a big input. Is the problem (and its impact) on the critical path to your strategic goals? If it is, the importance goes up. Do you have capability to solve the problem? The knowledge, the time, and the resources? Since those aren’t unlimited, what isn’t going to get done if you solve this problem? What’s the impact of not solving that problem? The costs and benefits aren’t always obvious, and the trade-offs are even less obvious.

The key though is the perspective you see it from. Is it an impediment or an opportunity?

Write Your Unit Tests FIRST

There are at least a couple of ways to take that. There’s the temporal definition of first, meaning you should write your unit tests before you write the code that makes the tests pass. That’s pretty straightforward. Write a set of tests that verify the functionality you’re about to build. Write some more tests that verify failure modes. When all your tests pass you’re (almost) done. That’s classic test driven development, or as I like to call it, design by example. It’s not perfect, but when you’re done you have working code, proof that it works, and some examples of how it’s expected to be used. If that’s the only way you interpret “Write your unit tests FIRST” you’ve gotten a lot of benefit.

But you can get even more by writing your test FIRST. That’s FIRST, the acronym. It’s not a TLA, but it’s close.

• Fast : Unit tests need to be fast. You’re going to be running them hundreds (thousands) of times. Even more if you’re using Continuous Testing. The longer they take the more intrusive they are, or worse, they start to get skipped, which eliminates much of the benefit
• Isolated : Each test should stand alone. It shouldn’t require any other test runs first, or set the stage for the next test. Ideally all the tests can run in parallel with no change to the results. They should be fully isolated from external systems, and if some kind of resource is needed it should be created by the test before it runs. Finally, the test should test one and only one thing. Test the happy path. Test a specific failure mode. Test initialization. In the limit there’s only one assert on the result. Not always possible, but a good goal.
• Repeatable : Each test should give the exact same result every time. Partly that comes from being isolated, but also includes handling of threads, race conditions, setup/cleanup, and (pseudo) random numbers. All flaky tests do is waste time. First you waste time figuring out it’s a flaky test, then you’re wasting time running tests that don’t mean anything.
• Self-Verifying : Unit tests either pass or fail. And they pass or fail as a group. They can’t require someone to look at the results and decide if it’s pass or fail. If there’s something important to the result in the console or log output then capture it and look at it. No asking a human to decide. There’s no such thing as needing 80% or better passing to call it success. No, "test A or B can fail, but not both". If that’s really the way it is then write your tests such that a single test passes or fails based on that criteria.
• Timely : Tests are written in a timely manner. There’s really two benefits here. You get the design by example from above, but you also make sure your code is testable. Sure, you can always refactor the code to make it testable, but why? If you do it that way the first time you don’t have to go back and redo it. There’s going to be a lot of re-work as you learn more anyway, so why add to the burden?

So before you go diving into some code, either to fix a bug or implement something, write the tests FIRST.

Sooner, not Faster

What’s better, faster or sooner? More to the point, what’s the difference between them, and why should you care?

Let’s start with some definitions:

Fast:

1. Moving or able to move quickly

   She’s a very fast runner.
   a fast car

2. Happening quickly : taking a short amount of time

   a fast race
   We’re off to a fast start.

3. Operating quickly a fast computer

Soon:

1. Without undue time lapse : BEFORE LONG

   soon after sunrise

2. In a prompt manner : SPEEDILY

   as soon as possible the sooner the better
   no sooner said than done

At its core, fast is about speed. It’s about movement. Speed is distance divided by time, so more speed is either more distance or less time.

Sooner, on the other hand, is temporal. It’s about time. It’s about the time between starting and finishing. The only way to be sooner is to take less time.

So faster is a function of distance and time, while sooner is just a function of time. Sooner is faster, but faster isn’t always sooner. That’s the key difference, and why you should care.

To know which is better though, first you need to define better. In this situation I define better as maximizing value added. That means value as defined by the customer/user, not as defined by the set of features provided. Which means getting feedback and acting on it.

Combined, that means sooner is better than faster. Sooner means whatever value is being delivered gets delivered before it otherwise would have. Getting feedback earlier and being able to act on it means you deliver what the customer wants/needs before you might have.

And that’s where being agile comes in. Working software. Customer collaboration, Responding to change. That helps you deliver sooner, not faster.