Recent Posts (page 18 / 70)

Faster or Slower?

Speed kills. Measure twice, cut once. Rework Avoidance Theory. We’re trained from an early age to slow down when faced with uncertainty. To be cautious. To not drive faster than we can see. And often it’s true. Especially when operating heavy equipment. Simple physics tells us that mass * velocity equals inertia. And as Newton said, an object in motion stays in motion. Changing directions or stopping means working against that inertia. So if you want to be agile and be able to change your direction easily then you need to slow down.

So what do we do in software development to make sure we don’t go too fast and make it hard to change direction? A common response is to add process. Long manual processes to ensure that we keep going in the correct direction. That we’re not wasting our time. That nothing bad happens. Cascading gates with lots of places to stop and roll back. That makes sure we’re going slow for safety. Right?

That’s one option for sure, but it’s not the only one. And in many cases, with software it’s not the best option either. Slowing down reduces inertia for sure, but it’s not the only way. Consider the other half of the equation. Reducing the velocity by 50% reduces inertia by 50%. But the same thing happens if you reduce mass. It’s a linear equation, so you can have just as much impact by changing mass.

In development the analog to mass is the size of the change. Smaller changes are less “massive”, so for a given velocity they have less inertia. They have less coupling. They have less surface area. They have less drag. They take less time. They’re easier to undo. So you can make more changes.

There’s another thing to keep in mind. When you slow down changes tend to accumulate, so the “mass” goes up. In fact, it’s not unusual for the mass to go up so much that going slower means there’s even more inertia and it’s even harder to make changes. Which is exactly the opposite of what you want.

Or, as I’ve said before, Speed is Life

Planning vs Scheduling

Q1 is done in most places, so Q2 planning is mostly done as well. Or at least what passes for planning. The list of expected (required?) work has been identified, ordered, and scheduled. Names have been assigned to each one. The “planner” now knows who’s doing what and when it will start and finish. How could anything possibly be more planned than that?

In theory, that is planning. But as they say, in theory, there’s no different between theory and practice. In practice, there is. That happens to the plan almost instantly. Someone is out for a day or two, something takes more (or less) time than expected, or some new piece of information becomes available and suddenly something that wasn’t important is now the most important thing. As soon as any of those, or a host of other possible things, happens your detailed plan, with names and dates and expectations, goes out the window.

The fact that the “plan” is invalid is obvious. What isn’t always obvious, and certainly not as well understood or accepted, is that you didn’t have a plan in the first place, so there’s no way that the plan became invalid. What you really had was a schedule, and the schedule that became invalid.

Now schedules aren’t bad. In fact, whenever you need to coordinate two or more people/teams/orgs/groups schedules are critical. They let the second party know when they can expect something from the first party. They provide a mechanism for sharing information about the current validity of those expectations and changes to them. They let you largely decouple things. They let you identify inverted dependencies and places where things won’t be ready when they’re needed.

Schedules are required for planning. The schedule is one of the inputs to planning, along with your overall goals and your priorities. Planning is the process of taking your overall goals and priorities and adjusting those schedules based on the dependencies discovered so that you end up with the thing you need when you need it. Planning talks about contingencies. Planning defines how you respond to things. Instead of a simple ordered list it’s a branching graph that winds its way to the goal. It’s reactive and responsive to the environment as you proceed. If A happens we do B. To ensure C doesn’t happen we do D. If E doesn’t happen we do F, then change A and D and switch the ordering of G and H to compensate.

So you need schedules to make plans. Unfortunately you can’t schedule everything. You need to know what the priorities and goals are. You also need to know what responses to the environment you might need so you can schedule them. But you don’t know those responses until you finish planning. And you can’t do your planning until you’ve done the scheduling. Which leads us to a dependency loop.

One really good way to break that loop is to do them at (almost) the same time. Top down **and bottom up. The goals and priorities give you a small set of things that need to be done, so schedule them bottom up. At the same time, do some sequencing and strategic planning of the those things to identify dependencies and possible environments responses top down. That gives you some new things that need to be scheduled, and the newly defined schedules give more insight into the dependencies. Iterate a few times until it starts to converge. Then stop planning and scheduling.

Start working and monitoring. As you progress you’ll figure out which of the responses you’ll need. Update the schedules with actuals. Feed the reality back into the plan. Adjust the plan. Lather. Rinse. Repeat. Because like Eisenhower said, Plans are worthless, but planning is essential

And one last thing. Consider not assigning tasks to people when you’re scheduling. Don’t ignore capacity, but putting actual names down is just one more constraint you don’t need. There’s lots more to say about that on another day.

QWAN

You’ve probably heard of the Gang of Four and their book, Design Patterns. You might even have a copy on your shelf. I know I do. There’s a lot of good stuff in there. Reusable concepts (patterns) that can be used whenever the situation calls for one of them. I’ve talked about some of them before, like the builder pattern, or the difference between adapters, facades, and decorators. They’re best practices, but as I’ve said, they’re only best if used at the right time.

One thing I haven’t talked about though, is where the idea of design patterns came from. They originally came from the world of architecture. Way back in 1977 Christopher Alexander coined the term in his book A pattern Language. The book lays out the common patterns that can be used as building blocks when creating a vibrant community. The book laid the foundation not just for the GOFs Design Patterns, it popularized the idea of a pattern language as its own concept.

Pattern languages are a way to express the fundamental building blocks in a field. Building blocks that can be combined to define, describe, and document whatever it is that you’re trying to build. The Bounded Contexts. The Ubiquitous Language. And not just define the thing, but when combined correctly, ensure that what you’ve created has that Quality Without A Name (QWAN) that makes it more than the sum of its parts. Its hard to describe, but you know it when you see it.

As the user of a piece of software, do people like to use the software? Does it simplify things? Make them frictionless? Does it align with the other things they are doing? Does it spark joy? As a developer of a piece of software, is the code easy to understand? Easy to extend? Do the internal boundaries match the external ones? Do you look at it and go “Of course that’s the way it’s done. Why would you do it any other way?” In either case, does it delight you, but never surprise you?

Christopher Alexander recently passed away, but his insights are as valid now as they ever were. So next time you’re designing something think about the pattern language you should be using. And always look for the QWAN.

Do You See What I See?

Context is a concept that comes up here a lot. Usually though, when I talk about context I’m talking about having enough context to understand what you’re doing and why. Or how the different parts of the code you’re working on fit together to provide value. But there’s a whole different kind of context. The shared kind of context. The kind of context that gets used when you’re trying to share information.

Because of course, you have your own context. The things you know. The assumptions you make. Assumptions about the state of things. Assumptions about what other people know. Assumption’s about what’s important to others. Everyone else has their own context too. With their knowledge and their assumptions. And there are some shared assumptions there too.

One of the biggest of those shared assumptions is that those contexts are about the same. That you have shared context. That the things you know are the same as the things as those you’re talking to. That the things you know is roughly the same as what the people you’re trying to communicate with and that’s roughly the same as what everyone knows. And that all you need to do is add that little bit of new knowledge and everyone will be in the same place.

In truth, the situation is a little different. What you know can be very different from what the people you’re communicating with know. If you’re on the same team, working on the same tasks you’ll share lots of specific context to the work, but your history and past experience with be different. Someone on a partner team will have some of the same context/knowledge, but not all of it. People on teams you don’t work with have less shared. People at a different company or in a different industry have even less.

Practically speaking, without ensuring everyone has the same context you’re not communicating. You might think people are agreeing, but without the shared context you don’t know if you’re agreeing about what the situation is. And if you’re not in agreement on what the situation is, how can you reach agreement on what to do about it?

One of the best ways to deal with the issue of differing contexts is storytelling. Don’t just throw out some facts and hope people hear what you hope they do. Take the time to tell a story. Start at the beginning and set the shared context. Tell the story of how you want things to look in the future. Walk them through the story of how you expect to get from where you are to where you want to be or tell the story of how you got from where you were to where you are. Then finish the story with how things have changed and how you’ve achieved your goal.

So next time you need to share information, regardless of whether it’s just information on what the situation is or what the best way to address a problem is, tell the story that makes sure you all have the same context and you’re all seeing the same thing.

Don't just take notes

I’ve got a Remarkable 2 next-gen paper tablet. I got tired of grabbing a new tablet for every meeting and having piles of notebooks with 5 or 10 used pages around my office. Since I’ve had it I’ve been using it for meetings instead of grabbing the nearest piece of paper. Since then I’ve discovered a few things.

First, I haven’t lost any notes since I started using it. I only have to keep track of one thing and it’s always with me. My desk is less cluttered and I’m wasting less paper.

Second, I’m not distracted. There are no pop-ups from Slack, email, zoom/meet/chime/teams, or document updates. There’s no browser to wander off into. No social media to watch. No emails to read/send. Just note taking.

Third, I can organize my notes better. Folders for various things like 1:1s, projects, etc. One shared/shareable space. Not just on my tablet, but online where I and others can see them. I can move things around inside my notes. If I need to go back and add something I can, I can move things around and make room for it, not just draw arrows or say the info is somewhere else.

But the most important thing I found is that having the notes with me is that I interact with them more. Prior to this I would only interact with them when I needed a bit of info that I had recently written down. If I needed it more than I few days later I either forgot I had written it down or couldn’t find the write piece of paper, then forgot about it. Which made most of them kind of useless.

Now, things are different. I see the notes and go through them. I see them in relation to other notes. I add things I remembered later and add follow-up questions. I add action items (and mark them as such) that I think of later. I add things I’ve done as follow-up. I refer to them when someone asks a question. I actually use them. Which was the point of the notes in the first place.

Now none of this (other than having them all on one tablet) is really because I’m using the Remarkable 2. That just fits my personal style of learning/remembering (writing vs typing) better. It’s really the repeated interaction with them that makes the difference.

While I’ll never encourage you to not take notes, I will encourage you to think about what you’re going to do with the notes. Think about how you’re going to get value out of them. Think about how a small change in your mechanism(s) can give you a much larger return on investment.

Hangers and Optimization

I’ve talked about context before. I know context is important. Even so, I still get reminded of its importance by the smallest of things. After just over 2 years I’m on a business trip again and something happened last night that reminded me again how important context is.

When I got to my room I went to hang up my shirts because I don’t want to have to iron them. When I went to hang them up some of the hangers were hung hook in and some were hung hook out. Which begs the question, hook in, hook out, or random?

Which gets right to the question of context and optimization. My father was a policeman. And a volunteer fireman. If you asked him how hangers should be arranged the answer you got would depend on which hat he was wearing. The fireman would say make them all be pointing the same way, preferably in. That would make it easier to grab lots of them at once and take them off the bar if you needed to move them in a hurry. The policeman, on the other hand, would say that you should hang them randomly. That will make it harder for someone to grab an bunch of them and run off. So which way you hang your clothes depends on what you’re trying to optimize for.

In engineering, software or otherwise, the answer is also often, it depends. There are lots of ways to meet the base requirements. Which one to choose depends on what you’re trying to optimize for. Are you optimizing for resource (memory, disk, CPU, network, etc.) usage? Maybe it’s throughput, or latency. It might be an one of those non-functional requirements, like reliability or uptime. Are you optimizing for time? Is the short term more important, or the long term? Or maybe it’s a business requirement. Minimizing cost in one area. Minimizing waste in a division. Maximizing revenue on a product.

Remember that regardless of what you’re optimizing for, there’s always something you’re giving up. Minimizing resource usage might increase latency or decrease throughput. What’s best for tomorrow might not be best for next week, next month, or next year. But there are times when you need to optimize for short term or there won’t be a long term to worry about. You can minimize development costs by not having any developers, but that’s really going to delay your release date.

The trick is to recognize that just because you’ve found a local maximum (or minimum), that doesn’t mean you’ve found the global maximum (minimum). Whether or not that’s the right answer depends.

Loops and Ratios

Test Driven Development, as I’ve said before, is really design by example. But it’s also a systems thinking approach to the problem. Loops withing loops. Feedback and feed forward. Change something (a requirement or a result) in one loop and it impacts the other.

What that image doesn’t capture though is the impact of time spent on various parts of the loop. It doesn’t touch on how the ratios of those times form incentives. And it really doesn’t say anything about how having the wrong ratios can lead to some pretty odd incentives. Which then leads to behaviors which are, shall we say, sub-optimal.

Since time isn’t part of the picture, a natural assumption is that each step takes the same amount of time, and the arrows don’t take any time. Of course, that’s not true. “Make the test pass” is where you spend the majority of the time, right? If that’s true then things work out the way you want them. The incentive is to make the tests pass and you don’t need to compensate for anything. Things move forward and people are happy.

We want continuous integration. We want commits that do one thing. We want reversible changes. We don’t want long lived branches. We don’t want big bang changes. Those lead to code divergence and merge hell. And no one wants that. When most of the time is spent in “Make the test pass” then incentives match and we’re naturally pushed in that direction.

Unfortunately, that distribution of time spent isn’t always the case. There are loops hidden inside each step. Those loops have their own time distribution. To make matters worse, the arrows are actually loops as well. Nothing is instant, and time gets spent in places somewhat outside your control. It’s that time, and the ratio of those times, that lead to bad incentives.

For instance, “Make the test pass” is really a loop of write code -> run tests -> identify issues. If the majority of time is in “write code”, great. But if it takes longer to run the tests than it does to write the code or identify issues then the incentive changes from make a small change and validate it to make a bunch of changes and validate them. This makes folks feel more efficient because less time is spent waiting. Unfortunately, it might make the individual more efficient, but it has all the downsides of bigger changes. More churn. More merge issues. Later deployment.

Consider the arrow to deployable system. It’s not just and arrow. That arrow is hiding at least two loops. The Create/Edit a PR/Commit -> Have it reviewed/tested -> Understand feedback and the Deploy to environment X -> Validate -> Fix. Those loops can take days. Depending on what’s involved in testing it can take weeks. If it takes weeks to get a commit deployed imagine how far out of sync everyone else is going to be when it gets deployed. Especially if you’ve saved some time by having multiple changes in that deployment.

All of those (and other) delays lead to bigger changes and more churn across the system. Churn leads to more issues. Incompatible changes. Bad merges. Regressions. All because the time ratios are misaligned. So before you think about dropping things from the workflow think about how you can save time. After all, chances are those steps in the workflow are there for a reason

Weird Person #2 or First Joiner?

Who’s more important, the person who comes up with a weird idea, or the person who validates it? Sure, if no one comes up with the idea then it won’t come to fruition. But if you someone comes up with the idea and no one joins in then it probably won’t happen then either.

According to legend Steve Wozniak was happily building technology demonstrators for himself, but probably would have stayed at his HP day job if it wasn’t for that other Steve. Steve Jobs, who saw what he was doing, thought it was cool, joined in, and started a movement. That movement grew and the rest is history.

Or consider the Sasquatch Music Festival in 2009. Some got up and started dancing. Nothing happened for a while. The guy was clearly noticed. Someone was recording him, so he was noticed. But not joined. Then someone joined him. That did two things. First, it encouraged the first guy to continue. Second, it changed the narrative from being one weird guy dancing to a couple of guys enjoying the music. The second guy made it acceptable for others to join in. Eventually the whole group joins in and it turned into an event.

It’s the same if you’re developing tools and libraries. Someone builds an API. It doesn’t matter if it’s REST, gRPC, a C++ shared library, or a Go package. Until someone uses it it’s just like that dancing guy. Out there, noticed but unused, waiting for someone to try it out. Your job, as a potential person #2, is to decide if you should try it out. Does it appear to fill a need you have? How reversible is the decision to try it out? While you’re trying it, what are you not doing? If it fills a need, appears to add something valuable, is a reversible decision, and you’re not risking too much time/energy, maybe give it a shot. You might end up being person #2, bringing goodness and joy to the world.

Remediation Surprise

Some call them Postmortems, Post-Incident Reviews (PIR), or Root Cause Analysis. Some call them something else entirely. Regardless of what you call them, taking the time to figure out why something bad happened and what you’re going to do to make sure it doesn’t happen again is a crucial part of a stable, performant, flexible systems. Unfortunately they also lead to remediation surprise.

A key part of the PIR is the list of action items. The things you’re going to do to ensure it doesn’t happen again. Or at least that you’re alerted to it sooner so you can react faster. After all, if you don’t do anything with the new learnings, why bother to learn them?

Things on that list of action items have varying priorities. Some are things that needed to happen soon. They’ll make the future better. They’re not going to change the day-to-day but will make the future better. Some are anti-things. Don’t turn knob “X” without first adjusting knob “Y”. Put that one in the knowledge base and a reminder to yourself to do some validation later.

And there are things that need to happen right now. Things that are critical to continuing to make forward progress. If you don’t do them, you’re going to continue to spend way too much time fighting fires. Some of those are simple changes, so you just do them immediately. They eat into your slack time, but that’s what it’s there for and doesn’t change any of your schedules.

Others though are bigger. And algorithm change. A dataflow change. Things that are much bigger then would fit into your slack time. You need to get them done, but something’s got to give. You just don’t know what. And neither do the folks depending on you. Which is bad. You want to surprise and delight your customer/user, but that’s not the kind of surprise they want.

By the time you get to this kind of remediation you’ve already surprised your customer once, so you really want to avoid surprising them again. One really good way to do that is to tell them what you’re not going to do. What you need to delay in order to do the work to stop giving them bad surprises. And then ask them if the trade-off makes sense from their perspective. Maybe there’s something else they’d rather delay. Or drop entirely. You won’t know until you ask. When you ask, you avoid the remediation surprise.

Poka-Yoke

I’ve talked about guardrails and Root Cause Analysis before. And one of the questions I often ask candidates during an interview is what they (or their team) did to prevent a recurrence of a mistake with significant impact. These are all ways to make things safer in the future.

A general term for building that kind of safety into a system is poka-yoke. The idea that you can, by design, put special steps or checkpoints into a system so that mistakes are hard to miss. Checkpoints along the way that are easier to acknowledge than ignore. So you keep mistakes in the process from turning into defects in the results.

There are many examples of this in manufacturing. Your typical Ikea bookshelf comes with a bunch of boards and a cardboard sheet with the exact number of nuts, bolts, anchors, pegs, and tools needed to build the bookshelf. When you get to the end of the build, if you’ve got parts left over you’ve missed something. You might not know exactly what the problem is, but it is clear that there’s a problem.

There are ways to do this with software as well. Some of it is input validation. It can be as simple as using an Enum instead of a string in your API. If you take a type called HttpVerb, and the only valid values are HttpVerb.Get, HttpVerb.Post, HttpVerb.Put, HttpVerb.Head, HttpVerb.Delete, HttpVerb.Trace, HttpVerb.Connect, HttpVerb.Options, and HttpVerb.Patch then the user can’t specify an invalid value. And if your API only supports a subset of those you can only define that subset, again keeping the user from making a mistake.

The Builder pattern is a great example. You create a builder, set all the properties you want, then call the “Build” method. If you get something back it will work. If you get an error you know why. There’s no way to get a half-baked, inconsistent, or otherwise invalid thing. It might not be exactly what you want, but it is what you asked for. Similarly, another way is through proper separation of concerns. If the only way to get an instance of something is to ask the service/library/factory for it then you don’t have to worry about someone creating an invalid one on their own.

You can also have a poka-yoke step later in the process. If a you’re doing some kind of image analysis and the number of answers is supposed to be the same as the number of images analyzed you can add a step to the process that validates the number of results. If doesn’t prevent a problem, and it doesn’t make sure the analysis is correct, but it does ensure that everything is analyzed. You can make it even simpler. If a step in the process is supposed to have an output then check if there is any output. If not, it’s an error. Regardless of whether the step says it succeeded or not.

Regardless of whether it happens early or late in the process, the important part is that you detect the mistake, and make sure the mistake is recognized, as soon as possible so it can be handled then instead of turning into a problem for others.

How it gets handled is outside the scope of poka-yoke and a topic for another day.