Recent Posts (page 26 / 70)

Speed

It's a truism that going too fast is dangerous. It's human nature to slow down in the face of danger or uncertainty. When you walk into a dark room for the first time you don't run. You walk in slowly, possibly feeling around for obstacles. There's even a Facebook group telling our local community to stfdlouisville.

On the other hand, when I talk to my fighter pilot friends I get a different answer. Speed is life. Even Maverick got it. In that situation they say speed, but they really mean energy. Trading potential and kinetic energy as the situation demands. Because when you're out of speed and out of altitude, you're out of options.

And at an even more basic level, that's what we're looking for. Options. And the ability to choose between them as information is uncovered and understanding changes. The ability to detect an obstacle while you still have time to react to it.

In software, there are a bunch of ways to approach this. Big Design Up Front (BDUF), or the traditional waterfall approach which seeks to know everything at the beginning so that there are no obstacles to overcome. You've either designed around them or done other work to remove them. This can work. Provided you know exactly what you are going to need to do. And the order to do it in. And when it will be done (which is often different from when it needs to be done). Given sufficient time to build the BDUF, you can do a pretty good job of determining what will be built and when it will be ready. Since you know what needs to be done you can keep everyone working. With more people doing more things in parallel and getting done sooner.

Another option is a much more iterative approach. Start with a team and a rough definition of the problem space. You try something. If it gets you closer to a solution you do more of it. If not, you try something else. You're constantly testing how things work. Of course, this requires you to have a definition of what makes a solution valid. And a way to see if you have a valid solution. And you'll need to take very small steps. Checking all the time to see if you're going in the right direction. The faster you can adjust the less work you'll do in a non-optimal direction. And even with small steps, there will be re-work and changes. As the understanding of the problem grows and the success criteria mature, earlier decisions will turn out to be, at best, less than perfect, or maybe even completely wrong. So you fix them and move on. Eventually you'll get to a valid solution. You won't know what it will look like or when it will be until you're almost done, but, as long as your definition of done is correct, it will converge.

On the surface BDUF seems like the right answer. Less rework. More parallelism. More predictability. But… Reality tells us that we never know everything up front. If we just do what's in the plan then at best we've only solved part of the problem, since we didn't understand it. More likely, we missed some interaction or condition along the way and the solution doesn't work. So we end up doing the rework anyway. At the end, when it's harder. And more error prone. And after the scheduled completion date. Which leads to overwork, burn-out, and the mythical man-month. And no-one wants that.

So we try the iterative approach. And that's where speed really matters. Not the speed of the solution. That's an outcome, a result, not an input parameter. The speed that matters, the speed that's the input to the system, is, as Charity Majors tells us, the speed of the decision cycle. How quickly can a developer/team go from recognizing a problem to getting real-world feedback on if the proposed solution makes things better or worse? Because that's the time that drives how fast you can react to new information.

After all, you wouldn't drive a car by looking out the window every 6 months and deciding how to adjust the  steering wheel, throttle, and brake for the next 6 months, so why would you build software for one that way?

Exploitation

Internet Explorer 3.0 had its 25th birthday the other day. In honor of that one of the original devs posted about his experience on Twitter. He tells the story of 2 AM foosball games and having all meals at the office. I joined Microsoft a few years later, and while not every team was working late and having dinner delivered, the number of days that there wasn’t a team nearby that had dinner brought in, with enough leftovers for folks on other teams without dinner to get some, was vanishingly small.

Prior to that I was in the game industry, so I was very familiar with the idea of crunch time and working 20 hour days to get a gold master released on time. Remember, this was before the advent of online updates, let alone downloadable content. Everything had to be in the box, and there could be weeks between sending the master off to the duplicator and getting things on shelves.

Which gets back to that twitter thread, which turned into a bit of a twitter storm. The author remembers it fondly. Talks about how much he learned and how it started his career. Friendships forged in fire that continue to this day. Pride in the effort and the result.

Others, who weren’t there, saw it differently. They see exploitation. They see people giving up their lives and their families for at best, money, or more commonly, a company that didn’t care. That the people who thought they were having a good time were fooling themselves.

I worked with some of those people on other projects. Some of the smartest folks I’ve worked with. And, at least when I worked with and for them, balanced. They knew when to stretch, when to work hard, and when to relax. Did they learn some of that working on IE 3? Almost certainly. Did they work hard then? Absolutely. Too hard? That depends. They didn’t think so. The families I knew didn’t think so. I think they made a conscious decision to trade the time and effort for the experience, learnings, and yes, money, that they got for being there.

Which is absolutely not to say that it is the right way to build software for the long term. In a year the team grew by 10x and they all needed a break after. That kind of effort isn’t sustainable over the long term. And no-one should offer or expect it. But just because someone chooses to work that hard and others benefit, doesn’t make it exploitation. It doesn't invalidate their choice.

Which leaves me very conflicted. At that time I worked that way. On multiple occasions. At the time I didn’t feel exploited. At the time I felt empowered to do the right thing and internally motivated to produce the best product I could. I remember the camaraderie we felt. The fun we had. The things I learned. The closeness and sense or purpose the entire team had. Looking back I don’t feel exploited.

On the other hand, I don’t work that way now. If I see someone doing it now I ask them why. If I’m involved I ask them to stop. I ask them to think long and hard about the choices they’re making and make sure they understand the sacrifices involved. I’ll tell them about my experiences. The things I missed out on. Because it was a sacrifice. I gave up a lot. Looking back, I wonder if I made the right decision. Would I be happier or more fulfilled now? There’s no way to know. Now I don’t make the same decisions. I have a different viewpoint and I don’t think it’s the right choice for me, or for others.

But that doesn’t mean they don’t get to make the choice. These things still happen. Back at Uber, when we were bringing up a datacenter one of the datacenter folks worked for 12+ hours with abdominal pain. He went to the ER, passed a kidney stone, and then came back and finished what he was doing. I, and multiple others, told him not to do something like that again. We had told him at the beginning of the day, and multiple times over the course of the day, to go home. We told him not to come back. But it was his decision.

It was part of the culture. Don’t let the other person down, regardless of the cost. Is it exploitation? I don’t know. Certainly demanding that kind of response and punishing those who don’t leads leads to that behavior and exploitation. If the bosses expect that behavior and treat it as the norm then it becomes exploitation.

But what about when it comes from the bottom up. When everyone is that passionate about what they’re doing? Is that exploitation? And how do you know the difference between a driven team and a team that doesn’t feel safe enough to speak up when they’re put in that pressure cooker? How do you keep passion and drive from becoming a positive feedback loop that turns into burnout? How do you keep from romanticising this kind of pressure?

What do you think? I’d love to hear about other folk’s experiences.

Architecture Isn't Everything

Architecture is important. It’s about the user/customer. About adding value for that person. Engineering, on the other hand, is often the art of the possible. How can we build this in a feasible way? What do we need to do now, not directly for the user today, but so that we can do better for the user in the future. Which is another way to say that it’s a question of balance.

Who the best architect of all times is hard to say, but few would argue that Frank Lloyd Wright and Le Corbusier are among the best. But both of them have designed buildings with significant engineering problems. Situations where the value at the beginning continues to cause problems today.

Consider Falling Water, not far outside of Pittsburgh. It’s beautiful. It fits the setting and meets all of the sage requirements the owners asked for. And it’s not strong enough. Or at least it wasn’t when it was built. Despite the builder adding extra reinforcement on his own because his calculations said more was needed. Or another of his works, Taliesin. Eminently livable, but construction issues and building material choices have created a situation where maintaining the site is far more difficult and expensive than it needs to be.

Another example is Villa Savoye. One of the initial embodiments of Le Corbusier’s 5 Points system, but it leaks. And it’s noisy. And built poorly. And late. And over budget. Because the engineering didn’t match the architecture.

It’s the same with software. For example, Uber’s microservice architecture.

At one point there were more microservices than there were engineers. That’s not always a problem, but it’s certainly something to think about. So much so that for the last few years Uber has been working to tame that plate spaghetti service links.

So while it’s critical to have a good architecture and focus on customer value, it’s also critical to not let the pursuit of value in the moment paint you into a corner where it takes a complete reset to move forward. Instead, maybe something like Evolutionary Architecture.

Intro to TDD

Test Driven Development (TDD) is an interesting concept. In many places the primary artifact of TDD, the unit test, has been used as an excuse to not have a test group/department. After all, if the unit tests pass, then the system has to work right?

Maybe. For a simple program or library that might be the case. But as systems get more complex that is less and less true. Because the behavior of the system is more than the behavior of the parts, and you can’t test system behavior by testing the parts. Integration tests help, but they’re not unit tests, and they’re not a complete substitute for trying the real thing in the real world.

To me, what TDD is good for is Design by Example. Write your example/demo code, then write the code to make it work. Unfortunately, that’s kind of counter to our usual process. Figure out what’s needed. Write a justification. Write a design doc. Write a detailed design doc. Write some code. Write some tests.

So here’s a quick way from Kent Beck, the father of TDD, to try out TDD, without having to change the entire system. In fact. it probably won’t take any longer than it would have anyway. You’ll end up learning about TDD, and probably end up with cleaner code.

1. Change the code as usual
2. Write a test that only passes after the change
3. Revert to before 1
4. Type the test again (copy/paste is cheating & invalidates the warranty of the exercise)
5. Make it compile by changing the code
6. See it fail
7. Change the code to make it pass

The only thing I’d ask is that you do this as soon as you have the test(s) passing, instead of after you do your pre-commit cleanup/refactoring pass. 

I bet you’ll find that the test you write the second time doesn’t look exactly like the test you did the first time. Because now you know how you really want to work. You have a better understanding of the usage of the code, and what the inputs and outputs should be.

So the change in the test drives a change in the design of the code. And the design change means that the code under test looks even less like the original than the test does. But that’s a good thing. Because the code now better embodies the intent. Instead of having the API reflect the underlying code, you’re incentivized (by the test) to make the code match the usage.

Which means your users will have fewer surprises and more joy. Which is always a good thing.

Prioritization

I’ve talked about priority before. And how priorities inflate. Even the opposite of priority. Lots of different ways to keep track of what’s important and how the importance of one relates to another. There’s a little bit about how you prioritize, and some about how you track priority, but nothing about the cultural context you make priority decisions on.

Consider these two approaches to the exact same problem:

"In a world of abundant opportunities we need to pick the best one to pursue next."

"In a world of scarce time we need to decide how to carefully allocate our overtaxed people."

    -- Michael Nygard

The scarcity is capacity. There is more potential work than there is capacity to do it. In all but the extremest situations capacity is fixed. Short term you can overwork and risk burnout. Long term you can add capacity. But right now, the capacity you have is the capacity you have. The question is, how to approach the issue? What potential work do you convert into work to be done, and what work do you leave in it’s potential state?

How to do the actual prioritization is not the question for today. That’s about minimizing WIP or important/urgent decisions. The question for today is the mindset going into the discussion. 

Are you looking to maximize the potential reward or minimize the potential cost of failure? It’s about risk/reward. What are the real risks? What is the real risk for each choice? What is the real upside? Remember, the quality of your decision is not defined by the outcome.

And making rational decisions is something people are notoriously bad at. Check your biases. Sunk costs. Gambler’s fallacy. False dichotomies. Slippery slopes. All ways to convince yourself (and others) that you’re making a logical choice when you’re responding out of fear.

So when you’re prioritizing, make sure you know which way you’re approaching it, and why.

Architecture vs. Engineering

Architecture must include engineering considerations, so that the design will be economically feasible; but the emphasis in architecture is upon the needs of the user, whereas in engineering the emphasis is upon the needs of the fabricator

-- Fred Brooks

How’s that for nuanced? Architecture emphasizes the user, engineering emphasizes the fabricator. Seems reasonable. Of course sometimes the “user” of the component is the next fabricator down the line. Or is the “user” the user of the system? They’ll certainly have different needs. And what about the person doing the maintenance/repair? Is that person a user? Their needs are certainly different. So what do you focus on? In true architect’s fashion, it depends.

Consider the late 70’s Pontiac Sunbird and it’s siblings. They sold cheap. They got you from point A to point B relatively economically. They were easy/cheap to manufacture. Designed to meet the users needs. Until you need to do a tune up. Which was typically once a year (every 12-15 thousand miles). It’s not that the parts were expensive, or hard to come by. But it was a little hard to change the spark plugs. In fact, to change the back spark plugs you needed to at least move the engine, if not remove it. Which upset the mechanic and the owner when they saw the bill for a tune up.

To me the mechanic is the user of the system. And this is a case where the system architecture missed the mark. Of course, the folks who designed the system weren’t the ones who needed to do the repairs, so they weren’t properly incentivized.

In our world, we’re the designers, builders, and mechanics. And in some (many?) cases the user as well. So we should be incentivized to make our systems not just easy to build, but easy to maintain and use as well. They should not only be easy to repair, but they should tell us when things are starting to go wrong. When we have time to fix them, Not right after the wheels fall off and it’s a big job to fix it.

So, in true architect’s fashion, it depends. It’s a balance. A balance of builders, users, and maintainers.

Missing Tests

Unit tests can do a lot of different things. Of course, they make sure things work the way you want them to at the beginning. They can provide documentation (and sample code) for your users. Your happy path tests are written the way the code should be used by customers, right? They can just look at the examples.

Unit tests are also great for debugging. If you’re going to be debugging you’re going to need a simple repro case so you can inspect the situation. One easy way to implement that repro case is with a unit test. After all, the fact that the bug got to users means that you missed an important unit test, so this is your chance to fill that gap.

Of course, now that you’re written the new unit test, your tests don’t pass anymore. So what’s a poor developer to do? You do the same thing you always do when a change in the code causes a test to fail. Just to be clear, that should not be to disable the test. No, you get out your favorite debugger and see where the problem is. Once you find the problem, you fix it. And make sure that your fix doesn’t cause any other tests to fail.

In case you were wondering, there’s a name for that style of development. It’s called Test Driven Development.

Finally, once you fix that bug, you’ll find that what you think is a bug is a feature to someone and they still want it to work the old way.

On Decisions

We make decisions all the time. Some with lots of degrees of freedom, some with only a few, or even one. Sometimes we make decisions with lots of empirical evidence and experience, other times, not so much. And sometimes, we choose to not choose, but as Neil Peart said,

If you choose not to decide, you still have made a choice

And assuming free will, choices have consequences. The question is, how do you know if you made a good decision or not? What do you measure to grade a decision?

At least, you should think about the information you had when you made the decision. How reliable was it? How much weight did you give it? Was the data biased? Were you biased in your interpretation of it? That doesn’t mean to exclude experience, history, or your “gut feel”, but it does mean that you need to be honest with yourself about how much you let intangibles influence your decision.

How much do you really need to decide? Do you need all the details, or is choosing a direction enough at this point? Often, the earlier you make a decision the less specific it needs to be. You can adjust to changes and new information. Other times, it’s a key decision that all other choices will guide off, so specificity is important.

Did you compare the cost (whatever that means) of implementing one decision or another against the potential benefits if things work out the way you want and if they go the other way? The less information and surety you have about the outcome the more important it is to think about not just the cost of implementing, but the cost of recovery.

Did you think about the impact of your decision on the decisions and plans of others? Does it work towards or against their goals? There’s no right answer, but a good decision takes those things into account.

In short, can you give a clear, logical, unbiased, and unambiguous explanation of why you choose what you did? If you did, it was probably a good decision. If you can’t go back and re-evaluate your decision and understand how you can make a better one.

And here’s an even more important point. Whether or not you made a good decision is orthogonal to the outcome of the decision.

Mutators and Accessors

That’s often how they’re taught, but really, they’re just another name for getters and setters. They’re a great way to encapsulate things. After all, you don’t want folks depending on the internal implementation of things. You want high coherence and low coupling, right? 

Or, are they a code smell, otherwise known as Accessors are Evil? Consider the lowly Rectangle class. Does a getter really give you isolation? Instead of declaring a public member Foo of type int you have a private foo and a public method Foo. Is that really any different? You’re exposing the internal structure with your getter. The only difference is some extra typing. Setters are just as bad. But what’s a poor developer to do?

The first thing to do is change how you think about it. Instead of mutators and accessors, think of the interface of your object as a set of commands and queries. You’re not changing the properties of the object, you’re telling it what to do itself, or asking it questions about itself. That’s isolation and encapsulation.

Sure, under the covers it might look the same. Ask the rectangle what it’s height is and you get back the height. Which might be stored as a member variable. Or it might not. But it doesn’t matter because you’re not asking the rectangle for internal data, you’re asking for it’s external property.

The same thing applies on the setter side. Don’t double the height and width bindly. Send the rectangle a command. Tell it to be twice as tall and twice as wide. Again, for a simple rectangle it might look like a setter, but it’s not. It’s an interaction with an object, A verb. A command.

It’s the difference between Object Oriented Programming and Programming with Classes. Superficially they’re the same thing. Especially in school. But when your objects get more complicated than rectangles, triangles, and circles it starts to matter.

So the next time you’re looking at the public API for your class/library/service/tool, think about how you’re thinking about it. Are you just putting some static sugar around the internal state, or are you presenting an API that drives users towards the domains and isolations your system is built on. And think about which one is going to be better for you and your users next week, next month, and next year.

Another 4 Questions

Not those 4 questions, but the 4 variations of one very specific question, Why are we building this?

Why are we building this?
Why are we building this?
Why are we building this?
Why are we building this?

Because until you answer the 4 variants, you haven’t answered the question. There are lots of things we can/should build. Assuming there are some form of constraints, we can’t build them all. So which one(s) should we build?

Start with the first variant, “Why are we building this?” What’s the value proposition? To the customer and the company. If it’s cool, but not something the customer wants (or a step on the path to get there), why?

“Why are we building this?” Are we the right team to build this? We might need it, but should we be building it? Does our team have the right knowledge/experience? Are there other teams or platforms that are a better fit?

“Why are we building this?” The classic build or buy question. Are we looking for a capability that is easy to buy on the open market, or is this part of the special sauce needed? There are costs both ways. Buying has a clear cost, and developing a solution has a cost that can be measured, but maybe not predicted. Even free open source software has a real operating cost.

“Why are we building this?” Should we be building something else first? And here, I mean first from a sequencing standpoint, not priority. If A depends on B, which depends on C, you should probably start with C, or at least as much of it as you know. If you build A first you’ll either need to change it significantly when you finish C or you’ll severely limit your options when it comes to C. Going the other direction lets you build on things, reacting to new information as it comes.

Which brings to Wardley Mapping. A way to use the answers to those questions, as applied to the various things you need to do, to help you figure out how to allocate your precious resources. The classic Wardley map is a simple 2D plot of linked components. The Y axis represents the customer/user value, which higher value being higher on the plot. The X axis represents the “maturity” of the component, with “No one’s ever tried this before” on the left, and commodities (cheap to buy exactly the right thing) on the right, and the ability to outsource it somewhere in the middle. The things at the top left are the most important to the customer and hardest to obtain. The things at the bottom right are mostly invisible to the user/customer and easy to buy.

Then, you link the components by direct dependency. What other things do you need to get the most important thing done? The further to the left the more likely it’s worth using internal resources for. The further to the right the more likely the right answer is to buy the solution.

Or, to put it simply, if you need a ½ inch hex bolt, buy one, don’t build a foundry. But if you need a fiberglass layup with compound curves around an assembly you’re still designing do it yourself.