Recent Posts (page 26 / 70)

Architecture Isn't Everything

Architecture is important. It’s about the user/customer. About adding value for that person. Engineering, on the other hand, is often the art of the possible. How can we build this in a feasible way? What do we need to do now, not directly for the user today, but so that we can do better for the user in the future. Which is another way to say that it’s a question of balance.

Who the best architect of all times is hard to say, but few would argue that Frank Lloyd Wright and Le Corbusier are among the best. But both of them have designed buildings with significant engineering problems. Situations where the value at the beginning continues to cause problems today.

Consider Falling Water, not far outside of Pittsburgh. It’s beautiful. It fits the setting and meets all of the sage requirements the owners asked for. And it’s not strong enough. Or at least it wasn’t when it was built. Despite the builder adding extra reinforcement on his own because his calculations said more was needed. Or another of his works, Taliesin. Eminently livable, but construction issues and building material choices have created a situation where maintaining the site is far more difficult and expensive than it needs to be.

Another example is Villa Savoye. One of the initial embodiments of Le Corbusier’s 5 Points system, but it leaks. And it’s noisy. And built poorly. And late. And over budget. Because the engineering didn’t match the architecture.

It’s the same with software. For example, Uber’s microservice architecture.

At one point there were more microservices than there were engineers. That’s not always a problem, but it’s certainly something to think about. So much so that for the last few years Uber has been working to tame that plate spaghetti service links.

So while it’s critical to have a good architecture and focus on customer value, it’s also critical to not let the pursuit of value in the moment paint you into a corner where it takes a complete reset to move forward. Instead, maybe something like Evolutionary Architecture.

Intro to TDD

Test Driven Development (TDD) is an interesting concept. In many places the primary artifact of TDD, the unit test, has been used as an excuse to not have a test group/department. After all, if the unit tests pass, then the system has to work right?

Maybe. For a simple program or library that might be the case. But as systems get more complex that is less and less true. Because the behavior of the system is more than the behavior of the parts, and you can’t test system behavior by testing the parts. Integration tests help, but they’re not unit tests, and they’re not a complete substitute for trying the real thing in the real world.

To me, what TDD is good for is Design by Example. Write your example/demo code, then write the code to make it work. Unfortunately, that’s kind of counter to our usual process. Figure out what’s needed. Write a justification. Write a design doc. Write a detailed design doc. Write some code. Write some tests.

So here’s a quick way from Kent Beck, the father of TDD, to try out TDD, without having to change the entire system. In fact. it probably won’t take any longer than it would have anyway. You’ll end up learning about TDD, and probably end up with cleaner code.

1. Change the code as usual
2. Write a test that only passes after the change
3. Revert to before 1
4. Type the test again (copy/paste is cheating & invalidates the warranty of the exercise)
5. Make it compile by changing the code
6. See it fail
7. Change the code to make it pass

The only thing I’d ask is that you do this as soon as you have the test(s) passing, instead of after you do your pre-commit cleanup/refactoring pass. 

I bet you’ll find that the test you write the second time doesn’t look exactly like the test you did the first time. Because now you know how you really want to work. You have a better understanding of the usage of the code, and what the inputs and outputs should be.

So the change in the test drives a change in the design of the code. And the design change means that the code under test looks even less like the original than the test does. But that’s a good thing. Because the code now better embodies the intent. Instead of having the API reflect the underlying code, you’re incentivized (by the test) to make the code match the usage.

Which means your users will have fewer surprises and more joy. Which is always a good thing.

Prioritization

I’ve talked about priority before. And how priorities inflate. Even the opposite of priority. Lots of different ways to keep track of what’s important and how the importance of one relates to another. There’s a little bit about how you prioritize, and some about how you track priority, but nothing about the cultural context you make priority decisions on.

Consider these two approaches to the exact same problem:

"In a world of abundant opportunities we need to pick the best one to pursue next."

"In a world of scarce time we need to decide how to carefully allocate our overtaxed people."

    -- Michael Nygard

The scarcity is capacity. There is more potential work than there is capacity to do it. In all but the extremest situations capacity is fixed. Short term you can overwork and risk burnout. Long term you can add capacity. But right now, the capacity you have is the capacity you have. The question is, how to approach the issue? What potential work do you convert into work to be done, and what work do you leave in it’s potential state?

How to do the actual prioritization is not the question for today. That’s about minimizing WIP or important/urgent decisions. The question for today is the mindset going into the discussion. 

Are you looking to maximize the potential reward or minimize the potential cost of failure? It’s about risk/reward. What are the real risks? What is the real risk for each choice? What is the real upside? Remember, the quality of your decision is not defined by the outcome.

And making rational decisions is something people are notoriously bad at. Check your biases. Sunk costs. Gambler’s fallacy. False dichotomies. Slippery slopes. All ways to convince yourself (and others) that you’re making a logical choice when you’re responding out of fear.

So when you’re prioritizing, make sure you know which way you’re approaching it, and why.

Architecture vs. Engineering

Architecture must include engineering considerations, so that the design will be economically feasible; but the emphasis in architecture is upon the needs of the user, whereas in engineering the emphasis is upon the needs of the fabricator

-- Fred Brooks

How’s that for nuanced? Architecture emphasizes the user, engineering emphasizes the fabricator. Seems reasonable. Of course sometimes the “user” of the component is the next fabricator down the line. Or is the “user” the user of the system? They’ll certainly have different needs. And what about the person doing the maintenance/repair? Is that person a user? Their needs are certainly different. So what do you focus on? In true architect’s fashion, it depends.

Consider the late 70’s Pontiac Sunbird and it’s siblings. They sold cheap. They got you from point A to point B relatively economically. They were easy/cheap to manufacture. Designed to meet the users needs. Until you need to do a tune up. Which was typically once a year (every 12-15 thousand miles). It’s not that the parts were expensive, or hard to come by. But it was a little hard to change the spark plugs. In fact, to change the back spark plugs you needed to at least move the engine, if not remove it. Which upset the mechanic and the owner when they saw the bill for a tune up.

To me the mechanic is the user of the system. And this is a case where the system architecture missed the mark. Of course, the folks who designed the system weren’t the ones who needed to do the repairs, so they weren’t properly incentivized.

In our world, we’re the designers, builders, and mechanics. And in some (many?) cases the user as well. So we should be incentivized to make our systems not just easy to build, but easy to maintain and use as well. They should not only be easy to repair, but they should tell us when things are starting to go wrong. When we have time to fix them, Not right after the wheels fall off and it’s a big job to fix it.

So, in true architect’s fashion, it depends. It’s a balance. A balance of builders, users, and maintainers.

Missing Tests

Unit tests can do a lot of different things. Of course, they make sure things work the way you want them to at the beginning. They can provide documentation (and sample code) for your users. Your happy path tests are written the way the code should be used by customers, right? They can just look at the examples.

Unit tests are also great for debugging. If you’re going to be debugging you’re going to need a simple repro case so you can inspect the situation. One easy way to implement that repro case is with a unit test. After all, the fact that the bug got to users means that you missed an important unit test, so this is your chance to fill that gap.

Of course, now that you’re written the new unit test, your tests don’t pass anymore. So what’s a poor developer to do? You do the same thing you always do when a change in the code causes a test to fail. Just to be clear, that should not be to disable the test. No, you get out your favorite debugger and see where the problem is. Once you find the problem, you fix it. And make sure that your fix doesn’t cause any other tests to fail.

In case you were wondering, there’s a name for that style of development. It’s called Test Driven Development.

Finally, once you fix that bug, you’ll find that what you think is a bug is a feature to someone and they still want it to work the old way.

On Decisions

We make decisions all the time. Some with lots of degrees of freedom, some with only a few, or even one. Sometimes we make decisions with lots of empirical evidence and experience, other times, not so much. And sometimes, we choose to not choose, but as Neil Peart said,

If you choose not to decide, you still have made a choice

And assuming free will, choices have consequences. The question is, how do you know if you made a good decision or not? What do you measure to grade a decision?

At least, you should think about the information you had when you made the decision. How reliable was it? How much weight did you give it? Was the data biased? Were you biased in your interpretation of it? That doesn’t mean to exclude experience, history, or your “gut feel”, but it does mean that you need to be honest with yourself about how much you let intangibles influence your decision.

How much do you really need to decide? Do you need all the details, or is choosing a direction enough at this point? Often, the earlier you make a decision the less specific it needs to be. You can adjust to changes and new information. Other times, it’s a key decision that all other choices will guide off, so specificity is important.

Did you compare the cost (whatever that means) of implementing one decision or another against the potential benefits if things work out the way you want and if they go the other way? The less information and surety you have about the outcome the more important it is to think about not just the cost of implementing, but the cost of recovery.

Did you think about the impact of your decision on the decisions and plans of others? Does it work towards or against their goals? There’s no right answer, but a good decision takes those things into account.

In short, can you give a clear, logical, unbiased, and unambiguous explanation of why you choose what you did? If you did, it was probably a good decision. If you can’t go back and re-evaluate your decision and understand how you can make a better one.

And here’s an even more important point. Whether or not you made a good decision is orthogonal to the outcome of the decision.

Mutators and Accessors

That’s often how they’re taught, but really, they’re just another name for getters and setters. They’re a great way to encapsulate things. After all, you don’t want folks depending on the internal implementation of things. You want high coherence and low coupling, right? 

Or, are they a code smell, otherwise known as Accessors are Evil? Consider the lowly Rectangle class. Does a getter really give you isolation? Instead of declaring a public member Foo of type int you have a private foo and a public method Foo. Is that really any different? You’re exposing the internal structure with your getter. The only difference is some extra typing. Setters are just as bad. But what’s a poor developer to do?

The first thing to do is change how you think about it. Instead of mutators and accessors, think of the interface of your object as a set of commands and queries. You’re not changing the properties of the object, you’re telling it what to do itself, or asking it questions about itself. That’s isolation and encapsulation.

Sure, under the covers it might look the same. Ask the rectangle what it’s height is and you get back the height. Which might be stored as a member variable. Or it might not. But it doesn’t matter because you’re not asking the rectangle for internal data, you’re asking for it’s external property.

The same thing applies on the setter side. Don’t double the height and width bindly. Send the rectangle a command. Tell it to be twice as tall and twice as wide. Again, for a simple rectangle it might look like a setter, but it’s not. It’s an interaction with an object, A verb. A command.

It’s the difference between Object Oriented Programming and Programming with Classes. Superficially they’re the same thing. Especially in school. But when your objects get more complicated than rectangles, triangles, and circles it starts to matter.

So the next time you’re looking at the public API for your class/library/service/tool, think about how you’re thinking about it. Are you just putting some static sugar around the internal state, or are you presenting an API that drives users towards the domains and isolations your system is built on. And think about which one is going to be better for you and your users next week, next month, and next year.

Another 4 Questions

Not those 4 questions, but the 4 variations of one very specific question, Why are we building this?

Why are we building this?
Why are we building this?
Why are we building this?
Why are we building this?

Because until you answer the 4 variants, you haven’t answered the question. There are lots of things we can/should build. Assuming there are some form of constraints, we can’t build them all. So which one(s) should we build?

Start with the first variant, “Why are we building this?” What’s the value proposition? To the customer and the company. If it’s cool, but not something the customer wants (or a step on the path to get there), why?

“Why are we building this?” Are we the right team to build this? We might need it, but should we be building it? Does our team have the right knowledge/experience? Are there other teams or platforms that are a better fit?

“Why are we building this?” The classic build or buy question. Are we looking for a capability that is easy to buy on the open market, or is this part of the special sauce needed? There are costs both ways. Buying has a clear cost, and developing a solution has a cost that can be measured, but maybe not predicted. Even free open source software has a real operating cost.

“Why are we building this?” Should we be building something else first? And here, I mean first from a sequencing standpoint, not priority. If A depends on B, which depends on C, you should probably start with C, or at least as much of it as you know. If you build A first you’ll either need to change it significantly when you finish C or you’ll severely limit your options when it comes to C. Going the other direction lets you build on things, reacting to new information as it comes.

Which brings to Wardley Mapping. A way to use the answers to those questions, as applied to the various things you need to do, to help you figure out how to allocate your precious resources. The classic Wardley map is a simple 2D plot of linked components. The Y axis represents the customer/user value, which higher value being higher on the plot. The X axis represents the “maturity” of the component, with “No one’s ever tried this before” on the left, and commodities (cheap to buy exactly the right thing) on the right, and the ability to outsource it somewhere in the middle. The things at the top left are the most important to the customer and hardest to obtain. The things at the bottom right are mostly invisible to the user/customer and easy to buy.

Then, you link the components by direct dependency. What other things do you need to get the most important thing done? The further to the left the more likely it’s worth using internal resources for. The further to the right the more likely the right answer is to buy the solution.

Or, to put it simply, if you need a ½ inch hex bolt, buy one, don’t build a foundry. But if you need a fiberglass layup with compound curves around an assembly you’re still designing do it yourself.

Simple Fixes

So I’ll need you to replace the cat-3 cable from port AA-825 with a new cat-5 cable to desktop SPA-251. And I need it for a demo in 10 minutes. What do you do? And for context, here’s the current status

If you really need to do it in 10 minutes what you’re probably going to do is unplug the cable in port AA-825 and run a new cable around that lump and route it to SPA-251. If you get it done in 9 minutes, you might write down that port AA-825 is connected to SPA-251. If you’re really fast and get done in 8 minutes you might even do something about the cable you just unplugged. But you probably won’t have time.

Instead you get what happened to me last week. The house my daughter and her roommates live in has a laundry room on the main floor. It’s got a laundry chute from the 2nd floor, hot and cold water, a drain line, and 120V and 240V outlets ready to go. But a previous renter wanted more pantry space near the kitchen, so they moved the laundry room to the basement and put in shelves.

My daughter and her roommates decided they wanted to have the laundry back on the main floor, so I took the shelves out and hauled the washer and dryer back upstairs. Hooked up the washer and all was well. Brought the dryer up and plugged it in. No workee. No electricity. Checked the circuit breakers. All on. Checked the sub-panel. All on. But somewhere between the breaker panel and the laundry room the electricity disappeared.

It’s someone else’s property, so I call the electrician. $100 later he confirms that there’s no electricity and gives us an estimate. $3K to run a new line. Because there’s no way of knowing where the problem is, or if it’s fixable. My guess is that somewhere in the basement ceiling, hidden behind a nice new piece of drywall, is a splice that comes off the old dryer line and goes to the new one. But there’s no way of finding it without removing lots of drywall. In this case the fix is going to be to haul the washer/dryer back to the basement.

There were at least 2 better solutions when they moved the dryer to the basement. Put the splice in an accessible junction box or run a new line from the basement sub-panel. If either of those two solutions had been used I could have fixed things, and for minimal cost. No cost if they ran a new line, and probably $10 if they had the exposed junction box. Running a new line would probably have cost a few hundred more at the time, so not doing that makes sense. An exposed junction box might have cost $10 more. Even less if there’s an unexposed junction box behind the drywall.

But they took neither of those routes. Instead, I’m out $100 and it would cost $3K to fix the outlet. So now there are extra loose wires hanging inside the walls, and the dryer stays in the basement. I lost money and I lost options.

That’s the real cost of cost control.

Synergy

Synergy - 

the interaction of elements that when combined produce a total effect that is greater than the sum of the individual elements, contributions, etc.; synergism.

Or, put another way, emergent behavior. Put a bunch of simple things or rules together and see what happens. DNA is only 4 base pairs, and the combinations are limited, but somehow, when they combine in the right ways you get people. Or viruses. And everything in between. Logic gates, whether they’re vacuum tubes or solid state, are either open or closed, 0 or 1, but put them together the right way and you have a computer. Or a self driving truck.

Another classic example of emergent behavior is Conway’s Game Of Life. Three simple rules, but endless possibilities. From static images to glider guns. And to top it off, small changes to the rules or initial conditions produce radically different results.

It’s about how observable phenomena emerge from the well defined and understandable interactions of entities. It can be seen at all levels. From DNA base pairs to societies. At each step along the way entity interaction becomes phenomena, where are then the entities that interact at the next level.

Which leads to Conway’s Law. It’s the same Conway, and another example of emergence.

But things can go both ways. Software can look like the organization, but the way the software is built and put together can also drive the shape and culture of the organization. The strength (or openness) of the walls between parts of your code can drive the org. Simple, reliable, straightforward APIs encourage others to find new and better ways to use them. Ways that you aren’t thinking about when you build them, Ways that only become apparent when you look at them not as phenomena to be used, but as part of the set of entities and interactions at the next level.

Just something to think about as you build your APIs/interfaces. So that the synergy can emerge.