Recent Posts (page 54 / 65)

Naming Things

There are 2 hard problems in computer science, cache invalidation, naming things, and off-by-one errors. I'm going to talk about the middle one.

Names are important. At the simplest, they are identifiers that are (hopefully) easy to remember. But they are more than that. They set context. They inform metaphors. They can guide you in the right direction. Good names also respect and help manage scope.

Scope of influence is a good place to start with naming. Variables defined inside the scope of a method have that context, so you can use that to your advantage when choosing a name. Method names usually exist in the context of a class/package/namespace, so use that as well.

Consider the lowly row. If you call something a row what is it? If you do it inside the context of a database iterator then there's a good chance it's a row in a database table. On the other hand, if you're processing an image the row is probably a line of pixels. When working on a spreadsheet it's straightforward as well. In the read method in an vehcle_model class in a fleet management database library you don't need to call a variable `row_of_vehicle_models_for_fleet_management`, just call it `row`. On the other hand, if you're working on a translator from the database to some over the wire JSON then you might want to have `database_model_row` and `json_model_row` instead of `row1` and `row2`.

At a higher level, names become metaphors and help guide your thoughts in the right direction. For example, in a UI you might have check boxes and radio buttons. Visually they're pretty similar. When selected they have a mark, when deselected they don't. The metaphor helps because it lets you know how they're used. Pick as many checkboxes as you want. With a set group of radio buttons you can only have one selected at a time.

At an even higher level, how do you name your team/project/toolset? I worked on a windows installer builder called Wix. When it started it was Windows Installer eXtensions, and didn't do much. Over the years a whole ecosystem of tools has grown up around it and the names all relate to "wicks". There's candle, light, dark, smoke, and a host of others. All named around the candle metaphor. It helps to group logical things together, which reduces the cognitive load.

So, regardless of your scope, think about naming. It will make a difference.

https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-6.0/aa260976(v=vs.60)

https://www.mediawiki.org/wiki/Naming_things

https://hilton.org.uk/blog/why-naming-things-is-hard

https://news.ycombinator.com/item?id=13789817

Compatibility Mode

Have you ever wondered what Windows Compatibility Mode is for? It's because APIs are forever. As new versions of the Windows OS came out they needed to handle the old APIs, bugs and all. So when Microsoft fixed a bug in a later version of Windows and broke a lot of 3rd party software they added a compatibility mode that lets you rely on that bug. Kind of a neat hack, but not a situation you want to get yourself in to. Just ask Raymond Chen.

On the plus side, from a customer standpoint, that kind of dedication to keeping things working is great. Back in my Flight Simulator days we did the same thing. Almost every 3rd party addon that worked with FS98 worked with FSX untouched. Great customer experience. Our customers and our ecosystem loved us for it.

The FlightSim team, no sot much. About 25% of our testing was to make sure changes didn't break existing addons. Lots of version checks in the code to either add reasonable defaults or disable features for those addons. Was it worth it? The jury is still out on that one, but here's a couple of datapoints to help inform your decision. Combat Flight Simulator used the same physics and networking code as FlightSim but a different graphics engine. More detail on the ground, more ability to interact with things moving on the ground, and more booms. But it didn't sell nearly as well and was cancelled. A few years later, as FS costs mounted (and sales remained large, but flat) the Flight Simulator franchise was shut down. I can't tell you how much of the increased cost was back-compat, but it's a non-trivial amount.

On the negative side, maintaining all that backward compatibility was tech debt. Every release a portion of our effort that could have gone in to new user facing features and experiences, or unlocking new capabilities for 3rd party devs was dedicated up front to keeping the old things working. Our forward velocity went down. Features sat on the backlog from version to version. And sometimes we ran into things we just didn't understand, like that section of assembly code in our keyboard handler. We didn't touch it because every time we did something bad happened. Eventually someone took the time to really understand what it was doing, what side effects it had, and to recode it in C so we could change it instead of developing around it.

Flight Simulator is coming back this year. It's not going to have backward compatibility with FSX, but they're fully embracing the 3rd party community. It's going to be interesting to see how they approach compatibility mode

Blameless Post-Incident-Review

I've been doing the software thing for 30+ years, but I come out of the Mechanical/Aerospace Engineering world. Man rated, safety of flight discussions. Long product cycles. Lots of physical prototypes, bench test models, and iron birds, up to and including destructive testing to figure out exactly where the limits are. Then flying models and test pilots to figure out where the edge of the envelope is.

And still, sometimes things go wrong. Pilots find themselves outside the envelope. Often they can recover, but not always. What happens next though is what I want to talk about. Whether it's the formal processes of the NTSB, articles in AOPA or Aviation Week magazine, or a group of pilots sitting around the FOB, hands waving around and saying "There I was …", taking one out of the luck bucket and putting it in the experience bucket, there's a discussion about what happened, how they got into that situation, how they got out, and how to avoid it as well.

Similarly, the medical field has the Morbidity and Mortality meeting. The goals here are the same. Something bad happened. What can we learn from it? What can we do so that things go better next time, or, if possible, never happen again.

In the software world it's the blameless post incident review (PIR). Sub-optimal things happen. We see build breaks, service outages, downed websites, late releases, or anything else that didn't go as well as planned. We have runbooks for dealing with them (your team does have them, don't they) while they're occurring and incident management SOPs for keeping people informed during the incident. And right there at the end of the incident management SOP is the PIR. 

It's at least as important as resolving the incident. Do the PIR correctly and we can avoid not only that specific issue, but other similar issues. Whether you call it 5 Why's, Root Cause Analysis, or Continuous Learning, it's critical to understand not just what happened, but why it happened. What part of the system design/implementation allowed it to happen? Only by understanding why can you do something to keep it from happening again, not just in that specific case, but in cases just like it. Think of it like EMPOC for incident prevention.

There are a lot of things that go into a good blameless PIR, but I think the most important are:

• Make it blameless. Talk about what was done and what happened, not who did what
• Document what you're going to do (and when) to keep it from happening again
• Have an accurate timeline. You need to know what was cause and what was effect
• Know how you mitigated and then resolved the issue

Counting From Zero

Whole numbers, counting numbers, natural numbers. Which of those sets has the number 0 in it? You'd think mathematics would be precise, but naming things is hard. Depending on who you talk to, the natural numbers include 0, or they don't. It depends.

What about array indexes? They're zero based, right? Not exactly. Yes, the C family, Lisp, anything JVM based, ECMAScript, Go, and now Rust are 0 based. Other well known languages such as FORTRAN, COBOL, R, and Matlab are 1 based. But why?

The standard reason on the zero based side is that it's easier for the computer/compiler. Just multiple the size of an element by the index, add it to the base, and you have the address of the element. That makes sense. And when compilers were big and slow and there were no optimizers saving time and memory was important. And look at who uses those 0 based languages. If you're using one of those languages you're probably either writing frameworks/libraries for others to use or really concerned about performance.

Contrast that with who's using the 1 based languages. For many folks using that set the computer is a tool that does simple math quickly. They're concerned with lift and drag or modulus of elasticity or probability or transactional cash flow. And when they think of the first item in a list they're counting things, so the first thing (1st) goes by the number 1. They're optimizing for cognitive load in their own heads while they work on the problems they're trying to solve.

So who's right? As with everything else engineering, the answer is, it depends. Figure out what you're optimizing for, make a choice, and stick with it. Or, if you're Edsger W. Dijkstra, you could go back to how to represent a sequence of numbers and make a decision based on that.

APIs Are For Life

APIs do lots of things. One of the most important things is that they make the work you've been doing available to others. It doesn't matter if your API is HTTP, gRPC, C++, Python, a string of characters, a bunch of 1's and 0's stored in some kind of persistent store, or something else entirely. Without an API it's unusable, locked away where no-one can get at it and you've wasted your time.

If your API is your user interface, then approach it like one. The best user interfaces don't just make it easy to do what you want, they make it easy to understand how you should be using them, and they make it hard to do the wrong thing. If you're using a metaphor for part of your API, use it for the whole thing. Ensure your objects, whatever they are, are consistently used and named. Your API is your contract, so you have to live up to it, but that goes both ways. If the user makes a mistake, you don't have to guess what they meant, just make it obvious what the mistake was.

And that's the easy part. The hard part is future-proofing your API. Once you've released your API and your customers start using it, they're going to expect you to continue to uphold the contract, so how can you change it? You can extend it. You can make it do new things and handle different cases. You can version it. You can (and should) support at least the previous version of your API. The worst thing you can do is make it subtly different, so that it appears to continue to do the same thing, but really it doesn't. You might think it's a bug fix, but does your customer? If they built an entire workflow assuming that the way it works is the right thing and you "fix" it they're going to be upset. But that's a story for another day.

Random Is Hard

Not only are people bad at understanding random events, seeing patterns where they don't exist and not expecting things that they should, we (and the computers we use) are bad at generating random things. The best a computer can do is a Psuedo-random number and then calculate the next one from there. While people have done a good job of making sure the overall distribution of random numbers from these calculations are flat, we know, from the word calculation in that description, that given the same input you will get the same output. And that doesn't take into account the number of times that people try to roll their own. Unless you really know what you're doing and have a good reason to, don't do it. You'll just get it wrong.

Of course, in many cases reproducibility is a good thing. We do thousands of simulations a day, and while we expect variability, we need controlled variability so we can have reproducibility. On the other hand, If you're writing a black-jack game and someone can figure out the way you shuffle the deck and the order of the cards because they know the "seed" it doesn't matter how well distributed your random numbers are, they're not random.

Speaking of card games, just shuffling a deck is harder than you think. If you do it wrong then even if you have a perfectly flat distribution from your generator (or even a truly random generator) then you'll end up with a non-random distribution of cards. Kind of annoying if you're playing solitaire online, but imagine if you were a casino operator and someone figured out how to predict what the cards were going to do?

Random is hard and applications of random are even harder. Sometimes it really does matter how random your numbers are. So think about what you're doing and how you inject enough (but not too much) entropy into your system. Or, just build a wall of lava lamps and use that as the source.

The Paper Of Record

From today's everything old is new again files, Blockchains and The Gray Lady.

While the most widely known usage of blockchains today is crypto-currency, all blockchains really are is an immutable, distributed, verifiable transaction log that lets you verify that not only is something what it says it is, it hasn't changed from the original. 

Bitcoin has been around for almost 11 years, but what if I told you that almost 15 years earlier a company called Surety started one of (if not the) first public blockchains and used notices in the NYTs classified section as the public, distributed ledger for their hashes? Send them a document and they'd send you back the timestamped/hashed document and then they'd add it to their public hash which was (and still is) published in the NYTs every week.

While I'm pretty sure we don't need to publish the git sha's of our releases in the NYTs, wouldn't it be great if every bit of software we released (internally and externally) intrinsically had its git sha and creation date immutably embedded? You'd at least be able to get back to the code it was built from. Being able to build the identical (except for creation date) thing and then do testing on it would be great too, but hermetic, reproducible builds are a topic for a different day.

The Year In Review

Everyone else does these, so why shouldn't I? Here are some of my favorite postings from this year. Enjoy and see you all next year.

Worse is Better

Won't you be my Neighbor

Asking good questions

Platforms

One on One Meetings

Silos

Hello vs NoHello

PM is more than Provider of Meals

You are not paid to write code

Programming In The Dark

We all know about serverless programming in general, and AWS Lambda in particular, but have you ever heard of Dark?  Dark is a holistic programming language, editor, and backend all in one. It sounds pretty intriguing. Everything is boiled down to http endpoints, datastores, background workers and scheduled jobs. Everything else is handled by the backend. It autoscales. it's highly available. Deploys in 50ms. Built in version control. Built in feature flags. See live traffic in your IDE. Of course, to get those benefits you need to learn a new language and IDE, and you give up all knowledge and control of how and where things actually happen.

I'm not part of the closed Beta, so I have no idea how well it actually works, but there are a lot of good ideas there. It might not be the right answer for something that needs to support starting hundreds of new trips/second and hundreds of thousands of concurrent trips, but there are lots of use cases that don't require that kind of scale. Think about being able to focus on the problem you're trying to solve and not needing to worry about all those pesky things that distributed systems bring to the table like consistency issues, race conditions, latencies, and SPOFs.

The value isn't really in the things that you can do, but in the things you don't have to do. Imagine a world where you don't need to set up databases, servers, routers, repos, hosts, CI/CD pipelines, or test environments. Complexity and cognitive load slow us down, and easing that burden makes us more productive.

Dark isn't ready for PrimeTime yet, and may never scale to our needs, but reducing complexity and cycle time is a good thing. That's what those of us down here in the engine room are trying to do. Our value-add is making it easier for others to focus on their value-add.

Lessons From Ancient Rome

A few years ago when Uber did the first Harvard Business School classes one of them was called Leadership Lessons From Ancient Rome. The pre-readings were a bit dense, and since they were direct translations from the original latin they seemed a but harsh to our way of thinking. When the class was done this year the reading weren't as hard to get through, but the intent was the same.

The basic idea was to look at leadership as a continuum along 2 axes, strict adherence to rules/standards, and deep devotion to a person/idea. In the class the first axis was discussed around the rule of law. If a law applies to an individual then how much more does it apply to a leader? Things are very black and white. Either the letter of the law was followed, or it wasn't. The second axis was about following a person or idea, regardless of the cost to you or others. If you have too little of both you have everyone doing what seems best for themselves. Absolute adherence to the letter of the law is absolute severity, prisoner 24601 goes to jail for taking a loaf of bread to feed a starving family, while blindly following a person or idea leads to demagoguery. It is by balancing adherence to standards and devotion that justice emerges.

The question was, how did that relate to being a good leader, and how do you create an environment where people can be free to do the right thing, while doing their best work. How do you create a culture with sufficient guidance/direction without stifling people and creativity? What does a healthy organization look like?

While there are lots of details in the linked articles, I think it comes down to a few things. Having clear, well defined goals and ensuring that everyone really understands what the goals and priorities are. Defining enough process and procedure so that information is shared and the left hand knows what the right is doing. Clarifying ownership and responsibilities, and then helping to resolve conflicts when there are disagreements over them. Finally, making sure that meeting the goals and priorities are what takes precedence, not who is doing it.

And if you're interested in the original readings about standards and severity let me know and I'll share them. Things were different back then.