Testing And Failure

"Testing leads to failure, and failure leads to understanding." - Burt Rutan

I’m an airplane guy. As such, I’ve been aware of Burt Rutan for 40+ years. I even once helped a friend build a VariEze. Way back before there was a Mark Zuckerberg, let alone Facebook, Rutan was experimenting and failing forward. Lots of iterative development. On paper, in wind tunnels, in the air, and in space.

As impressive as SpaceShipOne was, I think Voyager was even more impressive. 9 days in flight. 26K miles. Mostly one engine. All on a single tank of gas. That’s an endurance record.

To get there took a lot of understanding of the systems involved. Of course the aircraft itself, but not just that. Also the engines. The weather. The support systems. The pilots. The question is, how do you get that understanding? You get it from testing. And fixing the problems. But not just fixing the specific problem, but deeply understanding what the problem was and making sure there weren’t similar problems later.

You can (should) approach software development the same way. You know what you want to happen when everything goes right. And you write things so you get the result you want. And you do your best to consider other possible situations. And make sure that they are at least handled gracefully, even if you can’t do what you originally planned.

Then you start testing, You find the edge of the envelope. More load, more traffic, less time to respond, strange data, whatever your inputs and environments are. And something breaks. So of course, you fix it. But you don’t just fix it. You figure out why you got into that situation and how to make sure you never end up there again. Which means not just understanding the thing you’re writing, but the environment it operates in. And the people and systems upstream that contribute to the environment. And the people and systems downstream who’s environment you are a part of.

Because without that understanding all you’re doing is going fast and breaking things. You’re not actually getting anything done