Zero Bugs

Back when I worked on boxed products at Microsoft, we had 2-year release cycles. And towards the end of each one was a milestone called Feature Complete. That was the point in the project where all features we expected when we did planning 18 months earlier were done. Or at least the ones that we hadn’t decided to cut because we ran out of time. You would think that after feature complete, we’d be ready to ship. But that wasn’t the case.

Instead, the next big milestone was Zero Bug Bounce (ZBB). That was the second time in the history of the project that there were zero active bugs in our tracking system. The first was before we wrote any code. After that, the number of bugs climbed until shortly after Feature Complete. For ¾ of the project or more, the incoming bug rate was higher than the fix rate.

That wasn’t just our project. That was the way most software was written. You built it, then you tried to test quality in. It worked, after a fashion, but let’s not fool ourselves. It wasn’t very efficient, and it wasn’t a lot of fun. From the beginning of the project until some time after feature complete the backlog of work kept getting bigger.

At the same time, the early 2000’s, extreme programming and the agile movement were getting started. Borrowing some concepts from lean manufacturing, and the idea of building quality in instead of testing it in.

One of the ways that expressed itself was the idea of a Zero Bug Policy (ZBP). The idea that your software should have 0 bugs. At the time, most folks looked at that and said it was impossible. Of course, there were already examples of bug free software, but people still thought it was impossible to write bug free code.

And those folks are right. Even with Test Driven Development (TDD), and a full suite of unit, integration, and system tests, you can’t guarantee bug-free software. But that’s not what a ZBP is about. It’s not that you never make a mistake, or a bug never gets shipped to a customer. Instead, a ZBP is really about not having a bug tracking system.

While ZBB and ZBP have a Levenshtein distance of only 4¹, they’re completely different things. A ZBP means that instead of keeping track of your bugs and fixing them later, when you’re not so busy adding more bugs, you fix them now, for some reasonable value of now. You don’t drop everything and fix it², but as soon as you finish what you’re working on you fix the problem before you start something new. That means that every day is potentially a ZBB.

That’s a very different way to build software. It’s hard to do. You need to build the muscles for TDD and unit test. You need to build the muscle to say “No” when schedule pressure pushes you to move on to the next feature even though there are still issues with the current task. You need to build the deployment muscle so it’s easy to make the fix. All of these things and more are hard to do, and don’t show any immediate benefit³. It takes disciple and commitment.

Another benefit of ZBP is that you’re always ready to ship. You might not have the feature set you originally planned, and it might not be as pretty as you might have made things, but if you need to do a demo, you can demo everything you’ve done. If something happens and the release date moves forward, you have something to release. You can sleep at night and not have to worry about having the rug pulled out from under you.

Remember, even if you’re living in a ZBB world, you don’t have to stay there. You can bias your choice of work slightly so that your rate of finding issues is lower than the rate at which you fix them. Even if this doesn’t get to you ZBB before feature complete, the wall you hit at feature complete will be shorter.

And finally, you need to differentiate between planned features, feature requests, learning more about the domain you’re operating in, and software bugs. The first two have nothing to do with a ZBP. You can have as many of them as you see fit, and you can track them however you want. They key is that they are NOT bugs. That’s just future work you need to do.

New learning about the domain might or might not be a bug. Learning there’s a better way to do something, or an abstraction you should be using is not a bug. Finding your domain model doesn’t match the system you’re trying to model IS a bug and needs to be fixed ASAP.

Simple coding errors are also bugs. First, write a test that fails because of the bug. Then fix the code so that test, and all other existing tests, pass. Again, don’t add those issues to a long-term tracker and wait to fix those issues. Just fix them now.