We test to find bugs, or at least make sure there aren’t any, right? Wrong. Or at least it’s not just about finding bugs. Finding/eliminating bugs is one of the specific things we do as part of testing, but that’s not the goal. It’s not about lines or branches covered. It’s not about having tests pass or dogfooding. All of those are things we should do along the way, but they’re not the reason we test.
The purpose of testing is to increase confidence for stakeholders through evidence.
That’s why the things we do, fallbacks, guardrails, formal analysis, input validations, code reviews, unit tests, code/branch coverage, integration tests, dogfooding, etc., are important. These things give us concrete numbers that describe things we know about the code and the system. They are how we provide the evidence needed.
That data lets us show how we have reduced risk in different ways. They’re not foolproof though. We don’t test all possible combinations or sequences of inputs. Even tools like TLA+ can’t guarantee correctness. All we can do, especially in a complex system is increase our confidence (by reducing risk). Risk analysis is a complex topic itself for another time, but for simplicity, risk and confidence are inversely related. Reducing risk increases confidence.
Just understanding that isn’t enough for the people designing/implementing/running the tests. You have to understand who the stakeholders are and what’s important to them. Stakeholders are everyone impacted by you building the thing under test. Not just the users, direct and indirect, but you as the builder, your team, your manager, and your company. Each of them have different views, different requirements, and different levels of acceptable risk. Depending on who and what you’re doing you could be in many different stakeholder groups. You may also be completely separated from one or more of them. You still need to understand them and what is risky to them.
Because at the end of the day, we’re building things to add value for some group of people. We test to get the confidence that what we’re building will do what it’s supposed to an acceptable level for those people. That’s why we test.
In part 1 I talked about how your “level” or title was just a way to describe the scope of influence you had. Just a measure of how much your work, and how you worked, impacted others. Part 2 explained more about what I meant by “scope”. What the parts of scope were and how scope could grow.
Being able to describe how that works is crucial to managing your career. Just as important is being able to describe it to others and to visualize it. That lets you discuss it with others. Because while you’re responsible for your own career, you aren’t alone in a vacuum. Being able to visualize where you are in your career and your progression through your career lets you understand where you think your gaps and opportunities are and compare them to what those around you think.
To that end, I’ve been playing around with a bunch of different visualization formats. I wanted to show 3 different values for each of 6 different areas in one place. I tried text, line graphs and bar graphs, but none of them really captured what I wanted. Eventually I settled on a radar plot with an overlay and accompanying text.
The radar plot breaks the circle down into the 6 “competency” areas. The different rings represent the scope of influence for each level (self, sub-team, team, project, group, or division). Then, for each section of the circle the colored arc represents what you (or whomever is creating the plot) considers your scope of influence. Finally, you can overlay a circle on the plot with your current level. The text area gives you a place to describe significant information/events that led you to pick that particular scope of influence for that area. I’ve put together a progression of radar plots that lay out expectations in each area at each scope.
Where this gets really interesting and useful is when you use it to manage your career. Pencil in where you are on the radar plot. Then get your manager to do the same. Once you have both, compare them to expectations. If both you and your manager agree you’re exceeding expectations in all areas then you have an easy discussion about next steps. If you and your manager agree and there’s a gap between where you are and expectations then the conversation is still pretty easy. Identify the gaps and discuss what you can do more/less of or new areas to move into to close that gap.
If you and your manger disagree on where you are then you have a harder conversation, but it’s the one that’s probably most important. Making sure the two of you are on the same page is the single most important thing to managing your career. Those gaps indicate places where the two of you aren’t on the same page, so that’s what you need to be talking about. It might be expectations. It might be visibility. It might just be lack of awareness. Whatever it is, now that you know about it you can (and should) do something about it.
Finally, one last things you can do with these visualizations is look at them over time. I put one together for my career. It’s a lot of things, but one thing it’s not is smooth. And that makes sense. Over time, particularly around job changes, you perform at different levels. Don’t be surprised when it happens. In fact expect it. Take the time to look around, identify the gaps, then close them and move on. That’s owning your career.
If you’re really interested, you can see my history (and a couple of “generic” career progressions I’m putting together for discussion purposes) in the Eng Stories area.
Bias for Action is one of Amazon’s Leadership Principles. Bias for Reaction is not. They sound very similar. And from the outside they look very similar. But from the inside they’re very different.
Bias for action is about making decisions. It’s about being thoughtful and using the information you have at hand to make a good decision. It’s about understanding the options and then deciding amongst them.
Bias for reaction, on the other hand, is about doing something, anything, instead of taking the time to think and make a decision. It’s about motion. It’s about not appearing idle. It’s about looking like you’re invested in the situation.
Bias for Action is a good thing. Bias for Reaction, not so much. The key is to understand that difference and then act on it.
As part of Amazon’s leadership principles, Bias for Action is described as
Speed matters in business. Many decisions and actions are reversible and do not need extensive study. We value calculated risk taking.
Or, in other words, when faced with an reversible decision, use the information you have and make the best decision you can. Then move forward. If you need to change it later, change it then. Don’t agonize over the decision and don’t let yourself end up with “Analysis Paralysis”.
At Amazon the key to making decisions at speed is understanding whether the decision you’re making is reversible or not. Whether it’s a one-way door or a two-way door.
A two-way door is decision that is simple to make and if you choose to, simple to unmake. Realize you’ve stepped through a door into the wrong conference room? Step back through the door and find the right room. The cost of getting through the door either way is about the same and pretty cheap.
A one-way door is like the emergency exit at the bottom of the staircase. The one that says “No Re-entry To Building” and when you get outside there’s no door handle. If that door closes you can’t just step back in. You have to expend a lot more energy (pounding on the door and hoping someone responds or walking around to the main entrance) to get back in.
Bias for action means first understanding if you’re about to go through a one-way or two-way door. Before you can make a decision on what to do, you have to decide what kind of decision you’re making. Then you make your decision. With the appropriate amount of research and study.
That’s Bias for Action. Not doing that is Bias for Reaction. And that can get you in a lot of trouble.
As a <persona>, I should be able to <task to complete>
That’s a decent format, or at least it’s a minimum bar. Without identifying the kind of user and the task to complete it’s really hard to compare tasks to identify which has the highest value. However, it is by no means a guarantee.
There are 2 really common reasons that this template doesn’t guarantee that you end up with a quality user story that you can implement. The first is that there’s no mechanism to ensure that the task the user is trying to complete is actually a task that adds business value and not just an activity along the way to completing a task that adds value.
The canonical example of this is the login user story. You know, the one that reads something like
As a return website customer I should be able to login to the website.
That story hits all of the requirements of a user story. The customer is clear. The task is clear. Acceptance criteria, while not explicitly listed, is clear. Even with all that, it’s not a good user story. It’s not a good user story because the task, logging in, doesn’t add any value. It’s an activity that needs to be done, so by all means, do it. But if all you did was add the ability to log in to the site then you wouldn’t be adding value. You’d also be making your customer’s life worse. They would need to jump through some hoops, but things wouldn’t be any better.
A better user story, with a task that adds value, would be something like
As a return website customer I should be able to order the contents of my shopping cart using my securely stored personal information.
Do you need to be able to log in? Maybe. You need to access securely stored information. Logging in is one way to do that. But there are other mechanisms for authentication. The user story is not the place to specify the mechanism. The team should figure out the mechanism by looking at the situation. And once they have the mechanism determined/implemented, they can then use it to ensure that secure personal information is used. The customer doesn’t need to enter information every time and you don’t need to send the info over the wire making things even more secure. And that adds user value. When the story is done the user is measurably happier.
The other common reason, and the one I see more often, goes the other way. The story is too broad.
As a return website customer I should be able to buy something
That is certainly something a customer wants to do, and doing it adds value, but where does the story start and end? That story covers just about every e-commerce site ever made. To really be useful and completable in a reasonable amount of time it needs to be split up into many more much smaller stories or scoped down to what the story really means. That story is probably a conflation of at least stories for browsing, adding items to a cart, setting delivery address, and making payment. I needs to be split up into those stories. The challenge there is to not split the story up so far that it turns into a task list (see above).
Right-sizing stories is the key to making useful user stories. Making them big enough to actually add value, but not so big that they can never be completed. Tim Ottinger has a good list of resources for splitting stories.
In Part 1 I described how your level is a proxy for scope of influence. Knowing that gives you a framework to understand where you think you are. What it doesn’t give you a way to break scope of influence down into a series of areas that you can measure.
One way to do that is break down scope into categories. Different companies call them different things. Some call them competencies, proficiencies, or areas, others might call the capabilities or skills, It doesn’t really matter what they’re called, as long as everyone is talking about the same thing. In my experience they’ll usually map to different competencies that cover these 5 areas.
Competency Breakdown
Citizenship
Citizenship is how you support the company outside of the company’s lines of business. Things like being an active interviewer, onsite and offsite, presenting on behalf of the company at a conference, or contributing to an open-source project. It’s being part of or leading an internal employee group or donation project.
Software Engineering
Software Engineering is the one of the core competencies for the engineering role. It’s about writing code and use/promoting/defining best practices. It’s building something that works now and is scalable enough to work for the future. For other roles, such as PM it would be about project management and finding customer value.
Architecture
Architecture is the other one of the core engineering competencies. Architecture is about interfaces and designs and ensuring things work when then need to can change to handle new requirements. It’s about deciding what to build, what to buy, and what to stop doing. For other roles, like PM, it will be related to the role. For them it might be about what to features to build or not build.
Execution
Execution is getting things done. Getting them done on time. Regardless of whether it’s a current, short-term project, a multi-year project or strategy, spans multiple people/teams/groups or even companies.
Collaboration
Collaboration is a measurement of how well you work with others. The people working on the same functions, the people on your team or a larger organization or in out outside of your industry.
Efficiency
Efficiency is about doing more with less. Using fewer resources, people or compute. Finding and eliminating duplications. Sharing the shareable. Automating the automatable.
Once you know what those competencies are, you need a way to measure you scope in those areas. That breaks down into two areas, time and impacted people. The breakdown for the different areas looks something like this
Time
Immediate/Right Now
~ 6 months
~ 1 - 2 years
3 - 5 Years
5+ years
Impacted People
Yourself
A feature team (2 - 3 people)
A team
An org/division
Your company / industry
Combined, the competencies and the scope areas give you something you can mostly measure. What you DO with those measurements and how you think about them will be covered in part 3.
Brady is all about labeling for the workplace. They’re primarily aimed at the manufacturing/processing space. They’ve got a product line directed at lean manufacturing techniques, such as those that come out of the Toyota Production System (TPS), which I’ve talked about before. Since a lot of today’s agile and lean software development processes are derived from TPS and lean manufacturing some of them make a lot of sense in the software world as well.
We don’t have machines or storage racks that you could put that sticker on, but putting it somewhere prominent, like on your laptop or next to your monitor, so you don’t forget about it might make sense.
But what does it really mean for software? After all, no matter how many times you run an executable or iterate over a loop the ones don’t wear down to .6 or .7 and the zeros don’t start to fill in and turn into .3 or .4? The code, barring disk errors and cosmic rays, doesn’t change.
There are a few things that maintenance implies with software. Unless you’re running on an embedded system and there is no operating system you’re relying on third party code. And even if your code is perfect there are going to be changes and updates to someone else’s code you want to use. It could be to pick up new features or better performance, or it could be to close a security loophole, like the log4j issue. So there’s always a reason for maintenance.
Even without those reasons for maintenance, you probably want to do something to the code. Add a feature. Respond to new business requirements. Reduce support burden. All of those things are maintenance. And sometimes, to enable that kind of maintenance, you need to do internal work. To keep the external software quality (ESQ) high, you need to work on the internal software quality (see ISQ vs ESQ).
That’s where scheduled maintenance comes in. Doing the extra work so you can do the required work. The unseen work that lets you keep your velocity up. The unseen work that lets you respond to changes in the environment quickly. The unseen work that lets you add value.
Which means there is value in that unseen work. So don’t skimp on it.
I’ve often (usually? almost always?) found it easier to write code than it is to debug it. Even code I recently wrote. If nothing else, when I’m writing code I have all the context in my head and I know how all of the API calls are going to respond. Even if I’m wrong, I know what I’m expecting and I write the code accordingly. Of course, that’s not always the case, which leads to debugging.
Some days the debugging is easy. I can look at the incorrect result, error message, or stack trace and know what I need to do to fix the problem. Other times there’s enough info to point me at a section of the code and I can trace it through and find the issue. I can try different inputs and write special debugging code to help me understand what’s going on.
Other days though it’s a struggle. The error messages don’t help. Code inspection gets you nowhere. Debug code doesn’t even help. No matter what you do the error keeps happening and you get no extra information. This happens most often with 3rd party libraries and web calls because all you’ve got is the documentation. So you try changing something as a way to map the boundaries of the problem.
And nothing changes. Google searches. Stack overflow. The expert on the team. Nothing helps. So you go back to first principles and build a walking skeleton that has just enough meat on the bones to work. Then you keep adding to it until it breaks the way the thing your debugging breaks. Or conversely, you start taking things out of the broken thing until something changes.
Then you can really start debugging. You’re back in control of the system. Or at least having some kind of influence over it. Because without being able to influence the system you can’t characterize the problem or drive to a solution.
There have been countless times this has happened to me. I’ve worked for hours trying to vary the inputs subtly so that I can understand how the system is actually responding. Trying to get to the point where the error is in code I have influence over. So that I can fix it.
But as annoying as that is, it reminds me how much better things are now. Where I can click a button or run a single command and the system makes sure that I’m exercising the code that I’m working on. Not like things used to be. Before Makefiles having nothing happen was common. To actually see your change was a multi-step process. Change the code in some editor. Save the code. Compile the code. Link the code (at however many levels are needed). Run the new executable. Miss any one of those steps and nothing changes.
Time passes. Even if nothing else changes, time passes. That’s the same for everyone and you have no control over it. What you do have control over is what you do with your time.
How you use your time has a huge impact on your impact. At one of the highest levels there’s work-life balance and how getting that wrong leads to burnout and having no impact. One level deeper on the work side is managing work in progress (WIP). Just being busy is NOT the way to have the most impact.
One way to make sure your time has impact is to use it to prepare for the future. To be learning ideas and new approaches. Taking the time to explore and add a new tool to your toolbox or be able to use your existing tools better.
Which brings us to the most tactical of levels. How you learn. Learning is different for everyone. What’s best for one person is not going be be best for another. Some like to read in silence. Some like demonstrations and examples. Others, like me, do best with hands-on experience in a group. Knowing what works for you and taking that into account is important.
One thing that seems to be important for everyone though is down time. You can’t keep stuffing knowledge into your head for 8 hours straight and expect it all to stick. Periods of intense learning/studying/training followed by periods of “idle” time so that you can process those learnings and make them stick.
I have no idea if this is actually the mechanism, but the way I think of it is that there’s an inbound queue of knowledge. As you’re learning/experiencing things get pushed onto that queue. At the other end of the queue is a process that takes those things, stores them somewhere, and associates them with everything else you already have stored. As long as the incoming rate is below the acceptance rate of the queue everything is fine. If the incoming rate is too high things get dropped on the floor, never to be seen again.
At least that things are OK until the queue fills up. When the queue is full the acceptance rate drops to the processing rate at the other end and everything falls to the floor. There are lots of things you can do at that point. You can (to a certain extent) make the queue bigger or (somewhat) turn up the processing rate. You can make the queue deeper by recording the input and then replaying it later when the queue is empty. You might even be able to slow down the incoming rate. Good teachers, knowing about the limitations, will do that for you.
The other thing you can do, especially at work, is take a break. Not just a break from learning, but a break from concentrating. Doing something that doesn’t use the same part of the brain as the thing you’re learning, but keeps the body busy. The classic example of that kind of break is thinking in shower, but that’s not practical during the work day. Which is why you’ll sometimes find me walking in circles around the office. Sometimes just listening to music, sometimes playing solitaire, sometimes just walking. It gives the processing end of the queue time to catch up and drain the queue. So you can keep learning. Which is what you wanted to be doing with your time in the first place.
I’ve been doing this for a while. And like the Farmers Insurance guys says, I know a thing or two because I’ve seen a thing or two. I’ve written a little about it before, but I figured it’s time to add a little more detail to how I think about things and what I’ve learned. About what levels are and aren’t, and what they’re a proxy for (hint: it’s influence, not ability).
Some background. First, this is based on what I’ve seen and experienced at small startups (you’ve never heard of them), game dev companies (Microprose), mature, large tech companies (Microsoft, Amazon), and hypergrowth startups (Uber), and mid-sized startups (Aurora) as well as discussions I’ve had with folks at other tech companies, large and small.
Second, your mileage may vary. While many companies have standardized processes, your experience is very much dependent on your direct manager and the managers 2-3 levels up. At larger companies the experiences in one area can be very different from the experiences in another part of the company.
Third, the environment now is different than it was when I started. Pandemics. Work from home. The great resignation/reshuffling. Today’s cell phones have more compute/graphics power than any single computer (or cluster) you could buy.
With all that said though, at the core, things haven’t changed that much. Your job, and what makes a difference, is the business value you provide. The business problems you solve. Not the code itself. Which means the way to look at it hasn’t changed much either.
Let’s start with some basic facts about levels
They’re just numbers. Typically about 8 of them. They might start at 1, they might start at 59. But the important part is that they’re just numbers.
They do
Set expectations
Usually based on some written rubric, but it might be implied (especially at smaller companies)
How others see you. Coworkers expect more from a senior engineer than from a new hire
Often defines your title.
Often inform compensation. Typically there’s a compensation band for each level and there is overlap
They don’t
Define what you do. People at a given level can work on lots of different things.
Define your role. At a given level you could be a software engineer, a hardware engineer, a PM, or a manager
Define how you do things. Different people do things in different ways. As your level and role changes, the way you do things changes. What and how much design, implementation, coordination, problem solving, and problem identification you do are not defined by your level.
They are (generally) given in recognition, not aspirationally. You get the level after you’ve been consistently demonstrating that you are working at that level. Not as a reward for working hard or to help you reach the level.
You’ll notice that since level doesn’t define what you do or how you do it there are people with wildly varying roles at the same level. Which means there needs to be a way to compare people at a given level. To keep things simple I’ll focus on individual contributors (IC) and managers in software engineering, but the same idea applies when comparing PMs, designers, engineers, etc.
In software engineering, breaking things down into really broad strokes, there are two major categories:
Technical Impact
What things you know about
How much you know about them
Defining/Selecting Work Product
People Impact
Building/Maintaining Relationships
Building/Growing People/Teams
Defining/Selecting Work Product
For your typical IC, plotting those two areas vs. Level will give you something like these two charts. What this shows is that as your level increases (career growth) the typical IC will continue to grow technically, but the people management side levels off. I’ve left off years of experience (YoE) and any units on the vertical scale on purpose. It takes as a long as it takes, regardless of YoE. This is a unitless Y axis because it’s relative, not absolute.
For your typical manager though it looks a little different. They start the same since lower levels managers are typically ICs and follow the same growth. After that they differ though, almost reversed. The technical breadth/depth stays the same, but the people side grows.
Once you’ve got those graphs it’s easy to compare them, right? Just overlay them and you know. Or not. What you end up with is a graph of diverging lines that doesn’t really help at all. Like this.
But when you think about it a little deeper, you realize that there is one thing that grows with level, regardless of your what you do or how you do it. That one thing is Scope Of Influence.
It’s Scope of Influence that grows with level and how you can compare the impact of people at a given level, regardless of role or title. As a new IC engineer, whether fresh out of school or self-taught, your scope is yourself. You mostly implement things. A little internal design, but how the thing you’re building interacts with other components of the system is defined for you. You don’t have a lot of impact on how other people do their work. As you grow and your level increases that changes. First it’s another person or two on your team working on a feature and how it will be used. Then it might be how the whole team works to solve a specific business problem. As you get to “senior” levels you’re influencing how things are done across multiple teams. At the “staff+ (IC)/Director (Mgr)” levels your influence is across organizations or business units or products, depending on how things are organized where you are. It doesn’t matter if you’re an IC or a manger.
Understanding how levels and scope of influence relate to each other is critical to deciding on your path and managing your career. But it’s just the framework for doing it. More on the deciding and doing are topics for future posts.
Speed kills. Measure twice, cut once. Rework Avoidance Theory. We’re trained from an early age to slow down when faced with uncertainty. To be cautious. To not drive faster than we can see. And often it’s true. Especially when operating heavy equipment. Simple physics tells us that mass * velocity equals inertia. And as Newton said, an object in motion stays in motion. Changing directions or stopping means working against that inertia. So if you want to be agile and be able to change your direction easily then you need to slow down.
So what do we do in software development to make sure we don’t go too fast and make it hard to change direction? A common response is to add process. Long manual processes to ensure that we keep going in the correct direction. That we’re not wasting our time. That nothing bad happens. Cascading gates with lots of places to stop and roll back. That makes sure we’re going slow for safety. Right?
That’s one option for sure, but it’s not the only one. And in many cases, with software it’s not the best option either. Slowing down reduces inertia for sure, but it’s not the only way. Consider the other half of the equation. Reducing the velocity by 50% reduces inertia by 50%. But the same thing happens if you reduce mass. It’s a linear equation, so you can have just as much impact by changing mass.
In development the analog to mass is the size of the change. Smaller changes are less “massive”, so for a given velocity they have less inertia. They have less coupling. They have less surface area. They have less drag. They take less time. They’re easier to undo. So you can make more changes.
There’s another thing to keep in mind. When you slow down changes tend to accumulate, so the “mass” goes up. In fact, it’s not unusual for the mass to go up so much that going slower means there’s even more inertia and it’s even harder to make changes. Which is exactly the opposite of what you want.
