Recent Posts (page 2 / 65)

by Leon Rosenshein

Time Periods

workchronicles.com comic showing a manager translating estimates, roughly doubling the estimates, and still being wrong.

From Star Trek to Hofstadter’s Law, estimates are always a conundrum. There are lots of articles about it on the web, but here’s Ron Jeffries about it and it’s a really good place to start.

To me, the problem is not the estimates themselves, or the idea that you’re trying to measure something that is neither precisely or accurately measurable, but in what happens to after you make the estimate. Or to put it more bluntly, the problem is that very often inaccurate, imprecise estimates are taken as promises. When that happens, everyone is going to end up with some level of unhappiness.

If you’ve been reading these entries for any length of time, it’s probably obvious that I think it’s important to add value as soon as possible. And that you need to add the most value you can. The question is, how do you know what the most valuable thing to do is.

In fact, it’s not just the most valuable, it’s adding the most value, for some set of resources, over a certain amount of time, and knowing that for a given set of resources, every choice to do something is also a choice to NOT do a bunch of other things. That’s a good thing. Focus is good, and tends to get you to value sooner.

However, to maximize, or even go in the direction of maximization, you need to compare resources spent on all of the possible options. And since you need to do it before you’ve done most of the possible things, you can’t know how many resources it will take. Instead, you have to estimate it. Then you can do comparisons and make choices.

Sometimes, there really are dates you need to hit. My go-to example for that is delivering games for the holidays. Even today, with online games, internet delivery, and mobile gaming, in the US, if your product isn’t available by Thanksgiving, you’re going to lose out on a lot of sales. You NEED to work within that constraint. You can go a long way by sequencing work and doing the things that need to be done first, first, and leaving the rest for later.

But what if you’re making tradeoffs? Thing A is critical, things B, C, and D are nice to have and individually that’s the importance order. But together, C and D are more important than B and C. Should you do C and D first then? Only if you believe that you can get them both done. And to know that, you need an estimate.

Which brings us to my recommendation(s). Never give a precise time estimate. Instead, the precision of your estimate should be a function of the time period estimated. Something like no more accurate than ½ the time period. If you think the right unit of time for your estimate is days, the accuracy of your estimate should be +/- ½ day. If your estimate is 3 years, then it’s 3 years, +/- 6 months. And, reserve the right to change the estimate AND the precision as you learn more about the problem.

That’s not what the #noestimates zealots want, but it is as little estimation as you can get away with.

by Leon Rosenshein

Doing It Well And Truly

I’ve always been a reader of science fiction. In particular, the kind of science fiction where knowledge and competence make a difference. Stories like Andy Weir’s The Martian. Or Lucifer’s Hammer. There’s nothing magical or particularly special about the protagonists, they’re just competent. And lucky. And most importantly, they do the thing. They do it well and truly. But most importantly, they do it. The don’t (only) talk about doing it. They don’t (only) talk about how important it is that it get done. They don’t (only) wish someone else do it for them. They don’t (only) complain about who’s not doing it. They just roll up their sleeves and do it.

Most of Lucifer’s Hammer is about how different groups of people prepare for, survive, then move forward after a comet hits the Earth. They don’t complain about it (too much). They don’t wait for others to do something for them. They plan, they execute, they adapt, and they move forward. Lots of folks have read the book, and different people remember different parts of it. But almost everyone remembers Gil, the surfer. His choice on that day was to do what he loved, which was surfing. So he was out on the ocean off the shore of Santa Monica when the comet hit. When that happened, instead of giving up, he did what he did best. He decided to surf the tsunami. The last thing we get in the book is Gil thinking about the story he could tell if he survived.

Quote from Lucifer’s Hammer: Oh, God, if he lived to tell this tale, what a movie it would make! Bigger than The Endless Summer, bigger that The Towering Inferno: a surfing movie with ten million in special effects! If only his legs would hold! He already had a world record, he must be at least a mile inland, no one had ever ridden a wave for a mile! But the frothing, purling peak was miles overhead and the Barrington Apartments, thirty stories tall, was coming at him like a flyswatter.

Sure, Gil was thinking about going out in style at first, but he realized that Doing it well and truly might actually get him out of it alive. In a an extra last chapter that was published in Analog almost 40 years after the book, it turns out, he did survive. Once again we see that doing the thing, instead of just talking about it, makes a difference.

As in art (and life), so in development and career. If you see something you think needs to be done, but no one is doing it, think about it and figure out why. It might be like Chesterton’s Fence. There really is a good reason, it’s just not obvious to you. It might be because of priority. Or, it might be that it just hasn’t been important enough to anyone. That’s where doing the thing becomes important.

If it’s important to you, you have (or can find) the time, and there’s not a good reason not to do it, then do it. You’ll be happier. Chances are others around you will be happier. And people will see you as someone who gets things done. Any of those is a good enough reason. Together they’re a great set of reasons.

There are lots of things that sound like doing the thing, but they’re really things that aren’t doing the thing. They don’t get things done. They don’t make you happier. They don’t make others happier. They make you look like a complainer or a procrastinator instead of a doer. The key thing here is to keep the difference in mind. It might be fun to do things that aren’t doing the thing. It’s probably easier to do those other things than it is to do the thing. You might even get more support that way. However, you won’t get the thing done. And that’s the most important thing to remember. If you’re trying to get a thing done, you have to do the thing.

So the next time you find yourself in a situation where you have a choice between doing the thing and not doing the thing, think about what you want, what’s important, and if it’s your choice. Then, do the thing, and do it well and truly.

by Leon Rosenshein

A look at 2023

snowman with falling snow animation
Happy New Year! Time for my yearly retrospective.

Big changes on the work front. I started the year working at Amazon on their Just Walk Out technology, but then got moved to a new initiative. The work was interesting, and from what I hear, going well, but Amazon and that project weren’t a good fit for me, so after a year at Amazon, I started looking around for something different. What I found wasn’t Amazon, but it wasn’t that different for me. I’m back at Aurora, now working on the Developer Experience team, focusing on software quality, bothg internal and external. I still get to walk to work, and I’m enjoying the challenges, and once again, I’m learning new things all the time.

With those work changes, I wasn’t able to do as much work on the blog as would have liked, only 50 new posts this year. It averages out to one a week, but it was way more bursty than that. My goal for this year is to be not just more productive on the blog, but also more consistent.

From a content standpoint, this year’s articles have been more about the what and the why, and less about they how. Oh sure, I have opinions on how everything should be done, and I’m happy to share them, but the posts have focused on what kinds of things you should be thinking about and doing and why they’re important, not how to do them.

With that said, here are the 10 most popular posts over the last 12 months:

  1. Careers Pt 3 – Visualizing Progressions
  2. Careers Pt 1 – Levels
  3. Effective Software Engineers
  4. McDonalds
  5. Incident Response
  6. Let Me Tell You What I Want
  7. High Quality Quality
  8. Seeing Like A State
  9. Breaker Breaker Rubber Duck
  10. 1 > 2 > 0

I’m really happy to see that my career posts are still getting good traction, as are the Eng Competencies. How you look at your career drives what you put into it, and from that flows what you get out of it. And to round things out, 2 more posted this year since #1 and #2 are from last year

Hopefully you’ve found those, and other, articles interesting and useful. I know I found writing them helpful in clarifying and cementing my own understanding of the topics. Until next year, Happy New Year, and remember to enjoy whatever you’re doing. We only get one shot at this life, so make it worthwhile.

by Leon Rosenshein

McDonald's

I’ve been a subscriber of Kent Beck’s Tidy First? since almost it’s beginning. If you don’t have a copy of the book, consider picking one up. There’s a lot of good stuff in there. Not just things to do, but when and why to do them.

Since then, Kent’s been talking about his Thinkies, which are questions (or statements) designed to induce movement and provide you with more options and help you reach a decision. They’re short, straightforward, and are simple to apply. You start with a pattern, the situation that triggers the thinky, apply the thinky, then look at what you come up with. In many cases you have an entire new way of looking at things, a roadblock or two has been removed, and you get to make forward progress.

One of the recent ones was called McDonalds. It’s a great way to get a group that’s stuck in analysis paralysis moving again. When the see the group is stuck, make a suggestion. Not the perfect thing, and not something impossible, but something that’s unpopular, no-one really wants, but not completely ridiculous. The group won’t take your suggestion, and you really don’t want them to, but very often the simple act of suggesting a concrete solution will get the group to come together on choice that makes sense to everyone and is a good starting point.

This thinky’s name comes from the situation. The group can’t decide where to go for dinner. So you make a suggestion, such as McDonalds. Someone then says that’s a terrible idea. Let’s go to XXX instead, and the group comes to consensus. Deadlock broken and everyone gets to eat dinner.

You can use it in lots of situations. How should we manage workflow for this new project? Lots of discussion around rolling your own, buying one, finding an OSS solution, and lots of discussion around the pros and cons (mostly the cons) of each option. After a while folks are just repeating themselves, and you stop getting closer to consensus. You’ve got the equivalent of a super-saturated solution, but it’s just sitting there. To get things moving, you put out a throwaway choice. Something like “I know. We can use Celery/Flask to manage things”.

Immediately, things start to crystalize out of the solution. Celery/Flask won’t work because something. Instead, we should use Temporal because it solves all of our problems. It’s like opening Schrodinger’s’s box. All of the potential and possibilities collapse into a concrete situation that you can then move forward with.

So next time you and your group find yourself stuck trying to make a decision, try the McDonald’s Thinky. And check out the rest of them as well.

by Leon Rosenshein

Strong opinions, Loosely Held

Everyone’s got opinions. Some strong we feel strongly about, some not so much. Some we are willing to consider changing our minds on, others we aren’t. How you balance how strong your opinion is and how hard you hold on to it is the challenge. It’s easy to hold on to something too tightly or for too long.

That’s why companies have policies like Disagree and Commit, Meritocracy and Toe Stepping, No Jerks, or No A**holes. It’s to encourage getting the balance right.

What’s supposed to happen is that everyone has their opinion. When there are conflicting opinions, people bring their opinions and their reasoning together and make an informed decision, that everyone then carries out together. It seems like a good idea, and when it works it is a good idea.

The problem is that, like everything else, it’s a two-edged sword. Those policies are supposed to help the best ideas rise to the top. However, they rely on certain conditions being true. That all the people involved are equals, and that there are no, or minimal, penalties for being wrong.

Unfortunately, that is often not true. Building software is a social endeavor. At its core, it’s about people and the way that they interact. And people have their own motivations and goals, and not everyone has the same ones.

What often happens is that instead of a strongly held idea with the best evidence being chosen, the idea with the strongest voice wins. Sometimes that is the loudest voice, sometimes the most persistent, sometimes it’s the most senior person’s idea. That’s not what we want.

And no policy that relies on others to keep it will make it happen. Instead, it’s on everyone, but especially those with more situational power, to make sure it does. By watching what is happening. By making sure that there is a counter to the loudest voice. An amplifier for the quieter voices. A supporter for those that need it. The longer I’ve been doing this, the more often I find that I have the opportunity to help other’s ideas get the consideration they deserve.

That doesn’t give anyone the option of not needing the facts, the documentation, the proof, that their opinion is good. What it does do is make sure that a good decision is made with the best available information.

That takes care of the Strong Opinions part. Which leaves us with the loosely held part. But that’s a discussion for another time.

by Leon Rosenshein

Stop Designing

Stop designing up front the moment you start speculating.

          – Kent Beck

That’s a pretty bold statement. After all, how much do you really know about what you’re going to need to do at the beginning? At the beginning of a project (or feature, or task, or bug investigation) all you know is a rough idea of the problem to solve, not how you’re going to solve it.

To me, what Kent is saying isn’t that you should only do design on things that are known/fixed/unchangeable, but that the level of detail of the design should match the level of detail of what you know.

As an example, if you’re in the US, and you need to be in England in 2 weeks for 3 days, making the decision to fly instead of going by ship matches the level of understanding. Adding detail to that decision, i.e., deciding what day/time to fly on, or which airport you’re going to fly into, requires more information that you have. That info is probably available, and easy to get, but deciding to leave at 8 AM on Monday morning before you know if you need to be ready to work Monday morning in England. Similarly, deciding to fly into Heathrow before you know that you need to be in Edinburgh is just going to cause you to have to change your tickets later.

It’s the same with software. You have the problem you want to solve. You have some understanding of the situation as it currently exists. You cad (and should) do a design to the same level of detail. A handful of boxes on a whiteboard and some lines and arrows connecting them. You might have persistent storage. You’ll probably have a few different major steps in the process of solving the problem. There’s an expected flow between the steps, and some places where you might need to back up or start again. If you’re like me, you’ve already thought of 15 different ways things can go wrong, so you know you’ll need a way to get information about a failure late in the process back (or forward) through the system. Given what you know, that’s about how much you can design. After that, you’re speculating. You’re guessing really. It might be an educated guess based on past experience, but it’s still a guess.

That’s the point to stop designing and start implementing. Build some APIs. Let some test data flow through the system. Hardcode some good and bad responses and see how the system responds. Adjust your APIs and data structures so things flow well in both happy and sad cases. Learn from the system you’re build. If something seems hard to do with your flow, it’s probably not you. It’s the system telling you that you’ve uncovered an emergent property, and you need adjust to work with it. Once you’ve got it working, look again. Do some refactoring and cleanup. Listen to what the system is telling you and adapt. It’s Test-Driven-Development at work. Or if you don’t believe in that, you can call it Domain Driven Design. Or Behavior Driven Development. Or even Extreme Programming.

It doesn’t matter what you call it. The important part is that you design to what you know, leaving room to adjust as you learn more. Then you learn a bit, design a bit, and do it all again. Rinse, Lather, Repeat. It’s more than a way to sell more shampoo.

Don’t worry (yet) if it can handle 10K transactions/sec or if it’s resilient to a backhoe taking out a datacenter. You might need that. You might even know you’ll need it for a production system, and you might be right, but you don’t need it yet. So don’t worry about it. Don’t preclude it, but don’t code yourself into a corner where you are stuck with a decision you made before you knew what tradeoffs you were making.

Beware of biases. There are lots of them. Sunk cost bias. You’ve put so much effort into making this thing work you don’t want to give it up. Recency bias. I just finished this, so I need to use it now. Overconfidence Bias. I know what I’m doing, and I know everything I need to know already, so my initial idea must be the right one.

So don’t design for things you don’t know. Or, as Jack Burton said, I never drive faster than I can see. You just need to develop the reflexes to know how fast that is.

by Leon Rosenshein

Break It Down, Even At Small Scale

I’ve talked about taking breaking things down many times. I think it’s a really good idea. I’ve talked about how it applies to commits, pull requests, and projects in general. It can help you move faster and be more responsive to both changing conditions and things you learn along the way.

It also applies at smaller scales. You can have one huge, long method, or you can break the method down into logical parts, with another method around it to manage data flow and decision making. It’s the Unix Way. Each method does one thing, then you have another method that chains them together. It’s easier to understand, easier to modify in the future, easier to debug, and easier to test. What’s not to like?

It applies at even smaller scales than methods. When you have a long computation or an involved condition, there’s always an urge to play code golf and shorten it. Sure, it can work. It might even take a few less characters. But you should resist the urge.

Instead, make the code longer. Just like you should break down long methods into multiple methods that do one logical thing, then use a wrapper method to manage control and data flow, you can break down long calculations. Use well named intermediate variables for storage. Create specific variables to hold the results of comparisons. Then, combine those intermediate variables as needed. If you do that, you’ll find that the result is more readable, more debuggable, and more flexible.

Consider this case I wrote about before.

var filteredEntries []userstate.StateEntry
initialTargetLen := len(targets)
for _, entry := range entries {
    req := entry.Value.(authutils.GetCredentialsRequest)
    // This is getting complicated, so breaking it down. We EXCLUDE a profile if ALL of these conditions are false
    // 1) The user specified a list of profiles and the current profile is one of them
    // 2) The user did not specify a list of profiles AND specified the --all flag OR the --aws flag
    // 3) The user did not specify a list of profiles, the current profile is not an admin profile,
    //       and the user did NOT specify the --admin flag
    // 4) The user did not specify a list of profiles, the current profile is an admin profile,
    //        and the user specified the --admin flag

    if !((initialTargetLen > 0 && targets[entry.Key]) ||
        (initialTargetLen == 0 && (refreshParams.Flags.All || refreshParams.Flags.Aws)) ||
        (initialTargetLen == 0 && roleIsAdmin(req.AWSRole) == refreshParams.Flags.Admin)) {
            continue
        }
    filteredEntries = append(filteredEntries, entry)
    delete(targets, entry.Key)
}

That’s a really complicated conditional, and every comparison is written in low-level terms. A more readable, understandable version might be something like

var filteredEntries []userstate.StateEntry
userSpecifiedList:= len(targets) > 0
refreshableRole := refreshParams.Flags.All || refreshParams.Flags.Aws

for _, entry := range entries {
    req := entry.Value.(authutils.GetCredentialsRequest)
    // This is getting complicated, so breaking it down. We EXCLUDE a profile if ALL of these conditions are false
    // 1) The user specified a list of profiles and the current profile is one of them
    // 2) The user did not specify a list of profiles AND specified the --all flag OR the --aws flag
    // 3) The user did not specify a list of profiles, the current profile is not an admin profile,
    //       and the user did NOT specify the --admin flag
    // 4) The user did not specify a list of profiles, the current profile is an admin profile,
    //        and the user specified the --admin flag
    entryExists := false
    if userSpecifiedList {
        entryExists = targets[entry.Key]
    }
    roleAdminFlagMatchesCLIFlag := roleIsAdmin(req.AWSRole) == refreshParams.Flags.Admin

    if !((  userSpecifiedList && entryExists ) ||
         ( !userSpecifiedList && refreshableRole ) ||
         ( !userSpecifiedList && roleAdminFlagMatchesCLIFlag )) {
            continue
        }
    filteredEntries = append(filteredEntries, entry)
    delete(targets, entry.Key)
}

Overall, it’s more code, but it’s been a looong time since the size of the source code has mattered much. And in the areas that matter to future me, it’s much better. The conditional is much less complex, it’s easier to understand because it reads much more like the English in the comment, and it’s much easier to add to if needed.

I never did make that change, but if I ever do go back into that code that’s the kind of change I’ll make.

by Leon Rosenshein

It's A Trap

Overheard:

… coming from another language and trying to find the library that makes it look like the language you came from is a trap

Pretty much all of the languages we write code in today are turing complete, meaning they can be used to write a program that does just about anything you want. That means all languages are equivalent, and the choice doesn’t matter, right?

Wrong

And it’s wrong on many levels. Consider this simple case. You’re going to spend a day or two, by yourself, writing a tool to do a job you need done now. Somehow, you know that once you finish writing it, you’re never going to touch it again. You won’t need to debug it later. You won’t need to understand it again later. You won’t need to update it later. No one else will ever see the code. Even in that case, your choice of language matters. As Larry Wall said,

Computer languages differ not so much in what they make possible, but in what they make easy.

That means your choice of languages is important even in the simplest situations.

Of course, we’re almost never in the simplest situation. You will need to debug the code. You will need to understand it later. You will need to update it. And you won’t be the only one who needs to do all those things. After all, your code needs to be readable and understandable not just by you when you’ve got all the context, but by someone else in the future without all the context. And it’s the future which influences the choice of language the most. The context that the code and the people that will be using/maintaining it will be operating in. If the tool you’re writing is going to be used in a front-end website and extend and connect with a large body of JavaScript, you should probably use JavaScript, not PHP.

Of course, choosing a language is only the first part. There’s always a tendency, especially when you’re learning the ins and outs of a new language, to just adjust to the new systax, but do things just like you’re used to doing them. After all, you can write code in just about any language that looks like the C code in the K & R book, and as I said, it might feel easier to write at the beginning.

Which gets back to that original thing I overheard. It’s a trap. Just because you can make one language look like another doesn’t mean you should. There are better choices for how to write things.

Languages have idioms. And ecosystems. When you write Python you should be pythonic. Code in Go should follow the Go Proverbs. C++ has its own set of idioms, and they’ve evolved over the years. Regardless of the language, you should understand and work with those idioms. The libraries, tools, and examples will be using those idioms. Working with them is going to be easier than trying to make it look like the last language you worked with. You should never be in a position where you’re working against the code or trying to coerce it into doing what you want. You never want to make things harder for yourself.

You should do it not just to make it easier to write, but because software development is a social activity. Working with those idioms will make things easier and clearer in the future. For people familiar with the language, it makes it reduces the cognitive load of reading it. Things make more sense, and you’re working with people’s expectations, not against them.

After all, you, and everyone else involved, are going to be living in the future, so have pity on future you.

by Leon Rosenshein

Visibility

I don’t often give career advice. Or rather, while I often talk about what good engineers and developers do, the importance of adding value, and the importance of managing your own career, I rarely talk about specific things you need to do to get a promotion. I’m of the opinion that if you grow and improve yourself, add value, and create and take advantage of opportunities along the way, your work will be recognized and valued.

That said, knowing how to manage your career is advice on how to get promoted. The difference though, and it’s an important one, is the motivation. Managing your career in a way that lets you learn and grow and expand your scope of influence means doing things that will help to get you promoted. When you do those things to learn and grow, they satisfy internal needs. The satisfy the need for Autonomy, Mastery, and Purpose. t’s good for you. It’s likely good got your team. It’s likely good for your company.

On the other hand, when you do things with the goal of getting promoted, you might learn and grow, but those are collateral benefits. Checking boxes on a level rubric can get you promoted. Unfortunately, when your focus is on checking those boxes, you end up not helping yourself, not helping your team, and not helping the company you’re working for reach its goals. Worst case you end up with what Charity Majors described as roving bands of skilled, restless engineers competing for vanity projects. And when you get the promotion, you’re likely to find that while it’s satisfying for a little while, it didn’t really satisfy that internal need, and instead of enjoying things, you just keep struggling for the next promotion.

The thing is, regardless of your motivation, you’ll do many of the same things. One of those things is ensuring you have visibility. Which brings me to this image, from workchronicles

I’ve mentioned workchronicles and shared one of their comics before. Like many of those comics, it’s more than a bit cynical, but there’s a kernel of truth hiding in there. You need to get past the tyranny of or. You do need to talk about what you’re doing and what you’ve done, but NOT at the expense of doing the work. Advancing your career requires a bunch of things. Of course, you need the ability. You need to demonstrate that you can do the work, but ability alone isn’t enough.

After all, your coworkers, your peers, have roughly the same ability as you do. They might have deeper knowledge of something specific, or a broader knowledge base, but they’re your peers. Assuming everything else is working the way it should (yes, I know what happens when you assume) if they’re your peers, your abilities are roughly equal.

Since ability isn’t enough, what’s the next thing? The next thing is that others need to know that you have the ability. That’s where visibility comes in. To manage your career, to set yourself up for growth, to be in a position where you can see opportunities, you need to make sure that your ability is visible. Or, as I said last week, you need to radiate information. The key is to make your ability (and the value you’ve added by using that ability) visible to people without bragging or forcing it down other’s throats.

You do that by informing people of what you’ve done, and how it helps them and makes their lives easier. You do it by asking relevant questions. Not to show how smart or observant you are, but to help others make the right decisions and to make sure they haven’t overlooked any potential problems. You do it by helping others and sharing your ability when it’s needed, and you can help. You do it by documenting and sharing your learnings. You do it by not being indispensable in an area, but by helping others learn that area themselves. You do it by doing the work that’s important and valuable and move things forward for everyone, because it’s important and valuable to everyone, not because it’s flashy, or “level appropriate”.

If you do those things your visibility will go up. Your scope of influence will go up. You’ll see bigger pictures and see more opportunities. Your autonomy, mastery, and purpose will go up. You’ll be more satisfied. And, as a side effect, you might get a promotion.

by Leon Rosenshein

Radiating Information

According to the Hacker Ethic, per Steven Levy in Hackers, all information should be free. Stewart Brand anthromophized it and noted the tension betweemm the cost of distributing information going down and the value of having the right information at the right time going up. He said

It seems like there’s a couple of interesting paradoxes we’re working with here. That’s why I’m especially interested in what Bob Wallace has done with PC-WRITE and what Andrew Flugelman did before that with PC-TALK. On the one hand information wants to be expensive, because it’s so valuable. The right information in the right place just changes your life. On the other hand, information wants to be free, because the cost of getting it out is getting lower and lower all the time. So you have these two fighting against each other.

So now you know where the idea that information should be free comes from.

The thing is, the information might want to be free, and there are those that will seek out the information, when they feel they need it, but that doesn’t say anything about people having the information they need when need it. Because regardless of what the information might want, it doesn’t have the ability to go out and find people.

On the other hand, we, as developers of systems and platforms, do have access to lots of information that wants to be free. And it’s up to us to make sure that information gets where it wants to be.

You’ve probably seen an error message like this

That’s not very helpful, is it. And the worst part is that the person who made that dialog box show up almost certainly knew more about the problem. They just chose to hide the information.

That’s a pretty obvious example of information being hidden by not being shown. Another way important information gets hidden is by being drowned out by less important information. Let’s say you’ve got a service (like a Wordpress blog with a security buddy plugin) that sends you the status of a security scan every day at 2:00 AM. And every day it sends you the same list of 7 files that have been replaced by the overnight backup. How long do you think it will take before you either just ignore that email or come up with a rule in your email client to delete it, or at least shuffle it off to an ignored folder? So that when one day the email says 10 files have been changed you never see it, let alone do anything about it. In that case, the information is free and available, but it’s just sitting there waiting for someone (you) to notice/care.

What that’s really pointing at is the difference between data, information, knowledge, and wisdom. We’re drowning in data. The key is to take that data, distill out the information in it, and present it as knowledge to the user. I don’t need 100 emails from all of the services I care about every day telling me they’re working fine. An email that one of them crashed and has restarted is useful information, but that can happen for lots of transient reasons out of my control, like a power outage, network drop, or hardware failure. That’s useful knowledge and I’ll care about it soon.

(Almost) Nobody reads automated status emails. What I really want is the knowledge that the service has tried to restart multiple times in the past 5 minutes, and hasn’t been able to reach a healthy state. That’s knowledge I need to respond to NOW.

The same applies to letting my users/customers know how things are going. They don’t want daily emails that everything is OK. But a red banner that shows up on every page of the website telling them about current problems/outages is knowledge they can use.

Or as my GPM Brad told me many moons ago, “Nobody reads the status email. I need to be alerted to the problem, so do that instead.” It doens’t matter how free the information is or wants to be. What matters is that the people who need to know it do know it when they need to. The value of having the right information at the right is incalcuble. Even if the cost to deliver a header or not deliver it, over the cost of the lifetime of a website, is esentially zero.