Recent Posts (page 60 / 69)

Cautionary Tales

In the face of ballooning system requirements or the need to future proof your code, you may be tempted to add another layer of indirection. All problems in computer science can be solved this way, right? Not so fast. Overly-complex designs can be difficult to comprehend and maintain, and when they go wrong, they can go really wrong. A wonderful new podcast from Tim Harford highlights the risks--particularly for safety systems like nuclear power plants--of layering on complexity without enough thought. It’s a delightful romp from the 2017 Academy Awards to a 1638 post mortem by Galileo.

Cautionary Tales podcast
Benefits and risks of indirection

All problems in computer science can be solved by another level of indirection
    —Butler Lampson
But that usually will create another problem

    —David Wheeler

Kissenger, Schmidt, and Huttenlocher, and AI

What do a former Secretary of State, the ex CEO of Google, and the current Dean of the MIT College of Computing have in common? Not their age, not their education, and certainly not their jobs. But they all, through direct actions and through the people and companies they advised, led, and taught, have had big impacts on your lives and the world in general. And what do they have in common around AI? Two of them have an obvious connection, but a 96 year old historian isn't someone you usually think about when you think of AI. For today I give you a couple articles on AI and society. Something to think about.

Engineering Principles

Here's a few of my favorites. BSR, CFtM, DRY, KISS, PLA, SOC, YAGNI

According to wikipedia, a principle is a proposition or value that is a guide for behavior or evaluation. Following your principles is a good thing, both at work and at outside work. They help you make decisions and figure out what to do. Good principles help you make good designs. Good designs lead to good products and good outcomes. So what principles should we be following?

You've probably heard of those acronyms, but just in case:

Boy Scout Rule - Leave things better than you found them
Code For the Maintainer - which is most likely you, so do yourself a favor
Don't Repeat Yourself - If you need to do something twice, use the same thing, don't build a new one
Keep It Simple Stupid - Make things as simple as possible, but no simpler
Principle of Least Astonishment - Don't surprise your user. Do what they expect
Separation Of Concerns - The old Unix philosophy, do one thing, and do it well
You Ain't Gonna Need It - Don't create things you don't need now. Wait until you need it. You may never need it, and if you do, you'll know much more by then.

Following these principles helps at whatever scale you're working at, whether it's a function, feature, service, platform, or product. What principles speak directly to you?

PM Is More Than Provider Of Meals

I've been working with PMs of various breeds at multiple companies for years now. Many years ago at Microsoft there was a class called "PM is more than provider of meals". Yes, at MS the PM was often worrying about team morale and making sure we were fed and clothed (lots of t-shirts), but that's not really why they existed. They were the voice of the customer (whoever that was, internal or external), worked with partner teams, vendors, and generally filled all the gaps that no-one else was filling. They did things like communicating up, down, and across, providing product vision, leadership (but no authority), problem solving, and prioritizing. Not in a vacuum, and they were by no means the ones doing all of any of that, but they were a backstop to make sure things got done.

At ATG we have, at least, Program Managers, Product Managers, Project Managers, and Technical Program managers. Different roles with different skills and expectations, but it's not always clear which one you have, let alone which one you want or need. In most cases you need parts of all of them, and the best PMs (regardless of title) combine all of the different skills, to varying degrees, and just help you get things done. Unfortunately, our PMs cover very wide areas and we almost never have one of each working directly with each team/group, so how can we move forward most efficiently?

Most importantly, we all need to be clear about who's responsible for what, and that's going to vary by team. In the end, all of the bases have to be covered regardless of the mix of titles involved in any particular effort. So whether it's the PM, EM, TLM, TL, or an IC, we need to make sure things are covered. And we need to make sure things aren't covered multiple times so there's no confusion about who's responsible for what. How are you and your team handling it? What best practices have you come up with?

Prediction

As engineers we make lots of predictions. Some of them are about our what customers/users want and need, some are about how the things we build will interact, and some are about ourselves, what we are going to do and when we are going to do them. If you were to order our predictions by how accurate they are, that's probably the order to put them in. Which is kind of odd. The people we know the least about we are the best at predicting, while the people we know the most about (ourselves) we have the most trouble with. There are all sorts of reasons, but I think some of the big ones are sample size, using data, and optimism.

Consider our customers. As a company we have millions of them, and it's much easier to predict the average of a million things is easier than guessing a specific one. And we do experiments on subsets of them. We control for different situations, and run the experiments different ways, adjusting the experiment and the sample until we're confident in our predictions. Then we make the change and if our predictions were wrong we figure out why and use that knowledge for the next predictions.

For our system interactions we have unit tests, integration tests, simulations, and even track tests. Again, all designed and constrained to explore the boundaries of the system interactions and help us predict how things will happen "in the wild". Here too, if we find something that was missed we update/extend the situations we use to help us make predictions so that next time we're more sure of ourselves.

On the other hand, do we apply that kind of rigor to our development estimates? We have sprints, we have backlogs, we have story points. We do retrospectives. Then we miss our dates and do it all again. Does your team adjust the number of points in a sprint based on your historical completion rate? Do you have a way of correcting for new kinds of work or new areas? Do you account for the fact that not everyone is the same in all areas? What about dependency management?

That's not to say that it's terrible and we shouldn't try. Things are a lot more predictable now then they were when I started doing this. But we can do even better. As a small team we've seen good results from adjusting the number of story points we accept into a sprint based on our past performance. And we don't make anything about time. There is no conversion between story points and hours. We don't explicitly account for on-call work or KTLO, but that is built into our history of how many points we were able to close. It's not perfect. Our numbers don't directly apply to other teams. When we dive into a new area it gets a little rough, but it's a lot better than it was. And, it helps our customers. They might not like it when we say we can't get something done by a certain date, but the can at least plan around it. And that's a really big win.

Makefiles

Relevant or not? What's the point of a Makefile in a bazel world?

bazel is a turing complete, extensible build system, so if your core build is done with bazel is there any reason to use anything else? In theory, no. You can do anything with a collection of bazel rules and targets. In practice though, bazel isn't the simplest of systems to use, and the best tool around to understand what bazel is doing is bazel query, so there's a bit of a chicken and egg problem there. Your IDE isn't much help either. Many of them can do syntax highlighting, but they don't tell you why something was built or which rule(s) got fired.

On the other hand, Makefiles can't handle the depth/complexity of the trees we build, or at least not simply, and they just don't provide the power/flexibility to only do exactly the things you want done. They're also strangely prescriptive about the format of the Makefile itself. Not quite COBOL, but leading whitespace is important (and mostly invisible). But they are pretty simple to understand since the basic rules are simple.

So what's an engineer to do? For your actual build, bazel is a good choice, especially if you're in a world with lots of other bazel things (NA, Sim, atg-services, etc) then do your build in bazel. If you're not, consider it. After all, we're heading for a monorepo with the majority of the build being managed by bazel.

But there's another use case for Makefiles. Simple scripting to make your life easier and reduce typing. That's the Makefile of phony targets. It's a simple way to build/share a workflow and integrate 3rd party tools into a workflow, possibly including a bazel build <big long string of characters denoting the target>. you can even pass in arguments to help specify what you want to do. In that case Makefiles make sense, at least for a while. If you find yourself building/sharing complex Makefiles that's probably an indication that you should be talking to the DevX team about better shared tooling.

So what do you think? Makefiles, bazel, or some combination? Share in the thread.

Imposter Syndrome, Dunning–Kruger, or Just Do It

There are new things for us to learn all the time. Some of them are just new coats of paint on the same old thing, some of them build on existing tech, some of them are really new. Sometimes you end up in working on something tangentially related to your area of expertise that's new to you. How do you approach it? Is it a problem or an opportunity? Generally, I choose to see them as opportunities. I get to learn something new. I get to expand my repertoire. I get to show my boss I'm flexible. All good things.

But that doesn't make it easy. Sometimes convincing yourself is the hardest part. Then you've got to actually get started. I've found the easiest way to handle both is to ask questions. Find the person that understands the problem. Ask them to explain it. Figure out exactly what part you don't know and ask some simple, but well-formed questions. Use those answers to ask some more questions. Then try some things. See what works and what doesn't through your personal approach. Then draw the rest of the chart. You'll be amazed at what you can accomplish.

Hello vs. NoHello

Hello or NoHello? It's a thing. Some people want the social lubricant, some don't. The first thing to remember is that Slack (or any other asynchronous communications method) is not a face to face, real-time, discussion, so the rules/norms/best practices are different. It's also much more likely to be remote/cross time zone :)

The two biggest things to remember in chat are

1) It's very hard to communicate emotion/expression/intent, which means the failure mode of "clever" is "jerk"

2) Asking if you can interrupt is already an interruption.

John Scalzi handles the first one well enough that I'll leave that to him.

The second is more community driven and we collectively need to figure out what our norm is. Like anything else group based it's a consensus/compromise. It's also situational. Expectations are different in a 1:1 discussion about a new work issue or about last weekend's new movie, or a request in a large group. My position is nohello, or if you feel it's rude/wrong, then preface your question with the salutation of your choice, then immediately ask the question in the same message. Notice that I don't say hello before I send out a post here. So what do we want our norm to be? Share your thoughts in the comments.

Asking For Help

I love helping our customers. As a person on the infra team I get to see my customers every day and I get to see the results of making their lives easier, helping them build self driving cars that drive more than themselves. Asking for help can be a good thing. You save time, you get to learn something, Knowledge is shared. If you do it right you also make it easier for the next person who runs into the same problem. But that's a topic for a future post.

Today's post is about a pet peeve of mine. As much as I loved to help people, sometimes it's hard. The worst is when someone posts a screenshot of some error text in the console. No context, no idea what the command was, not even copyable text. Just an image. And usually a blurry jpeg. Please don't do that. Whether it's email, slack, a bug report, or carrier pigeon, post real text. And post the command that led to the error. The more context you can provide the easier it is to help you and the less back and forth I (or anyone else) needs to do.

There are lots of lists of why this is a bad idea online, but my top few are:

1. I can't copy/paste the text, so it's hard (impossible?) to reproduce
2. Screenshots usually miss something important
3. Images aren't searchable (makes it hard for the next person with the problem to find the answer)
4. Invisible/non-printing/unicode issues are hidden.

Talking To Customers

How do you talk across functions? When engineers talk to other engineers, even across disciplines, there's usually at some shared language/terminology. When you start talking cross-functional, that doesn't always hold true. In fact, many of the assumptions aren't shared, and that can lead to huge misunderstandings. For example, you would expect your customer to know what they want.

And they do. They want it to not suck. They want it to be easier. They want it to be faster. There you go. 3 simple requirements. Go build the thing they want. What's that you said? Those requirements don't mean anything? Maybe not to you, but they do to your customer.

So how do you delight your customer? By understanding them. It's not their job to understand you, it's your job to learn enough about them to understand what they want AND to understand how to communicate what you're doing to them. If you don't know their world find an SME who does, or make sure someone on your team becomes one. Understand how they do their work. Watch them do it. Ask them why they do it that way. Find the business value they're looking for and provide it.

That's how you understand your customer, but how do you make sure they understand you? First and foremost, talk about what they can see. The business value. The workflow. The interface. Don't worry about the features you're building, but talk about the benefits you're delivering.

And a side note. Don't over-rotate on this, but never confuse selling with installing. Whenever you build something for a customer you need to continue to convince them that what you are building is worth the time and money it's going to take. Provide clear deliverables/demos with functionality. Figure out what your MVP is, then add to it. And listen when they provide feedback. That doesn't mean scrap everything you've done, but understand the problem behind the complaint and work to make it better.