Recent Posts (page 39 / 65)

Semantics

Debugging customer problems can be a challenge, especially over zoom or email where you can't see the exact input/output. To make it easier I have a set of questions I usually ask at the beginning. One of them is "What version are you running?" That's really important info since there can be many versions out in the field.

For the infra CLI it's pretty easy to get the version. Just run infra version and it will tell you. This is really important in *NIX based systems, because unlike Windows DLLs/executables, posix ELF files don't have a standard way to store/retrieve the version info of a file. Instead, for shared objects you use file name and a system of symlinks with prescribed names to manage version. Using that is a topic for another time.

The topic for today though is what to put in the version. The simplest is just a continuously incrementing number. Every time your build process produces a public version the version number gets incremented by one. This covers the most basic requirements of versioning. It ensures that all public versions can be uniquely identified and you can tell which of two versions is newer. That's a good start. But it leaves you with a problem. You can tell that one version is older than the other, but you can't tell how much older, and you don't know from the version number if it was yesterday to 10 years ago.

Another versioning scheme is to use the build date/time as the version. Something like YYYY.MM.DD.hh.mm.ss.<buildnumber>. Year/Month/Day/Hour/Minute/Second. That's almost certainly unique, but if you have a parallel build system that can start multiple builds at the same time then you can add the build number for disambiguation. It's important to use that order because that ensures that you can sort by version. The date scheme gives you the ordering of a simple number, and lets you know when the releases were done. That's a lot more helpful, and users really like it because it's obvious to them if something is old

As a developer though, that doesn't tell me something I really want to know. Should I expect that version to have what I'm looking for? And that's where semantic versions (SemVer) come into play. Semantic versions are of the form <major>.<minor>.<patch>. The theory is that all versions for a given <major> version are backward compatible, and increasing <minor> versions only add functionality, Removing functionality would be a breaking change and require a new major version. For a given <major>.<minor> everything is the same, and new versions fix emergent bugs/issues that are discovered after release. Pretty simple. SemVer trades the absolute time information for functionality/compatibility info. In my experience that's a trade worth making.

As with any system, you'll find that sometimes you need to extend it. You might have a on-off release, a shared fork, or simply need to test a version locally, but still be able to keep it distinct from all of your current and future released versions. A common way to do that is with an extension. That's where user name and date could be very helpful. Let's say I want to give you a test version I built based on the code in version `0.17.1`. In that case I might give it version `0.17.1-leonr-test_myfunc`. That way everyone who sees it would know the base version, who built it, and why.

And of course, you'll want this tied in to your version control system so that it's easy to get the code that went into a specified version and your build system, so you don't have to worry about getting it wrong.

Rockall

Back in my early Microsoft days on the FlightSim team we were always looking for better perf. If nothing else we needed to give as much time to the rendered as possible because our customers always wanted to run the graphics with everything set to 11. And since this was somewhere between C and C++ (home made cross module vtable and dynamic linking, but that's another story) memory management was a thing. And Intel's vTune told us that allocating memory was taking a measurable amount of time.

Our solution was to build free lists for each type of object we wanted to make faster. The initial implementation was mostly replacing free/delete with putting the object into a linked list and having malloc/new check the list then allocate it if needed. This worked, but we needed to be careful with re-use and re-initialization, and didn't give us much protection from stale pointers to existing objects. Slowly we added more safety and protection. But it was always work and made things more cumbersome.

One of the nice things about working for Microsoft was access to Microsoft Research (MSR) a whole division of folks whose job was to think about big problems in computer science and figure out solutions. And they were always looking for new problems and ways to get their solutions into shipping products. We got all sorts of interesting tools and ideas from them. One of them was a memory management system called Rockall.

Rockall was based on individual heaps, basically a heap per type of object, and gave us a couple of advantages over simple new/malloc. The biggest was that it was a drop in replacement. We didn't need to do anything but set it up at the beginning, and then it did it's magic. Don't ask me how that part worked. I just used it :) We would pre-allocate the space for however many of each of the different types we wanted, then getting a new one was blazingly fast since the memory was already allocated and waiting. Initialization magically happened. It was also contiguous, so we got better memory packing. Just that level of usage gave us a lot of benefits.

But it also had a bunch of debug features which helped out. If you had enough memory you could tell it to put each object at the end of a page and mark the next page as unreadable. Any memory overruns are quickly pointed out in that case. You could set it to mark the page as unreadable on delete, and again, any use after delete quickly became apparent.

It was also faster on cleanup. Transitioning out of flying and into the UI just required a few calls to delete each heap instead of making sure each and every object was individually deleted, in the right order, so we didn't leak anything. At debug time it could also tell us which things hadn't been deallocated manually if we wanted, but since we used it on the way out we didn't care.

That kind of thing can also help out in GC based code as well. Memory that's alway in use is never subject to GC, you don't end up with things moving around in memory to compact, and your allocation times go down. That was really important when we were doing 3D rendering in the browser for VirtualEarth. We didn't have Rockall, so we had to use a manual free_list, but we still got most of the benefits.

Something to think about.

In Defense Of Agile

And just to be clear, it's not that I think everything was better 30 years ago and all you kids these days should get off my lawn. Like everything else, there are cycles and sometimes things swing too far, or more often, people pick up the terminology / trappings of a new idea and then ignore the details.

Agile can be like that. Pair/mob programming. Just in time design. Customer feedback. Short cycles. On the surface, or blindly applied it makes no sense. Two people working on the same line of code is half the efficiency. Just in time design turns into no design. We can't ask customers what they want. They just want a better buggy whip. And how can you get anything done in two weeks? It takes two weeks for us to agree on who's doing something, let alone get it done.

But really, these new ideas aren't about how. They're about why, and the how is the means to an end. Pair programming isn't about efficiency. It's about a broader perspective, making sure things are clear and understandable, knowledge sharing, and up-leveling a team. Just in time design isn't "Make a choice then correct it later by fixing the mistakes" it's "Make the informed decisions you need when you need them and know enough".

Customer feedback is really customer involvement. You need to understand what they're doing, speak their language, and understand their pain so you can collaborate on a solution. And short cycles is part of that. It's ongoing collaboration to iterate on a good solution instead of providing the wrong solution later. It's about providing value sooner so in the ling term everyone is better off.

And that's the big take-away/conflict. The focus on the short term requires a long-term view. You need to make the right choice at the moment that balances what you're doing today with what you're going to be doing the next day/week/year.

Pick 2

We've all heard that when it comes to quality, scope, and the time you can only optimize 2 of them at a time. And for any given, localized, immediate decision that might be true. But what if I told you that over the long term you're better off with all three?

Following up on yesterday's topic, another one of my issues with two week sprints is the emphasis it places on time and showing value quickly. If every decision is based on what's going to produce the most value in the current sprint then you end up pushing a bow wave of cruft ahead of you. Eventually the resistance from all that cruft brings your forward progress to approximately zero. That's the classic description of technical debt.

Or to use a car analogy, consider Harbor Freight tools vs Snap-on. Harbor Freight sells project tools. Buy them, use them for one project, two if you're lucky, then move on. Snap-on sells tools for life. If you've ever had the opportunity to A/B compare the two you know what I mean. If you need a specialized tool you can run down to Harbor Freight, pick up the tool, and be using it for less money long before you can get the Snap-on truck to stop by. The second time you need the tool it will probably still work, but the third time you'll need to go buy another one, wasting time and money. And if you need it regularly pretty soon you realize you would have been much better off, getting things done faster and with higher quality, if you had just spent the time and money to get the good tools the first time.

Software development is the same thing. There are times when you need to be fast or cost conscious. Back in the day when I was working on games we needed to have the game on the shelf the day after Thanksgiving. If we didn't we'd lose a large percentage of initial sales, and never make them back up, so making the trade-off made sense. If we didn't get the game out and get the revenue the company might have gone out of business. On the other hand, on Falcon 4.0 we missed the next product cycle because we were still digging ourselves out of the hole we dug. It was the right choice, but only because we literally couldn't afford to spend the money up front.

Over the long haul you're better off doing it right. You'll find it's cheaper, faster, and provides more customer value.

Cycles

There's a lot of value in short sprints and frequent planning cycles. At any given time there's so much we don't know about the future needs and requirements of whatever piece of software we're working on that to write a detailed 6 month plan, let alone a 2 year plan, doesn't make a lot of sense. Moving away from waterfall development has codified that reality and helped to normalize the level of effort across the development cycle. Instead of the 3 month long crunch times (12+ hour days, 6-7 days/week) at the end of a 2 year development cycle we have a busy day or two every two weeks.

While short sprints might be better for the project, and definitely smooth out the level of effort, I'm not so sure that's good for the developers. We've done away with the crunch period at the end, but we've also done away with the low effort time at the beginning. Back in the day the first milestone was often a planning milestone. The output was a set of documents. Those documents themselves didn't have a lot of value and were often out of date before the virtual ink was dry, but writing them had a lot of value. Ideas were floated, adjusted, readjusted and prototyped. Some things were discarded and some things were discovered. Blind alleys were explored and opened or closed depending on what was found.

And for the developer there was very little pressure. Expectations were low. Time was available for random exploration. I think we've lost some of that now. In a sprint there's not a lot of time, so it all needs to be accounted for and there's pressure to make sure it directly accrues to some higher goal. And I'm not saying that's a bad thing. In fact, it's good to make sure that work done has some connection to the bigger goals. But it makes it harder to experiment. And it makes it harder to do things that take more than two weeks or don't have explicit outcomes. Yes, a Spike is a thing, but officially it shouldn't be longer than a Sprint. And it should have defined acceptance criteria. But it's not the same thing as "Here's a big hairy problem. You two folks go off and noodle on it for a month. Try things out. See what works and what doesn't. Come back with a design."

That's what we've lost. The ability for developers to have the scheduled downtime to decompress from the daily grind, then come up with something new without a ticking clock. Supersprints are good, but they're not providing the kind of freedom I'm talking about, and that's not what they're for. Whether we call it M0, a Planning Period, or something else entirely, I think we're worse off without it.

Don't get me wrong. I'm not suggesting we go back to waterfall. I much prefer the current model. I just think we need to figure out a way to make our cycle a little more dynamic. I don't want a 3 month crunch, but I do want to figure out how to get the periodic unstructured time back.

Circle Of Life

Did you know that Microsoft released a 3D world map with the ability to look at any part of the globe from any angle? Roads were mostly accurate, as were rivers, lakes, and islands. There were thousands of highly detailed 3D models of significant buildings and landmarks, and over 20K highly detailed areas. Of course, since it was Flight Simulator 2000  those 20K areas were airports and your house almost certainly wasn't one of them. I was on the team then and it was a pretty significant accomplishment at the time. We used digitized satellite data, digitized topographical maps, survey data, government data, and some 3rd party content to make the world look real.

A few years after that Google Earth was released and Microsoft, in true fast moving tail lights fashion, needed to have one. So a small group of us left the FlightSim team and went over to the newly formed VirtualEarth team to build a 3D world in the browser. We did that too. C# management in an ActiveX component over Managed DirectX. We used FlightSim data for roads, rivers, airports and all of the landmark 3D models. It was cool and all, but not many people cared. They didn't have the interest, need, graphics card, or internet bandwidth to really use it. And the world was pretty flat. So we bought a company that made aerial cameras and turned the imagery into a 3D model and helped them scale up. Oh, and we put a very simple 6DOF flight model in the rendered so you could fly around. No cockpit/instrument panel/navaids, but it felt "airplane-like". So Bing Maps is a direct descendant of FlightSim.

 By the time MS sold our part of Bing Maps to Uber we were building 1 meter or better resolution 3D models of the world from 5 - 30 cm resolution imagery. We were generating our own road networks. We incorporated data we derived from street level imagery to get road names, turn restrictions, speed limits, and business names/addresses. Because we needed a routable map for our users.

And here we are in 2020. Microsoft just released a new version of Flight Simulator. This time outsourced to a 3rd party developer. And guess where they got their data? Bing Maps. So a new game, with a highly accurate 3D world, built from satellite data, aerial data, government data, and user generated content is now available on-line (or as a 10 DVD box set).

So we've come full circle. FlightSim started from existing maps then turned into a map with VirtualEarth/Bing Maps, and now Bing Maps has turned back into FlightSim

Linkapolooza

I use StackOverflow as much as anyone when I am looking for the answer to a specific question/problem, but when I'm looking for a deeper description of a topic or approach there are usually better places to go. And sometimes I'm just looking to see how other folks do things and why. Not just to solve today's problem, but to add tools to my toolbox. You can never have too many tools, you just need to know when to use them.

The first few are very active, while the latter are more static, but still have a lot of useful info and current comments. Enjoy these and add your favorites in the thread.

https://www.developertoarchitect.com/

https://martinfowler.com/

https://medium.com/@jamesagwa/38-company-engineering-blogs-to-follow-as-a-software-engineer-c369e4be9afe

https://www.joelonsoftware.com/

http://wiki.c2.com/

No-Code

Lately I've been reading a lot about no-code platforms and the "end of programming", but I don't buy it. Not just because I'm a software engineer and want to keep my job, but because it's just another bubble. We've seen this happen before, in all sorts of fields.

First, as I wrote last year, no-code is just another domain specific language (DSL), albeit a visual one. And done well they're incredibly useful. LabView and Mindstorms are great examples of them. Any visual form builder is another. If you're building something visual then doing it visually is the way to go. For well understood situations where customization is limited then it makes a lot of sense.

Second, to use an automotive analogy, when cars were new you needed to be your own mechanic. You had to fix/fabricate things as you needed them, and only people who could (or could afford to have full-time people to do it for them) had cars. This peaked in the 40's and 50's when the dream was to get an old Ford and make it better than it ever was. Today, cars are (mostly) commodities. The majority of people in the US can operate one, but fixing/modifying them is left to the professionals.

Third, someone needs to build those visual DSLs. Doing one of them well is non-trivial. They take time, effort, and a deep understanding of the problem space. And even then, it's best to have an escape hatch that lets people actually modify what's happening.

Back in the Bing Maps days we had a tool called IPF, the Image Processing Framework. It was designed and built for embarrassingly parallelizable tasks, such as image processing, color balancing, image stitching, and mesh building. You could define steps and tasks within them, and the dependencies between steps. It was the second gen framework, and one of the things we added was a visual pipeline building. Since it was Microsoft the tool for this was Visio, different shapes represented different things, and nesting them inside each other and drawing lines between them managed containment and dependencies. And you could add any custom or required values to any shape in the form of a note, and they would get translated into runtime parameters. Things like required cores/memory, timeouts, retries, and what to do on failure.

And it was great when things were simple. When blocking out a pipeline it was great. Put in the big boxes. Draw some lines. Set the overarching parameters. Move things around as desired to see what happened and if it could be made more efficient. We did a lot of that. And for pipelines of < 10 steps we made quick progress.

But as things got more complex and the pipelines got bigger it got pretty cumbersome. For many things we ended up falling back to the XML the visual tool generated for details and just used the visual tool to move the big blocks/dependencies.

There's no question no-code and visual tools have their place and we should use them in those places. You could get all the info presented in RobotStudio from tables, graphs, and images, but playing time series data back visually is just more efficient and makes more sense. Add in the ability to swap out a node in an AXL graph and run the simulation again and see what happens can be a magical experience. That's no-code at work and we should definitely do it. But to get to that point requires a lot of code, so I don't see the end of coding any time soon.

SESE

Single Entry, Single Exit. Is it a Draconian or outdated rule? I'd argue that tools and languages have changed enough that for what most people do, it's more of a guideline that should be part of separation of concerns.

When Dijkstra first wrote his paper on structured programming things were different. In assembly GOTO was one of your only branching mechanisms and you could go anywhere. Jump into the middle of a routine. Jump out of a loop into some other block of code and never come back. And when he talked about single exit it wasn't so much having one return statement, but always going back to where you came from, not going to some other place and continuing execution. So at that time ALL structure was the responsibility of the developer.

In today's languages those things aren't really possible. Pretty much everything is part of a function/method, and it's approximately impossible to jump into the middle of a function. Some languages have some form of exception handling for not returning to whence you came, but generally you go back up the call stack until something handles the exception. So how does SESE apply today?

It still applies because the underlying desire is to help the reader understand the code. If there's one way in and you always go back to where you came from that's a big reduction in cognitive load. And we still want/need that. We always want that.

But that doesn't mean there should only be one return per method. In some cases having multiple returns can make the code easier to understand. Guard clauses, or input validation is a great example. Validating the input up front, with individual checks for each input and case, gives you a logical SESE for each check. It makes it easy to see what's been checked and lets you return (or throw) specific info back to the caller. Then, when you get to the meat of your method you know that the inputs are at least mostly correct and you don't need to worry about them, A good check for this is if you find yourself indented halfway across the page when your doing the actual work, you should probably have some guard clauses and early returns.

Similarly, there's the finally or catch clause at the end of the method. Sometimes there are things that need to happen on the way out of a method, regardless of whether or not there has been an error. This is very common with languages like C# or Java where you are dealing w/ native resources and need to manage them yourself, but also other things that don't have that kind of interop. One of the most common causes of deadlocks in multithreaded code is having a return path in a method that doesn't release a lock. A single return path makes that much harder to get wrong.

So while the notion of SESE, as written 50 years ago don't strictly apply, the reason for it is still important and there's a lot to be gained by following the principles instead of being Draconian or ignoring it completely.

Dependency Injection - NOT

I first heard the term dependency injection (DI) a couple of years ago when some folks were talking about UberFX and how DI was the hot new thing and would save us all. And I was very skeptical, because why would I want someone injecting random code into my process? Code injection is an attack vector, not a design principle. Why would anyone build a system around that?

But the thing is, Dependency Injection isn't actually injection. And it's certainly not the injection of random code without the developer's consent. I wasn't in charge of figuring out the name for DI, but if you were to ask me, I'd say they got it almost completely backwards. It's not "injection", its "demand" and "consumption". Instead of finding/creating the things you need, you just tell the "system" what you need and it hands it to you.

There are lots of advantages to this method. First, you're programing to bare interfaces, not specific implementations, so separation of concerns is baked right in. That's a big reduction in cognitive load right there.

Second, whatever system is providing you with your dependencies can update them and the next time you start you get the update. That's great for bug fixes, vulnerability batching, feature updates, and infrastructure changes. As long as the interface doesn't change the calling system can make whatever changes it wants.

Third, automated is much easier. When the system under test is created, the test framework can just hand it special implementations of the interfaces that have additional testability hooks. For instance, the infra CLI accesses both the network and the local file system. When testing the network and FS interfaces passed in provide all kinds of hooks to let the tests control the results of calls, but to the infra CLI itself, absolutely nothing has changed.

UberFX (and JavaFX I'm told, but haven't used) goes one step further. It includes a bunch of standard interfaces (network, logging, metrics, etc), and you can add your own interfaces, each with its own dependencies. Then, when you try to start the program it analyzes those dependencies and builds an ordered graph of the order to call them and then passes the results on to the next layer as needed.

So despite its terrible name, DI is actually a useful system that enforces separation of concern and control inversion. The next question is of course, "If DI is so good, shouldn't we use it everywhere and stop linking against libraries?", but that's a question for another time.