Eventual Consistency

Consistency, availability, and partition tolerance. CAP theorem. If you're working on a distributed database then a lot of what you do is guided by CAP theorem. The world is an imperfect place, and networks break. So you have to deal with network partitions. And in the face of a network partition you can't guarantee consistency and availability. That's the CAP theorem in a nutshell. But what if there were a way to, if not break the rules, at least ignore them in some cases?

Consider the lowly status API. Is this flag set? Has this job finished? Is the car door open? Things like that. Assuming good intent and the network is working, _at best_ those are just snapshots in time, The answer you got was true when it was sent, but by the time you got it it wasn't. So you make the call again, looking to see if anything changed, and behold, something did. You get the response you're looking for and get to move on.

That's eventual consistency in action. Ask later and eventually you'll get the correct answer. And you can take advantage of that to make the issues of CAP theorem less of a burden. Because different operations have different requirements and make different promises to the user.

For database writes you might reasonably decide to require consistency at all times. If you can't be sure everyone knows, then don't accept the change. On the other hand, in many (most?) situations availability is key. Getting the most recent answer you can is better than not getting any answer. Recognizing this opens the door to lots of opportunity, from working through network partitions to read-only status copies of your database.

So when you're thinking about the requirements of your distributed database remember to that the requirements for data accessibility and consistency are not consistent for all accesses.

Side note: Both ACID and CAP have consistency in them, but they're not the same thing. That's a topic for another time.