Dealing with eventual consistency

00:03 Dennis van der Stelt

All right, welcome, and it's already quite late, so I hope you're all up for it. You've been watching to this slide already a bit, so you know who I am. I work at particular software. We have a booth currently here, so if you think you didn't have time to ask questions or you came up with anything, visit me at the booth. I'll be here the coming two days. I want to talk about eventual consistency, and for that I want to use a small story that I've experienced and what made me create this slide. And this picture that I'm currently showing, it's been a while, but do people still know where it's from? And I see some heads nodding, but-

00:50 Audience

Game of Thrones?

Game of Thrones, yes. It has been a while and I still miss it and I don't know about you, but I was majorly addicted to it. But in the Netherlands, the last episode was the idea to display it on movie screens. So, that was interesting to me and I would love to see it. And they put out this message, which is obviously in Dutch, so I suspect that most of you can't read it, but there were some peculiar things in it. In the message it said, "We are going to try to sell the tickets," and what they also said is you can order them for free. So free is always good, especially in the Netherlands. And then, the most interesting part came and they said, "We are not going to announce the exact time when the tickets will go live because we are afraid that our website will crash."

Guess what happened? So their website majorly crashed. Luckily I was able to order some tickets, so I was one of the lucky few, but they also send out a tweet also in Dutch saying that tens of thousands of people have issues ordering tickets, not just for the Game of Thrones finale, but for every other movie out there because the entire website broke down. Even though they, and that's also another interesting thing, they used more server room space than during Avengers Endgame. I don't know what getting a bigger room helps in this, but I'm assuming that they were scaling out the web servers and it didn't really help.

So as developer I said, well, there are solutions for that. Now no-one can join or access your website and order tickets. If you want more information, come and direct message me. So I hope someone from the movie theater is in the room or watching this later. Then maybe we can help each other because the movie theater Pathé in the Netherlands also isn't the only one, especially during Black Friday. You often see, or just after Black Friday, you see pictures like this. The right bottom one I took myself, they basically had a virtual guard because they said, "The store is full of people," which is ridiculous because it's a digital store.

Another thing was that they had this waiting time and they would predict how long you would have to wait before you were allowed in the store, but the store was so busy that even calculating the waiting time crashed. So, it's always a good thing to look in the past and how come that we came to this point where so many websites struggle with a lot of people coming to their website and just scaling out the web servers or increasing the room that the web servers are in, doesn't really help.

But we've had similar problems in the past where the business always found a solution. So I imagine before computers, there was someone who would pick up the phone and call the movie theater and say, "I would like some tickets." And the person on the other side of the phone said, "Sure, no problem. How many would you like?" But phones or the phone computers, I don't know what to call them, they would queue up people until the phone was red-hot and the queue was way long and people had to wait for a too long of a time. The A company would say, "Well, why not increase the number of people that pick up the phone?"

Which caused another issue because if we had the last five tickets for sale, then the first person said, "Sure, I can have four tickets." But the second person said, "I want three tickets," and there's only one ticket left. But like I said before, the business or companies would always find a solution and one of the solutions for example, is to have one person that picks up the phone only sell even rows and the other person only un-even rows. And these days where you have stadiums where 40, 60, 80,000 people can get in, you don't sell seats or whatever, there's just, what is it? Seats A and seats B, and it's like a predefined set of seats and you can sell per window or per person, different kinds of seats.

Maybe VIP seats, and you get into a different line. So they always found solutions, but then the internet happened, and there was probably one more and more smart developers and they said, "I want to sell movie tickets and I'm going to build a website, movies.com or whatever it's called, and I'm going to use a web server, some database, a shopping cart, and all will be fine." And people would be able to order tickets online for the very first time.

Now, initially, you would have rows of data where you would sell, I don't know, Star Wars Rise of Skywalker and Game of Thrones tickets where the Game of Thrones tickets are free and you would have a transaction where you would first receive or query how many tickets were left, and then if the number of tickets left in the store or in the database was larger or equal to the amount of tickets that the customer wants, then you update the movie stable and decrease the quantity of the available movie tickets by the amount the customer ordered.

And initially this was blazing fast because this transaction only takes, I don't know how few milliseconds, and we can serve a lot, a lot of people. Besides the fact that a lot of people is like a relative, if it becomes a million people or in the case of the movie theater, I wanted to go to, tens of thousands of people, there's also another issue and that is what if you want to order movie tickets for only one specific item in the database? Then you get single-item congestion, and a lot of people they want to scale their database out and for example, create a cluster of SQL servers or whatever database server you're using.

Sometimes we shard data, so everything from 8F goes on server one, et cetera, et cetera. But single-item congestion is very, very hard to solve because we might have 24 database servers and only one of those database servers gets hit hard and not just the entire database server, but exactly one row in the database. Besides that, more and more features get added, which often means that we place locks on the rows, which means that someone else wants to query the database, finds that it's the thousands in a row of transaction locks that it's trying to reach. We get transaction timeouts after 10 or 20 minutes, so the more features are being built, and probably everyone experienced that sometime and we sometimes lovingly call it a monolith where it's very hard to solve this problem.

We are not the only one obviously who tried to solve this problem, so other people thought about it as well. But let's dig a little bit deeper first into the relational database before we look at some alternative solutions. The relational database historically doesn't just when you query or update a table, update just a row in the table, it also updates the index and the index takes way longer depending on a lot of factors to update and especially when you start creating or scaling out your database servers and creating a cluster. I've once worked for a company where the DBA, consistency was everything to the DBA, so when he started scaling out, he was spreading out data over multiple servers, but he insisted that the index of the data was complete and within the same transaction as the initial transaction for every single database.

So even though he was scaling out data, it just took longer and longer and longer because more indexes had to be updated. Another solution where you don't have as big as a problem with the indexes is document databases. Not every single document database and relational database works the same, but document databases, if you update the documents, they update often the index asynchronously, which means that inserting data can seem to go much faster. But the problem is that when you then query the data, you are making use of an index and it seems as if you didn't insert the data yet, especially when document databases just came out.

It's already been quite a while or let me rephrase that. When they become popular, more and more people figured out that, hey, the indexes are sometimes pretty stale, especially when you insert large amounts of data, which became an issue because the data was in the database, but you weren't really able to access it because of the asynchronous updating of the index. Another thing is that we try to shard data or spread data over multiple data centers, and if one is in North America, what people sometimes forget is one of the fallacies of distributed computing where they think that bandwidth is infinite and there's no latency, but even data across the world travels at most at the speed of light, which isn't always really fast. Which means that again, it takes a while before data is on the other side of the world. And I'll get back to that in a bit.

An alternative solution that I like a lot is the one where we use messaging and what happens there is that we basically embrace the asynchronous way of communicating our data to the backend or across servers or however you want to see it because when you say, I want to submit an order or I want to order a movie ticket, you submit a message onto the queue and then the web server for example, goes on with its life without immediately expecting an answer back. The message is in the queue and on the other side there's a backend server which picks it up, store something in the database, maybe publishes more messages that flow through the system.

But also here you might be searching for data that isn't there yet because the backend server didn't process your message fast enough. And that's not an issue because what the movie theater that I ordered ticket on had as an issue was that everyone was trying to query and update the database at the same time they were ordering tickets, the system was placing transactions on it... And that's a lot of assumptions I make, but it's not the first time that I see that happen. So I don't know the exact details, but I can kind of imagine what's happening there. The thing is that it's often difficult to live or develop with the fact that you don't immediately get an answer back. So that's asynchronous communication.

And this strip or this comic is already pretty old and it was originally in French, but someone created some English text and you can see that there's a kitten in the tree and grandma is, she doesn't know how to get it out. Mr. Invincible comes, but instead of picking it out of the tree, he just grabs the kitten from the comic below and then comes the curious part because he gives the kitten to grandma, and grandma then sees two kittens. And Mr. Invincible says, "Yeah, that's no issue, it just needs to sync up again." And then in the last part, suddenly the kitten is gone. That's basically how you might experience eventual consistency, and I thought I had to add it.

So let's look at how to deal with it. The problem with eventual consistency and the fact that there is consistency, but in the database and the way we look at it from our code is eventual, because maybe some databases are replicating the data among each other or there's messages in the queue or indexes haven't been updated and we have to try and deal with that. But the problem is that often in our code, in our applications and systems, we are reading data that we were just trying to write. So for example, here I have an example from the same movie site that I'll show you in a minute because I rebuilt it for if the movie theater is actually watching, but here's an example of when you can get a subscription.

I don't know if that's something that works like this in other countries in the world as well, but in the Netherlands where I come from, you can get a subscription, different types of subscription, and you get major discounts on the movie theater. So for example, you could pay like 50 euros a month and then you can get to as many movies as you want and you get nice discounts on popcorn and I don't know what else. The form on the left, you can see the submit button which says, "Get subscription," and oftentimes in the past when people submitted it, we immediately moved to the next screen and showed the data that user just submitted, like as a summary or verification or whatever.

That next screen retrieves the data from the database and thus has a problem if the data isn't yet in the database or the index isn't updated, et cetera. So this is one of the problems when you are not reading your own rights. This is for example, a good example where some back office person adds a new movie to the website. When the back office person does that, it's not an issue if the movie doesn't appear immediately on the website. If it's like one minute later, it's not the end of the world. Obviously, you have to take care of the fact that you're not placing a movie in the list of movies and then when you click it, suddenly the details page isn't available. But yeah, that's another kind of problems when not all data is updated.

What you can also do is instead for registering for an account or for a subscription, say we've sent you an email and you have to click a link in the email so that we can verify that you actually own this email. It could be that that is a requirement of the website, but it's also a great way to postpone the rest of the process for a couple of seconds until data has the time to synchronize over your system. I don't know if that's something that companies do, but it's one of the options that you have to postpone the fact that you immediately read data.

Here's the subscription again. So another option here is thanks for your submitting your details and then a link, take me back to the movies. So in other words, we are not going to the details of the subscription. The user then has to click like his profile picture at the top and then go to the subscription and then maybe you can show something like, "I'm sorry, but we haven't processed it yet." But that's another option to not directly go to a page where we show the details. Another thing is what you sometimes see, "We are reviewing your changes and this can take a few minutes."

That's also a very cheap way to go about it, and one that I don't particularly like is when they refresh basically or they clear out the form that you're filling in and then at the top they show this green bar or whatever color it is, "Thanks for submitting your details," because sometimes I'm on my mobile and I filled in a form this long and I press submit and feels like nothing's happening, and I press submit again because that's what users do, and suddenly validation errors go off because the entire screen is empty and I panic because then when I scroll to the top, I see, "Thanks for submitting," and I'm like, "Oh, okay."

Another great example is this. Probably everyone is familiar with Stack Overflow as developers. There's this comment section at the bottom of each page or at least of all the questions where you can give your answer or give a comment on, for example, the question or somebody else's answer. And once you hit the submit button, it's instant on the page. Right? What they're not showing is that they basically have this diff or whatever they're using which was initially invisible and maybe they added dynamically, I don't know, or sorry, this is the text area and the button and it's visible. And once you press the submit button, they make it invisible and just add the text that you tried to comment at the bottom of the already comment list there. And then in the backend, which is initially the server, or sorry, the browser doing some JavaScript and it's submitting something to the backend, and then in the end it's trying to retrieve, for example, all the comments.

And if some of the comments haven't been processed by the backend yet, you might store the comments in the front end because it's not a lot, it's just your comment. All the other comments come from the backend. And then when you press F5 and refresh the screen, you can go, for example, I've used a repository here, load the comments from the repository and then verify if the comment that's in local storage is in the list. And if it isn't, you add it, and if it is, you just remove it from local storage. So, I told you that I've built the website from the movie theater and I wanted to look a lot like the same. So I created this hover over effect where the background changes and everything else, and it took me a couple of hours to build the website and then it cost me like a week to build this graphical effect.

But here you can see Game of Thrones and I can click reviews, and here you see four reviews and they're all by me because no one else can access this website. So it also helps with performance issues if I'm the only one that... Because otherwise maybe I might fail in my presentation, but there are four comments on the website and can anyone guess which one are from local storage? For example, I can, "Hello... Oops." "Hello NDC," and then submit a review and it's instantly there.

22:20 Audience

All of them.

All of them. No, the top one come from the database and I just pressed a link, clear local storage, which means I delete all the comments that were in local storage and I can keep submitting them, but I don't submit them yet to the database because that would basically defeat the purpose.

I can press clear local storage and even I wouldn't know which ones came from the database. Maybe it's already in the database, so I never actually submit them to the database, but I only store them in local storage. And the same thing for any other one. Submit review, boom.

So what I did here as well, the same thing basically a Stack Overflow does and probably Facebook. A nice way to test that is setting up a VM virtual machine somewhere in New Zealand and then going to Facebook or Stack Overflow there, and then here in Norway or wherever you live, submit a response and then refresh the screen on the other side of the world and see how long it takes before the answer is on the other side. But what I did here is first I took the comment field on the submit review button and click, I took the comment field, created an actual review, stored it in local storage. Obviously, I cleared the comment field so that someone else, or another comment can be added, and then I simply appended it to a list of comments that was already there.

Options for storage, there's first the browser, there's a session storage, which means if you close the tab or the browser that your data is gone. Local storage is oftentimes maybe more usable. IndexedDB is an actual database inside the browser, so you can query your data. It's probably not necessary for most scenarios. You can also store it on the web server, obviously in cache or in distributed cache like Redis or even have each web server, have its own local small database. If you load balance your web servers, then you probably need to use sticky sessions so that a user always ends up in the same database.

Redis is probably the best because if you have a scaled out web front end, then Redis can also be distributed if you need more query power and it allows you to store stuff really, really fast and retrieve it. Benefits of storing it on the server is that not just you can see the comment before it's processed and into the database. Everyone else can retrieve the comment as well. You can also store much more data and distribute it over multiple servers and you have better access over what's there. I mean, with comments it's not that important, but it could be that people alter local storage because they can simply access it as it's on their own computer and then make use of the fact that there's invalid data in there. And obviously you can also store it somewhere else on a database.

Another thing that you can show, which often people don't really like is this rotating spinner. I don't know if the rotating spinner is the same thing, but whatever. And even this spinner. The funny thing, or I think funny is whenever I create a virtual machine in Azure, I see all these dots continuously rotating and everyone accepts that it's taking a long time because yeah, setting up a virtual machine, no one can do that within five seconds, so it's okay to wait 10 seconds or half a minute or something.

The thing is that users in general also have no idea how long it takes to submit some data or purchase a movie ticket or whatever. So usually they're also okay with some hourglass or whatever, if you show them, to let them know that it needs to take a while before you store the data. Another thing is, you can give the command to store something and continuously while you're showing the spinner, query the backend if the data is already there; but obviously that's a lot of queries to the backend, et cetera. There are these days better options like SignalR or gRPC. The one I built in the demo website is this one. Whenever a user submits some data to the backend, I set up a connection or it's already set up, but basically I set up a connection using SignalR to the backend server, which is basically ASP.NET.

And there I put a message inside the queue and I immediately on the proper occasion, report back to the user that something happened. And then, the backend server starts processing the message asynchronously. Stores, for example, something in the database, retrieves some other data and reports back using a message. And then the ASP.NET web server reads the message from the queue and reports back via SignalR to the user that something changed. And this is for a subscription for example, where initially I said, "Thanks for replying." And then once the message came back, we provided some additional information.

And I can show you how that works on the website because... And I first have to, it's already running, I thought whatever. So I press run, and in Rider you can see that this is the website that's running, and here you can see the server, which is basically processing the messages. So if I, for example, do a Top Gun Maverick, and I want to say I ordered tickets, and here are some movie theaters in the Netherlands, and this one is the best because it's very, very close to Feyenoord Stadium, my favorite football team.

I select a time, I say two tickets, and it waited for a very short time because this was the first message that I sent, and with the back end and the front end need to boot up, et cetera. So, I ordered a ticket. Now, when I shut down the backend, because for example, it's very, very busy, I order tickets and I explicitly made two versions of this. This is I order three tickets for Maverick, and now I can wait forever because I put a message on the queue and that message is still on the queue and it's never processed, right?

Oh, and you can probably still see here in the backend. Yeah, the first movie I ordered was also Maverick. Here you see order arrived for a movie, yada, yada, yada. Then I shut it down. So the second one never arrives. Now, going back to the problem with Game of Thrones, when I ordered the ticket, I had browsers open on my mobile, on my desktop and on my laptop hoping that one of them would figure out with continuously refreshing the page and going to the next step how to get tickets to the finale. And luckily, I was one of the people that got through, but it's basically just a lottery.

It's at random people who get through the entire process with continuously the browser saying that the website isn't responding or that half of the data isn't loading or it's giving me some error. So, why not do it differently and say, "I want to go to the Kuip again," and there was only one time, and let's order four tickets. So, now we can order tickets. And even though the backend is down, the immediate reply is, "Thanks for registering." And instead of actually making sure that consistency is perfect in the backend, because we might have, I don't know, 500 tickets in a movie theater in a big one, and we might now sell 50,000 or might not sell, we might register 50,000 movie tickets; but some people order one ticket, some people order four tickets, and now there are lots of messages in the queue for everyone who registered and everyone says or gets a response saying, "At some point in time, we'll have a lottery or a raffle and select a few people that get tickets."

And it's ideal because you know exactly how many tickets everyone ordered, and you can place them very slowly and process all the messages slowly or store them initially in the database and then figure out who gets what seat, et cetera, et cetera. But the user doesn't notice that the backend is having a very hard time processing Game of Thrones messages. So now if I run this thing again, here you can see this was the Maverick movie and this was the Game of Thrones movie. So, if a movie theater decides, probably Avengers Endgame and Game of Thrones will be very, very busy, we can decide on other functionality features, other ways of selling those tickets and making sure that the website doesn't crash.

So, all the code is in GitHub, and I'll share it at the end of the presentation, a link where you can get more information about it. But here you see the submit button for buying the movie tickets. I set up a connection with SignalR. Obviously it doesn't have to be here, because then every press opens up a new connection. You can do that beforehand, but you get the idea. Then when an order submission SignalR message returns, in acknowledgment, some diff or whatever, I want to display this message. So I made it for myself, very simple here. There's some HTML coming from the SignalR hub into this JavaScript part, and then I serialize the entire form so that it's one string and I can submit it via SignalR to the back end.

Then just like, I don't know how many of you know SignalR, but just like in the back end, just like with a controller, SignalR has also kind of its controllers, but it's called a hub. And here you can see the ticket hub and I first inject an iMessage session in the constructor. That's an NServiceBus thing. So at particular software we built NServiceBus, and I won't bore you with all the details, but I use an iMessage session to send a message and put it on the queue. And here you can see the submit order that was initiated via SignalR, from HTML, from JavaScript. And here you can see I do messagesession.send. And then in the case that it's a lottery, I reply to the sender so that I can immediately provide some answer or whatever.

So at this point, the message is in the queue. Then on the server we have a message handler, the submit order message, and I register the order, and then I do a reply if necessary of some information that I retrieved that the purchase was successful, et cetera. And now again, we go from the server in the back end to our website again, where our SignalR Hub is, also via message on the queue. So here you see another message handler, but this time in the website we inject the MovieTicketHub, the SignalR Hub again, so that we can use it when we have the handle methods and our order submission messages pulled from the queue, we can then use the TicketHub context to send back to the user the fact that their purchase succeeded or whatever.

So this is a technical solution and I've provided some easier ones already on how to deal with eventual consistency. And this tries to mimic the fact that we do asynchronous communication or some other stuff is asynchronously updated. And this is just an example of how we can report back to the user and make the end user believe that a response was immediately.

Now, if the website gets very, very busy, we have to decide, do all of them go over the queue? Do all of them act like they're a lottery, et cetera? That's up to the business. And like I said at the beginning of the presentation, in the past, the business always found a way. And I think we need to discuss more with the business, not just decide on ourselves, which technical solution we choose, but talk about it with the business on how we can embrace eventual consistency on solving problems.

Here's for example, another one where we might get single item congestion issues, Winds of Winter. Does anyone know what it is? Yes.

37:30 Audience

The famous awaited book?

The famous awaited book, yes. So if you've watched Game of Thrones, George R.R. Martin hasn't finished his books, but Game of Thrones has finished and fans were furious about the quality going down, et cetera, et cetera. But I blame George R.R. Martin because I think he's taking already 10 years to write this book if it's not longer and everyone is waiting for this book. So again, when this book comes out on Amazon or I don't know what else, I am assuming that at least a thousand fans will order it. I have no idea how many people order this book, but probably quite a few. The thing is, we run into single item congestion. So how we can solve that is by not embracing or not having strict consistency but letting go of the strict consistency. But that is an issue.

We can, for example, do this. Only if people order five books or three books or four books, only store how many we sold, no matter how many we have in store. And then for example, if we add 250 books to our inventory, again, we add 250 to the Delta, and then at some point we can take a large number of records, count, do a sum of all the minuses and the pluses, smash it together and store the number as, I don't know, plus 180 or something. It depends on obviously what the sum of all of them is.

The thing is, first of all, this is basically where event sourcing came from and event sourcing done simple, we're only storing the events basically. They're very simple, but the benefit is we don't have to do locks, right? We don't have to lock an entire row, update the inventory, release the lock, while there are tens of thousands of other people waiting to do the exact same thing. Something that databases aren't very fast in and people keep waiting and waiting.

The downside is that the inventory is eventual consistent, which means we never know, if people keep buying the book, we never know what our inventory is because we query it from the database, but while we query it, other rows are added to it already. But it's kind of the same if we do think we use consistency in our database because we query the database and we might even place a strong lock on it when querying it. But then when we display it in some client interface, we say there are 30 in stock, but it could be that 25 of them have already been sold in the backend. So either way, it doesn't really make a difference of what our inventory is at an actual moment in time if we just want to display it or maybe figure out if we need to order books or not.

But we're solving the single item congestion with this and we can still at any point in time do a sum of the current total. And if we figure out, apparently this book is very popular, but we have less than a thousand left, already then do we need to order? And this is used more often. I don't know if anyone has ordered the Steam Controller, but Steam thought this was going to be a success and it wasn't really. So they wanted to get rid of the Steam Controller and offered them for $5 a piece on Steam, and I ordered one and I was successful. So I was very happy and I was waiting in front of the mailbox for it to arrive when my phone burst and I got a nice message saying, "Due to technical difficulties, we sold more than we had in store and unfortunately you are not getting any."

So if we look for example at Amazon.com, how important is it that all the numbers on these pages are 100% accurate? For example, there are 42 customer reviews. Can I see hands from people who ever checked if the number of customer reviews was actually 42? So probably this isn't a very important number to be accurate. If someone is submitting or lots of people are submitting comments and the number isn't up-to-date with the actual comments, it's not that big of an issue. What about this number? The fact that the book costs $42? Who thinks it's important that this number is extremely accurate?

Most of the people. Who thinks it doesn't really matter how accurate this number is? Oh, one person in the back.

42:41 Audience

His salary is too high.

I don't think it's very important that it's accurate, but there's a reason for that because imagine if I'm browsing a store and I put a book in my... No, let me take a step back. I'm browsing the store and while I'm browsing the store, someone in the back end or in the back office add a 10% discount to the book, but the user browsing the store doesn't see that until five minutes from now. Is that a major issue? If the price in the back end changed, but the user didn't see that, as long as we make sure that if the customer sees $42, they are actually paying $42 and not if we increase the price, suddenly he pays $50. That would be a bit weird. I think it's more important that the price is accurate from what he sees and puts in the shopping basket. If someone in the back office says, "I'm raising the price of this," but the person already added to the shopping basket and suddenly when checking out the price is 10% extra, that probably would be an issue.

Another one is this. In the past you would always see we have 15 in stock and more and more websites don't say we have 15 in stock, we have less than 20 in stock, which means we know we can't be 100% accurate, but we try to come up with a number that's reasonable. And it's funny because there was another website that were selling flight tickets and at the top it also said, "There are 37 people looking at the same trip as you." Well, as a developer you sometimes look at how they do things and whatever. So I looked at the HTML and I saw, "View notification at random," and that invites to further the investigation. So I checked out the JavaScript for references of view notification random, and it said this, "Math.random times a lot," which is quite funny. So if you are not very accurate with the numbers, at least hide it better.

Another one where the business came up with solutions to make sure that concurrency isn't that big of an issue or whatever you want to call it, is for example, when selecting seats in a plane. These days, more and more airlines, they let you pay for seats and one of the reasons could be not because they need to make money because their flights are way too cheap, but it's also because not everyone at the same time... I mean, in the past you would have that, they would say at four o'clock in the afternoon you can check in and select your seat. And I remember that there were points in time where I thought, "Oh, I need this seat." And it took two seconds before the user interface said, "No, not that seat because someone just took it before you did."

But an option could be to spread the number of people that perform the same action over your website. So first you do business class, then you allow people to pay for a seat, and then at the very end if you have a very large plane or I don't know what else, you decide who gets to sit where, right? That's for the cheapest tickets and it's a model to make money for airlines, but it could also be a way to spread out people so that they don't all access your website at the same time. Instead of saying, "We're not giving away the time when tickets will go on sale because otherwise our website crashes."

This is another example of a friend of mine. He built a website. Oh, it was a Scandinavian website. Does anyone know this website? Happy Pancake? No? Okay, then it wasn't a success. That's not how he told me. But they had a certain feature that required you to pay for it. So I used Tom Cruise as an example. He's 57 and when Tom Cruz decides to pay for more features on this website, and I didn't know what kind of websites he got, so I made his name red and underlined and took off 20 years of his age and I don't know what else, and he has a nice background image.

The thing is, they did the payment in the backend because they knew they sometimes would have issues with communicating with a third API or the fact that it couldn't provide details about certain bank accounts, et cetera, et cetera. So they chose a better user experience over strong consistency because it could be that people were literally able to modify and upgrade their user interface or whatever it was for two days. And after two days, the website would figure out, we are unable to take the money or charge your bank account for the money you owe us.

So they then rolled back the features of this customer if he didn't pay in time, but it didn't really harm anyone. Right? So just like with the Steam Controller, I also got an email afterwards that unfortunately I didn't get it. This is an other way where the company or the website chose a better user experience over actual strong consistency because they knew there was the chance that it failed. So, what I'm trying to convey here is that eventual consistency is definitely not scary, that it's something that is solvable, because I hear a lot of people say, or quite a few developers say, that they can't introduce eventual consistency because the business doesn't allow it, the customers don't agree with it, et cetera, et cetera. I think there are ways around that, and I also think that the business always had eventual consistency baked in. Right? Until computers came along and we had extremely fast transactions and solved a lot of problems for them.

But now we're introducing another set, or reintroducing old problems, and I think it's more important to strive for high customer satisfaction and we definitely should use the business doing that, than giving them a response like this or a nice background or a virtual card and don't let you into the website. That was my presentation. At the bottom you can see the link. If you follow it, you come to a Gist, which is basically a notepad in GitHub where I provide some more details on the presentation, a link to the GitHub repository where you can find the code. Here's my Calendly link. If you want to talk about eventual consistency or anything other related to distributed systems. You can schedule a meeting with me there. It knows when I'm available in my meeting and we can have a conference call. Thanks for your attention. Are there any questions?

Dealing with eventual consistency

About this video

🔗Transcription