Messaging Patterns for Modern Software Solutions

00:00:11 Irina Dominte

Okay, so welcome. I'm very glad to see how many people are here to listen about messaging patterns in modern software solutions. Catchphrases or clickbaits or click, I don't know, interesting words, right? Modern software, we hear it all the time, but let's see what we have in store today.

Hello again. My name is Irina. I'm a software architect at Particular Software. They have a booth downstairs, so make sure you pass by. They have goodies. And I blog from time to time at Irina Codes.

Well, we cannot talk about modern software without remembering where we started with monoliths. And after a while we realized those monoliths have problems and we started to chunk it down and to modernize architecture. So now we have microservices. Everywhere you look or you ask, people talk about microservices as being, I don't know, maybe the silver bullet of architecture in maybe .NET space or in the software industry space. I must admit, I loved working with monoliths and I'm going to ask you who here still works with monoliths? Okay, awesome. They're not gone. Yay.

We've basically been fooled that splitting the monolith into microservices will solve all our problems. Most of the problems were related to scalability because now we can scale only those parts of the system that are in need of scaling. But the problem is the smaller bits and pieces that we have in our system tend to become monoliths again. And not only that, when we started to split them apart, we realize that their boundaries are not very well-defined and we always need some data from elsewhere or we always need data sent to some other microservices because sometimes we kind of overdid it and our microservices were smaller than they should have been in the first place. And we tend to get things like this where every microservice has a gazillion integration points with other services. The problem is no one told us how small or big is this microservice and we took it for granted and we didn't realize that it will hurt us in the long run.

Well, in order to make two services communicate between each other, you have options, right? And the first go-to option is to have HTTP APIs. Clients, right? You create an HTTP client, you know where you want to get data from or to send data to, and you do HTTP requests through those services, you de-serialize the response and you're done. I admit, I did that a while ago when it was the first time for me working with microservices because I didn't knew better and it was the familiar path for anyone. What do you know how to do, HTTP APIs or RESTful APIs? And you use those because it's simple for everyone. Okay, I must admit microservices are cool until you need to make them communicate and you need to have them reliable.

And then we realized these microservices are chatty. They talk to each other more than they should do and it hurts the business. So most of these integration points use HTTP as a protocol. We soon started to realize that, hey, we have gRPC as an alternative and at least the downstream APIs maybe can be replaced with gRPC, but in fact, gRPC still uses HTTP underneath. It's HTTP/2, but it's still HTTP as a protocol. And these HTTP APIs have a lot of problems, and not only that they have a lot of problems, they hurt us in production when, well, who wants to debug code in the middle of the weekend because we deployed on Friday, right?

Well, at first glance, we're built as developers to consider that everything will go smooth. We called with the best scenario in mind, we implement features with the best scenario in mind. But every single time when we have microservices talking with each other and they know who they are, we have this thing called coupling. In clean code, we heard of a term like loose coupling. The code should be decoupled. We shouldn't have too many dependencies. We should have the dependency inversion and so on. But at the exact time, when two services know about each other's existence, when A knows who is B and B knows what to respond to A, you're coupled. Even though you're not coupled in code, you do not have a direct dependency. As long as you have to have both services up and running to get data or send data, you're coupled. Sorry. So basically that's why your microservice architecture becomes the monolith again. It's just smaller pieces, but it's very well tied together.

So temporal coupling is another issue with HTTP APIs. Would be very nice if we wouldn't have it. So this way when we do chain of requests, and this happens more often than you think, it's not just only one service requesting something from another. That service also requests something from another service. So A wants something from B, B wants something from C, and so on. And what happens is that you need everything to be up and running smoothly in order to get data. Another thing is, when A does a request, it has to wait until C finishes and sends the response back. So the chain of requests needs to return back with a successful status code. This means everything is synchronous. You might lie to yourself that, oh, I'm using this syntactic sugar in code. Async, await. No, no, it's not synchronous, it's async. Look. Well, the protocol on which you run, it's still synchronous by nature. So yeah, sprinkle whatever, how many async, awaits you want. It's sync.

Another thing is about the performance bottlenecks that I wanted to emphasize, even though I might repeat myself. Every resource, every single HTTP call that you make ties up server resources, it ties up threads, memory, IO operations, database access, other API calls and so on. If one of them in the request chain gets blocked, it will hang in there and it will wait for quite some time. And then you'll have to think about different mechanisms in which you relieve resources and that is not so easy to implement.

Another thing just to complete this is if A does a request to B and C, it's not there, it cannot serve responses, it has a spike of load and it's so overloaded that it's stuck, then C won't be able to respond. And what do you do? Implement retry policy, right? Well, yeah, but C will still be overloaded. B will fail just because C doesn't reply. Or if it doesn't fail, it hangs in there waiting for a response. And A, well, because of the other reasons, has the same issues. So we get cascading failures, like it or not. Sometimes when you call other APIs, we might think that we're calling only one single API, but in fact we don't know what's behind the wall. That API that we're calling may call 10 different other APIs and we need to wait for those two.

Some other thing that is worth mentioning is that we do get limited communication patterns with HTTP. This sound familiar? Sounds familiar? Client-server request-response pattern, even though we're fooling ourselves that we're doing async with different constructions or restarting threads and so on, that's the only communication pattern that we get with HTTP. Request response. That's it. There are no others like pub/sub, fire-and-forget and many others that are available in the messaging world. So our options are pretty limited.

What else do we get? Well, when we use HTTP, we get reduced scalability, and now you might want to contradict me and I invite you to do so. You would say, yeah, what's the problem? I'm going to have X instances of my services that's scaling, I'm going to put more resources on those machines. And that's scaling also. Well, yeah, but if you're not in cloud, how do you handle bursts in traffic? If, I don't know, something happened, someone notices that you have a very nice product that is on sale and everyone rushes to buy the product. How do you handle this? You cannot. If you know that you will have a surge in traffic, then yay, you might prepare the infrastructure and everything, but traffic bursts cannot be handled very well by using HTTP calls.

Another thing, if we want to say that we have a serious robust application, we have to use extensive caching to do performance reason. So we might need to identify in our system what are those resources and how many they are that don't change so often. And then you might introduce another layer of caching like Redis caching or, I don't know, maybe in memory, that's the easiest thing to do. We have to use it because it saves up database roundtrips. But if we introduce low balancers, then we have some other problems. Some other infrastructure, small pieces to manage. Someone needs to know how to handle those and someone needs to supervise those. Not everyone is lucky to be in cloud and to click some buttons and allow auto-scaling and blah, blah. Some still run in their own hosted environment with virtual machines and stuff like that. Auto-scaling, sometimes that's not possible to happen.

Well, if I didn't convince you so far that HTTP is bad or the way we're doing things and I used to do things is bad, how about this scenario? Long-running operations. I mean, we know that sometimes our APIs are used interchangeably with RESTful APIs and we said that we have RESTful APIs and we do create read, update, and delete, for most of the scenarios. But how about these scenarios when we have to have long-running operations? For example, we upload, I don't know, documents to be processed, to be read and to CSV files, for example, or PDF to be, I don't know, corrected, redacted or whatnot. If we upload the document with an HTTP request and we expect a response back, that doesn't go well. You don't know how long it will take, and then you are forced to use additional mechanism like, oh, 202 status code, accepted. But then you have to force your client to implement a mechanism in which you give your client an ID and it has to poll and ask you again and again if that document is ready and it is processed, right?

So this scenario with long-running operation, it's not impossible to implement but is not out of the box when we use HTTP, because when we get a document to process, we have to save something in the database or the easiest way that we can do is to save it in the database and we have another background process that looks at the database, extracts data from there, do whatever it has to do, update, saves and so on. So this scenario is not feasible or out of the box at least.

Well, there's another thing, deliveries. Now we use the delivery services to get food, to get orders and stuff like that, and we're pretty sure that we will get those things that we order, but HTTP has no delivery guarantees. When you do a request, there is nothing that guarantees you that the request will be received and processed successfully. You might get back a 200 OK status code or 200 in range status code, but by default no one can guarantee you that the request has been received. More than that, if the API that you're calling is down, you might implement, I don't know, timeout policies. If you do not get a response in 30 seconds, then, well, kill it. You move on.

But from the cold party perspective, it doesn't have a way of knowing what was the content of the request. So that request is lost. Then you might say, I'm going to log the request when I receive it and then you overload the server with writing unusable and not so useful data in different logs and you have no way of looking at a log file, say, oh, this was my initial request, I'm going to extract it from there, I'm going to replay it. That's not working like that. So there is no delivery guarantee when we use HTTP.

Again, I spoke too early about this, but I need to emphasize it. HTTP is synchronous by nature no matter what. So keep that in mind. I didn't have this idea when I started to work with microservice architectures gazillion years ago, and even though you say you might want to use gRPC or other technologies that had a newer implementation of the protocol, it will still be synced.

Another thing, when you decouple your microservices from your monolith or you start to write them from scratch, how do you do this mechanism of service discovery? Because you need to identify inside your system what is available and where are those services that you want to call. So you need the mechanism that helps you discover what's in there. And how do you manage everything if you're not in the cloud? It's another piece of software and infrastructure thing that you have to manage. That adds extra complexity, right? So how about the chained calls? If one of the calls in the chain fails, you would have to correlate which is which, right? So you would have to have some tracing in between based on correlation IDs or other mechanism.

Another thing that I personally haven't considered, when you do an HTTP request and you racefully packed your HTTP client in a library maybe and you start to distribute that across the teams to be used, how do you decide what's a good timeout policy? When do you stop waiting for a request? It's something that you need to add it yourself. It's not out of the box because you know I'm calling you an API, I know you have to respond in this amount of time. How do you measure that amount of time? And if you have a retrying policy, that needs to go on top of this.

So on top of the timeout considerations, then we have to have retrying policies, which is another thing. Polly might be familiar. Now we have another library that includes part of this library in the Microsoft .NET space. Patterns like circuit breaker, bulkhead, fallback, retrying with backoff exponential or linear, stuff like that. To me it sounds like, well, it's more of an OPC part than dev and these are other concepts that we need to learn, but all these are not built in when we do HTTP request and all these are things to be considered. Do I consider this when I first started with microservices that called other services? I'm going to tell you, nope. How do I learn that I have to do this when the production was down? Just because, well, we called with the best thing in mind.

Well, if I didn't convince you so far that HTTP is not the go-to solution, then how do you not use HTTP when you have microservices. Because that's what you want to hear. Well, it's fairly easy because by using message brokers and queues and what is a message broker? Well, essentially is a piece of software that we introduce that knows to deliver things to different other components in our systems and it has delivery guarantees and a lot of other things that we will talk about.

So queues, everyone knows what is a queue, like the most important computer software concept. Well, just to summarize and a quick reminder, a queue is a place just like we are queuing you up to a store to buy something. This is what we can do with our requests pretty much. They work on this principle called first in, first out and basically if you look here, the newest in the queue is at the end and the oldest, it's in the front. We can get the queue length and we can leverage the power of this. Why do I mention queues is because all these message brokers know how to work with queues. They internally have such a concept. So if we were to look at the simple "architecture", we have an ordering client that does a synchronous request to a catalog API and it expects a response, this would be the go-to.

How we would change this? It's fairly easy. We introduce that queue in between. So basically the client will put a message, we're going to talk about messages, but in here our initial request or traditional requests become messages. They have the same data. We can have metadata on top of it. And the catalog that is supposed to respond and look at whatever we want to get from it, it will pick up the message from the queue or based on the delivery models, we'll have a message pushed to it that will process.

And now being in the traditional model, we have to change a little bit the terminology. We won't have a client and server or API and API. Sometimes we might hear the term publisher or sender, which is the old client that we used, and the other part would have subscriber or consumer or, even more, handler. It's funny how us in IT world, we use a gazillion terms to describe the same thing pretty much. But depending on the context we call it in a way, depending on the other context, it's a different thing. So if we're sending a message, we have senders and receivers. If we're publishing events, then we have subscribers. So basically components of the system that are interested in those things that happened.

Welcome messaging. So we talked about the queues, but now we have to think a bit of the history. People used to send messages under different form from the beginning of time. We've seen this in the movies with a love letter written in a bottle that's suddenly someone discovers. Then we started to use carrier pigeons. This again, had no delivery guarantee. We send the message and it become fire and forget, right? But we used a synchronicity pretty much. So we send something and we do not expect at the same time a response. We send it and, well, hopefully we would get a response or the receiver would receive our message. And then we modernize a bit and we attach GPS and stuff to the pigeons to know where they are, but it's still asynchronous.

So if we try to translate a bit, the requests that we know to messages, they're pretty much similar because we would have metadata on a message and that metadata that usually goes in the header of the request, it will go in the header of the message. So a message would have an envelope and the body would be the actual request content, the request body, and it will be in a messaging world the same, a request body or a message property.

So basically so far we write messages or data, DTOs, we do something, we write where the letter should go, and then after all the metadata is written, we go and dispatch that message to something like a post office. Well, the post office is our message broker. What happens from this point on, it's not our job to know. So the post office, the message broker knows where to deliver the message based on the metadata that we added. And not only that, it makes sure that our interested parties, the consumer receive that message. Cool.

What kind of messages we have? Many, but we'll talk about just a few of them. So we have document where we send and we do not expect a response. We just send things to be processed in a queue. We have commands, we have events, we have queries that might be familiar in architectures like CQRS, and then we have request/reply. Request/reply, the traditional one can be used with messaging and queues. So what will happen here is whenever you send a message, a temporary queue will be created. At the other end the consumer will wait and it will respond through the same queue.

Well, events are something that why do I do this and like events and messages and blah, blah and commands? Well, because sometimes there is a fine line between what is a command and what is an event. And I've seen teams that are not very, let's say, familiar with them. So discrete, small, they're actionable, independent. They travel around the system without anyone following them. So, for example, an event is an order created. You send something in thin air through a message broker and some other components in the system might respond to it like, oh, I'm interested if an order has been created. Payment received, again, is an event, another payment received, and this can be of interest to many other components in the system. So if we have these three big components: order processing, payment, and shipping, maybe payment is interested if a payment has been received and maybe the shipping is interested in this event, payment received. Because you might only arrange shipping if the payment has been received, right?

In a similar way, commands are instructional and usually don't require a response. It's something you do something. So an order API when it receives an order, might send a create order command with the content of the request received to an order processing component that is separate than the order API. You see, sometimes when we introduce messaging, we have APIs that are public facing and we use those APIs just to dispatch things further down the stream to these small components that we can scale independently. So we have order processing, payment processing, and blah, blah, blah. But the front is still the order API that is public that receives HTTP requests from, I don't know, a browser.

Well, I talked about all these, but if we're using queues in a messaging system, we get something you might have heard about, load leveling. This is new. Have you heard about this term before? Okay, cool. So load leveling, it's a pattern that you will meet, you'll see in this space of messaging. So what happens with this? When a publisher sends a message to a queue inside the messages system, that queue becomes populated. So now this first event, or event because that's a publisher, is the first in line. Then if we have more publishers that send data, events, those get in the same queue from which are buffered, so the queue levels the load. And the other side, the consumer will consume those events as soon as they are available. So now if we want to consume more messages and have increased throughput, we would spin up many consumers. So we would process these messages faster.

So this way we can have an indefinite number of publishers that publish at an inconsistent rate, but we know for sure what is the rate that we're consuming at. So we're leveling the load. What does this do is to handle those bursts in traffic that I told you that HTTP cannot handle. So you get as many orders, those will sit and not be lost in a queue and we will consume them as a consistent rate. So if we know we are consuming one message per minute, then that's the consistency over consumption. One message per minute.

So then cool, we have issues when we modernize the application. With introducing the messages, we will look briefly at a small architecture. Let's say we have an order service that receives some changes, a new order usually. If we want to save what we receive in a database, it's easy enough, we get what we receive, we save, we do data transformation under the hood and the database receives that record. Cool.

But what we do if we want to save something in the database and also notify other components that we received in order? So let's say we want to notify the shipping component or an administrator or to send an email to the CEO, I don't know. Well, when we publish a message, we suddenly have two systems that we would want, but we cannot, to behave like they're the same thing. How about we would wrap everything in a transaction, the publishing of the message and the saving of the message? You cannot because they are two different systems. One is a database and one is a message broker that has queues. You cannot add suddenly a transaction over it.

So how do we solve it? Well, maybe first you save it in the database and then you publish the message without having a transaction because it's physically impossible to do so and then maybe you say, oh, okay, no, no, no, first I'm going to publish and then I'm going to save in the database because, well, you're going to solve a lot of issues, right? No, you're not going to solve anything because you cannot make sure that if you publish a message that, hey, order was created, the order was saved in the database, the network is not reliable.

Have you heard about the fallacies of distributed computing? Awesome. So there are a set of 11 statements that are true for the development world and one of them is that the network is not reliable. So when we get a new order and we save it in the database and something happens with the database and we cannot save the order, but we publish the event, what happens? The team is preparing the shipping, but they don't have an order in the database because we didn't save it. The same happens in reverse. There are two independent operations and we cannot control and fool ourselves that these can happen at the same time because the network is not reliable.

So no matter what is the order in which we want to do this, the notification and the database saving, we will get to issues. In a particular NServiceBus world, you'll hear the terms ghost messages and zombie records, which are two cool things. So if you save the order in the database, but something happens and the message sent doesn't happen, you have an order that is stuck and the other component in the systems interested in that don't know they should respond, they should do something because they weren't notified.

So zombie records is the other way around. You publish the message, they receive the notification. Hey, new order, yay, happy, happy. You send emails, but you don't have the order data, right? Sucks to be you then. So these two things can happen in the real world because we cannot have this consistency anymore because we have two different systems. We cannot wrap it in a transaction, the notification and the database saving, because they're two systems. So we have to think about ways to have this thing all or nothing that we got previously with transactions. It's either we save and we notify or we roll back and nothing happens, so we don't get zombie records and ghost messages.

So how do we fix this issue? Very simple, but it's very simple if you do not reinvent the wheel and code yourself all the infrastructure and use a reliable platform like NServiceBus or MassTransit. So the outbox pattern comes to solve this issue where we have changes that we have to save in the database and we have to publish a notification. Now what will we do is we would have a table that will contain the exact content of the message in our table. So being in the same system, now it is allowed to wrap it with a transaction. So the content of the message that was supposed to be there will get saved in the outbox table and now it's our job to look at that outbox table and to send it to the parties that are interested. So save the order data and save the message that is supposed to go out to notify other parts of the system.

Well, so far so good, doable, but you have to do a lot of coding. Platforms like NServiceBus and MassTransit, they have this out of the box. I'm going to show you a demo. So what happens after you have the record saved in the database inside the transaction and everything succeeds? Well, there is this outbox worker that looks at the table and makes sure that each message that was supposed to be sent to the outside world to notify things gets sent to the message broker. In this case I use RabbitMQ because it's the most popular, but this outbox worker knows how to publish what was supposed to be published before. Okay, awesome. We solved the issue of the network reliability, but there's still some things.

Messages are not lost anymore. We have everything saved in the database. It either saved their worries, it isn't, but it is consistent. So we don't get this inconsistent state anymore that we used to do before. Two-phase commits are not practical and not scalable. So that's why we cannot use them. But there's a thing. Once you try to process items from the message broker the network is not reliable. I'm going to emphasize this. There are cases when they are resent and messages are retried out of the box and you might get duplicates. So you basically solve half of the problem. You solve the message sending, but now you have to solve the message receiving.

So why use the outbox pattern? Because events are saved in the same database transaction, and I'll let this here in case you want to get the slides and look at those yourself. The source of truth is in the database, so you have the domain object and the events saved in there. And also, it helps us with decoupling for reliable message delivery. Now we don't get those ghost messages and zombie records. Also, what it does is that allows downstream services to process the same message over and over again without having bad side effects. Also, these outbox tables can work like an audit trail for the emitted events. Well, but in the best case scenarios we would want to have end-to-end reliability, right? I'm receiving an order, I'm saving everything and I also make sure I notify other components and those other components are receiving my event. So we only solved half of the problem.

Well, message brokers have a delivery guarantee and I wanted to have a note in here. They deliver it exactly once, at least once and at most once. So depending on the message broker you have or you chose to have, you might have more or less reliability and higher or a lower throughput. And here it is, the inbox pattern. If this was the entire part that we looked at before, now we're switching the perspective to the part that is supposed to receive our messages, our notification. So we're looking only to these things. What happens after the message gets published to the broker? So from the broker to the processing part.

Let's say that now we have this consumer or processing server that is supposed to receive messages from RabbitMQ and we are supposed to, I don't know, do some math to create invoices and notify other services. Let's say we want to send emails, that's a big issue. So RabbitMQ pushes messages usually and then we want to successfully process those messages and notify an email and component. What we have to do now is because we have RabbitMQ and there is a slight chance that the messages will be retried and resend over and over again, we have to deduplicate the messages. So because we have to deduplicate the messages, we have to have our own logic to do that if we want to have barebone code.

Well, after we do all that, deduplicate the messages and so on, we have to acknowledge that we successfully processed the message. Well, a lot of things can happen in between. We might receive the message and acknowledge that, hey, I received the message but something happens and you cannot notify a process crashes, the server crashes, something happens. What did you do? You just lost the message. You lost the message and you told RabbitMQ that, hey, I successfully processed this by acknowledging, but in fact you didn't finish your processing flow. This can happen and it's not so obvious that this can happen because network is secure and reliable, right?

Well, basically inbox pattern solves the problem of these duplicate messages because it does exactly that, acts like an inbox. We all have emails, right? We have the emails in the inbox and we'll look there and we read them. This is exactly what the inbox pattern will do. We'll look at the message received, saved it somewhere and from there it will be processed. So in case RabbitMQ crashes or the processing of the message is not okay, we still have the message in the inbox and the next time when we get the same message it's there and it's deduplicated for us. So the message state is tracked in the database.

And also introduces a bit of history. Think about inventory updates, notification, payment processing. Wouldn't be cool at all to process the same payment several times. Well, bad for business. Or a notification, I'm sure it happened to you at least once to get several duplicate emails just for a reason. So basically this pattern stores the incoming message in an inbox table and processes the message and it records the processing outcome. So, for example, sometimes, not sometimes, always, these two patterns, outbox in inbox, work hand in hand. And sometimes if you hear, for example, particular you'll hear the term outbox, we're not mentioning inbox, but actually it's the same pattern. MassTransit has this very explicitly, it has an outbox table and also has an inbox table. So one acts on the part of the sending and the other one is on the receiving part. So this is how it is and they work in conjunction to make sure that you have end-to-end reliability even though it might not be so obvious.

Let's look a bit at some code. I think I have to stop the presentation. Yay. And it didn't want to be stopped. Okay, code. So what do I have here? Not a very fancy example. You'll have access to the source code, I will push it to GitHub so you'd have the slides and other resources in there. Also, you'll have the same implementation with NServiceBus, but I use MassTransit just because it used to be free. Used to. In case you didn't know, MassTransit is going commercial. So if you just start your projects and you consider MassTransit, maybe have a look at other things in the ecosystem if you still want it to be free.

So what do I have here? I have an orders API with some layered... This is how I like to split code for the demos. I have data that contains repositories and the domains are the exact representation of the tables and services acts like a pass-through layer in case I ever need to add some business logic in there. And I do have a lot of things here. I have commands, events, models, responses. But what is most important is the order API that will be the front for the example and also this admin notification worker that contains two consumers. I'm going to expand this. So order received consumer, order created notification consumer. So both of these look at different queues in case they get messages. So let's see how it looks.

First I'm going to make sure the SQL Server database is clean so I get a clean slate when around the app. Orders API should be the one that starts. And while this is doing what it should do... Yeah, of course. Okay, it's up and running. I'm going to show you how easy it is to enable such a program, such an outbox. So basically if you want to use MassTransit, you add the middleware, you say the naming conventions if you want, and then you specify that, hey, I want to use an entity framework outbox. You pass in the order context. You said that you want to use SQL Server and for the demo purposes I'm going to disable that worker process that delivers the message so you can see them in the database. You can set things like duplicate detection window because it works hand in hand with the inbox. So this is on the part that receives my messages.

So let's see. I'm going to do a thing. I'm going to stop RabbitMQ. Come on, Docker. Okay, so I'm going to use some selects. Now I should be able to see an orders database that has some tables. Those tables are either domain ones like customers, order items, orders, and the other three that I didn't highlight are as part of this outbox. So I'm keeping state in there. So if I select them, you'll see these are empty and let's push an order. Where are you? API order, product ID. Okay, so lifetime supply of rubber duck chickens. These are two products. So if I send the request, nothing changes, but we get this 202 accepted. This means that I have no guarantee that the order is in the database and everything is notified.

But if I look here in the orders API controller, I see that I call the order service with an accept order method and if I look at the code, you'll see that I'm doing the exact same thing I showed you in the slides. Come on, show me. Anyways, here, I'm going to try to minimize this. So basically what do I do? I get whatever I send, I save it in the database. I do not validate. By the way, everything is good, best case scenario. Then I publish an order receive event and an order created event. So other than that, I save it in the database and that's it. I don't care who listens to these events, they will react if they will receive this.

Okay, now let's look in the tables. So we have the orders. I don't know if it's visible, but I do have an order, not validated. Good, everything is fine. And if I look here in the outbox-related tables, I will see some records. So the first is the outbox message. Outbox message, yes. And you will see that I get a sequence number. So this is the order in which those events were sent. I get some timestamps and then I have some properties. In here, we'll see that... I'm going to expand this. First is a contracts events order received event. And the other one is order created event. I get assigned an outbox ID number. So these are part of the same "transaction", the same pack sort of. And if I look even, you see that I get a message body, a message type. So I get two things. I'm notified with two events.

So now these are not sent, but are saved, which is very good. Along with my order I save the events. Remember that I closed RabbitMQ and I do not have a message broker, so my Docker instance is stopped, means that if I didn't have the outbox, this event published would have been lost. So I would have an order saved, but I wouldn't have notified anyone. So now what will happen is if I go ahead and try to access RabbitMQ, I should get nothing because this is not up and running. And as soon as I started, in theory, if my laptop cooperates. Yay. This is the RabbitMQ management UI. And in the queues you'll see some things that are running, a lot of other queues that are not related to this specific example. But if we look again in the tables, those are still waiting for me to get them. Why is that? Well, because this is how MassTransit works, you have to have a consumer up and running in order to process the things.

So let's start the components that are interested in my events and that is admin notification worker, not before I add the breakpoint. What are you going to do? Anyways, let's see. I'm going to start this. Okay, as soon as I start the component you'll see that I pushed the message and I can expand and see what is the content. So if I look in the message property, I have created that order ID and some other things. But the most important bit is that I can look at the receive context and you'll see that we get it from the RabbitMQ receive context. This means that if we now look in RabbitMQ interface in those two queues corresponding to my consumers, we get one in one message. So these are currently under processing.

And let's see what happened in the database. They are not in the first two tables anymore because I just delivered them to RabbitMQ, so I cleaned the table. But I do have it in the inbox state. I have message ID, one of them at least. It was received as this, you see the received count is zero. So it's the first time I'm receiving that. The sequence number, which is not yet there. Let's continue. Okay, transaction order received. Bad order. I have an exception somewhere that I manually throw in just to show you how it looks like in there. You'll see that I'm not losing data, messages go to die in error queues. So you are not losing messages even if you cannot process them and an error happened in your code during the processing time. So if we want to see what was going on, you just go there and look at those messages, you'd see that you have the stack trace and down below you'll see the actual content along with some other information that might or not be relevant to you.

And if we continue and send a new order, 202 accepted. And now in the database, let's see what we get. See, for a brief moment these are saved to events that are waiting to be picked up and put in the RabbitMQ interface. Let's see. Now if we look at the receive context, you'll see that outbox receive context is not RabbitMQ anymore. So what it does is inbox pattern is working and it looks at the tables and saves the items over there in the inbox table. Okay, so F5 is another one and so on. And now if we look at this, we should see two outboxes successfully processed.

Okay, saga pattern is the last one that we should talk about. What I'm going to do meanwhile is to delete, to open a new solution, and clean up the tables and the database. So bear with me for a second. Saga pattern. Who here uses sagas? Okay, cool. So saga pattern... Just a second to open Visual Studio. Now my laptop wants change with Mac, I think, and lately misbehaved, that's why I opened two individual solutions and not both at once. It seems to be opening. So let's look.

Well, the saga pattern is very useful when we have several microservices to coordinate and we have workflows like from order we should go to payment, from payment to inventory and from inventory to shipping and some other components. Because as you might notice in microservice world when we use messaging, we suddenly have a lot of other moving parts that we do not control. So what the saga pattern can do is to roll back operation in case something happens. We can have compensating transactions, we can cancel payments, we can refund payments and so on. So this way the sagas ensure reliable, asynchronous orchestration.

And saga is a generic term, but there are two types of them. We have orchestration saga and choreography saga. And depending on the scenario that we might have, we might use, for example, choreography if we want to couple the independent microservices with minimal coordination. Orchestration is just like a puppet, you control everything, puppet master, you control everything. You send events, you respond to events and so on. So if we were to look at how a architectural diagram would look like in orchestration saga, we would have a payment service that invokes different other services. In choreography, we just send events across the system and every individual component knows what to do in case of events that happen. So just like here in orchestration, you did need someone to coordinate, in choreography, everyone knows his place and knows what they should do.

Okay, let's look at an example. This would also be available if you want to run it. I will start multiple projects because we have many components. We still have an order API, we have saga and the saga state that needs coordination. We have billing administration worker, order creation components, so many other components that are somehow closer to real life. I didn't clean anything in the RabbitMQ. I think that's the issue. So that's why I got a... Okay, so now I should have a new database with some more tables. The new table is this outbox states in which... Where are you? Order states, order states. Let me copy that. Order states. Okay, nothing happens in here. What do you want? Nothing. Some leftover messages in the RabbitMQ and that's why the component's listening to the same queues, like were triggered, hey, I have something to process.

Okay, now we're pushing another order because why not? We have the same model, we accept an order and we send back 202 accepted. But now when the saga goes in we have an event that we're listening. So initially you're creating workflows in code. So initially when an order is created, you do something in the database. You save the total amount, you add created at and you change the order status to pending for this specific status, and then you would transition to pending. So it's state transitioning, it works like a state machine in a way, covering flows.

So order receive event has happened. If we look in the database in the order states, we'll see here already information. This is the only... The last one is the one that we're interested in. So we have a current state, it's pending for the order. Some events have happened in the system. So we consume them, we get an order received event. You see that is an outbox. Then we transition to... And we scheduled for those events a payment timeout. I will consume two timeouts and then we're done.

Okay, now I got an order and now I have to pay the order. And if I go in the database and I look at the order ID, it should be the last one and I order here, I say okay, I want to pay it. I'm publishing order paid event, I get the 200 OK. And the state of the order should be changed. The same happened for the payment canceled, which I'm about time and I don't have time to show you, but you can dig through the code and see what happens in the order state machine and what are the states that are transitioned. So if you get the cancellation requested event, then you save something about the saga state and then you transition to cancel event by publishing a new order canceled event. So what happens in case of order canceled, I don't know, you can say like an email, hey, I'm sorry to see you go, blah, blah, blah. Notify, deduct or add back the stock, refund the payment and so on. And you can add a lot of conditions here.

So basically what is my advice to you, if I may, is to not reinvent the wheel because there are tried and tested frameworks that make available all these complicated patterns to you and it's a pity to just spend time to re-implement them. So use abstractions over the messaging infrastructure because it shouldn't be your job to understand exactly how RabbitMQ innards work. You should abstract the way like we used to do with repository pattern, right? So thank you very much for listening. This QR code will get you to my GitHub. So thank you for being here. You'll find me at the particular booth in case you have questions all day pretty much. So pass by just to say hi, and enjoy the conference. Thank you.

Messaging Patterns for Modern Software Solutions

About this video

🔗Transcription