Live coding your first NServiceBus system

00:00:00 David Boike

Hello, everyone. I am David Boike. And in the next hour or so I'm going to be building a complete NServiceBus system from scratch. This is a live coding presentation, which of course, is the most dangerous form of presentation only a little bit nervous. To paraphrase Julia Roberts, I'm just a boy standing in front of a compiler asking it to love me. But if I get in trouble, I also have my colleague, William Brander, here with me to jump in and save me if I forgotten that semicolon or some such thing. Say hello, William.

00:00:29 William Brander

Hello, William.

Thanks. All right, let's get started off by having a look at a picture of what we're going to build. It's an example of a retail business domain. And it's going to have three main messaging endpoints. Messaging endpoints are the things NServiceBus that actually process messages and send messages to each other. We're also going to have a fourth application, a ClientUI endpoint, which will be a send-only endpoint.

So I will use that endpoint to push buttons and it will send a message to the sales endpoint, sale a place order message command, and then the sales endpoint is going to publish order placed, and we will have two subscribers to order placed both billing and shipping. So billing will turn around, and in a real system, it would charge the credit card and do all those sorts of things. And when all that's completed, it will publish an order build.

And then the shipping endpoint will subscribe to both the order placed and the order build event. And the interesting thing with the shipping endpoint needs to do, is it needs to decide when to actually ship the order. So it needs to wait for both the order placed, and the order build event to arrive, which could happen in any order, and then figure out what to do. So we'll see how to do that later. Well, let's get started. And let's build it.

So I'm starting from a completely new solution, I have nothing right now. And I am also going to be using the kind of newfangled terminal driven method of creating stuff, which I'm not super familiar with. So if I'm going to screw up, that's probably what I'll screw up. So let's get started. Let's do make their retail system, and then CD to retail system. And then let's do dotnet new solution. If I open that over here, there's the retail system solution.

And now I'm going to start by creating the ClientUI endpoint. And this is going to be a regular old console application. So I'm going to do dotnet, new console application named ClientUI. No, I did named instead of name making mistakes already. Okay, so we've got a ClientUI, and we're going to add that to the solution. So dotnet solution, add ClientUI, we can't see that happen in Windows Explorer, but I'm sure it probably did. And then we are also going to add the NServiceBus package to the ClientUI.

So we're going to do dotnet. And client dotnet add to ClientUI, we're going to add the package NServiceBus. And this is going to grab the latest version, which is currently 7.5.0 from nougat and install it to the ClientUI. Now I think we can open this up in Visual Studio and start to write some code. And waiting for Visual Studio as always the funnest part.

Okay, so we've got a ClientUI with dependencies, and it's working on it. Now it has and NServiceBus 7.5. So now we're going to get started in program.cs. And this is probably the longest code snippet of my entire presentation. So I'm going to cheat just a little bit, and I'm going to paste it in here. And then I'll go through every line that I'm doing. So in the ClientUI, the first thing we're going to do is eventually when we get all of the endpoints going, we're going to have a bunch of different console windows to keep track of.

So starting right off the bat, we're going to make our lives a little bit easier and do console dot title equals ClientUI, just so we can keep track of them. The next thing we're going to do is create an endpoint configuration. And this is how you configure an in NServiceBus endpoint. All of your configuration methods will come from endpoint configuration to start off, ClientUI is the name of the endpoint. This is a logical name. So later, we will talk about routing and how you start with ClientUI.

And when you want to send a message to somewhere else, you have to tell it where to go. These are the names we'll use. For this demo, we're using the learning transport and this is a transport that kind of fakes a queuing system by just using files on your local disk. So it's not something you can use for production. But it's something that is really nice for demos like this. And so NServiceBus also supports Azure Service Bus, RabbitMQ, Amazon SQS, we even have a SQL Server transport, there's a whole bunch of different transports you would use in production.

This endpoint is going to be send only, it is not going to have a queue that it processes messages from, it is only going to send messages because all it's doing is taking my commands from the keyboard and turning them into messages. So after we've done the endpoint configuration, then you can start an endpoint, this is called self hosting, we'll be looking at using the dotnet generic host a little later. So we start the endpoint with endpoint dot start.

And then we've got something here with which we can send messages and stuff like that. So I'm going to run a run loop, which you can see down here passing in the endpoint. So from the run loop, we will basically do an infinite loop through this while and we want to place an order if we press P or quit, if we press Q, and so we're doing a console read key, we've already got the stuff in here to quit if we press Q, so that's where we'll want to start creating our message.

But first, we need some place to actually send the message to so it's time to create that sales endpoint. So coming back to PowerShell, we can do dotnet, new, another console application named sales. And we can add that to the solution. This is always the thing that gets me but hopefully Visual Studio will be nice because it's going to reload the solution. I have unsaved changes here. So I will say, yes, I want to save those for you do that. And hopefully it hopefully won't screw me up. And it didn't thank goodness.

So in sales, I didn't quite get where I wanted to go, we also need to add the NServiceBus package dotnet add to sales, the package NServiceBus. I don't know if it'll, it would probably not care about the casing issue. But whatever, perfectionist at heart. And so in this endpoint, we are going to use the dotnet generic host. So we're going to do dotnet add to sales, the package NServiceBus, dot extensions, dot hosting, which brings in the dotnet generic host.

So now that, that's done, we can look at the sales program DTC But first, it's always good to make sure it loaded the packages and it did. So in the sales program dotCS. First of all, we're going to want to make this I'm just going to paste in this code again. I also have an extension that adds most of the most of the usings you need when you paste in. I don't know why it's doing this, maybe we should just try to build it. Build failed. No. William, can you help me out?

You need to have you're using statements using NServiceBus.

Of course. I don't know why my paste code, they didn't work. Okay. So now, and thank you, William, we have our sales endpoint kind of setup. And here I'm doing console dot title equals sales. And this is kind of a very standard way of setting up a dotnet, a .NET Core, dotnet five application. So we're doing a weight host dotcreate default builder passing in the arts, and we're doing use NServiceBus.

This is an extension method that comes from this NServiceBus dotextensions, dothosting library where your job is to take the host builder context and basically return the endpoint configuration that you want to have created as an endpoint. So here we're doing the same thing as we were doing in the ClientUI, we're creating a new endpoint configuration with the endpoint name sales, we're saying we want to use a learning transport because they the two endpoints basically have to be talking the same language.

And then we're going to return the endpoint configuration. And then the run console async makes it so that everything starts up, and runs, and does what it needs to do. So now we have somewhere where we want to send messages from, and somewhere we want to send messages to, but we the next thing we need is an actual message. And an NServiceBus messages are classes. But you don't want to put that class in ClientUI. And you don't want to put it in sales because both projects need to know about it.

So what we're going to do is we're going to create another class library to hold that messages class. So we're going to do dotnet, add new classlib name messages. And we're going to add that to the solution. And in messages, we're also going to need a reference to NServiceBus for this presentation. Now in your real life applications, or systems, you don't have to do that there are ways to identify messages without having to reference NServiceBus, but that's more of an advanced topic.

And so now we're also going to add references for the messages assembly to both ClientUI and sales, we're going to done that add ClientUI, reference, and then we do the path, that's not going to work, reference. And then we do the path to messages dotcsproj. And then rather than type that, again, I'll come back here and just replace ClientUI with sales. Okay, and now we'll add Visual Studio refresh. So we've got our messages assembly, and on our dependencies, projects, messages, and dependencies, projects, messages, so we're good to go.

So we will take this class.cs. And we will actually change that this is going to be our place order class. Yes, you can rename it, public class place order. So the first thing to note about creating a message, a command message in this case is the naming. We want to make commands being named in the imperative sense, that basically means I am telling you to do something, this is not optional. This is not a request, it's not place order request, it is saying place the order.

Now even some people will take this and they will do and they will do place order command that makes it easier for them to see what's a command and what's not, or the spell out command. That's a personal style choice. I don't care what you do, but what is important is that you get that naming so that it actually says do something, it'll make everything make more sense later. The other thing we're going to do, is we're going to identify this as a command for NServiceBus by implementing I command.

And now we'll need to using, be careful, there is a system that windows that input in there, you don't want that, that'll screw everything up you want NServiceBus in there. And we are going to have a public property, and public string order ID. That's just so we can tell the difference between different messages, between different place order messages. So now we've got thing that wants to send a message, thing that wants to receive a message and we've got the message.

But how are we going to tell the message where to go and where are we going to send it from? Well, let's start in the ClientUI. Here where we're doing the P for place order, we pass in the endpoint as an iMessage session. Now, you might notice that back here, endpoint was an iendpoint instance. But we passed it into the run loop and it becomes an iMessage session. An iendpoint, or an iendpoint instance is basically an iMessage session, but it also has the capability to stop the endpoint.

So I message session is the interface you're going to want to pass around if you're creating a web API application, that's the thing you're going to want to inject into your controllers and actions and stuff. imessage session gives you the ability to send, publish and do all the things except kill the endpoint. So we want to take the endpoint, and we want to send a new message and you Place Order, and we want an order ID and it doesn't know the messages that we'll do using messages.

And we want to do order ID equals and this is just something I used to get a random order ID it's a little bit long in the tooth, but guid.newguid.tostring and then let's just take the first eight characters of it just so we're not looking at huge long things. And it doesn't like me because I am not awaiting it. You notice NServiceBus has analyzers built in NServiceBus 0001 a task returned by an NServiceBus method is not awaited, or assigned to a variable.

We will help you with your async await when you're using NServiceBus API's. So we want to do await here and we've already got async, so that looks good. Now there's only one thing missing. And that is well okay, the endpoint is going to send a place order but where is it going to send it to? We want to create a separation between the business code of what command you're actually sending and the routing, or topology information of how your system is actually set up with endpoint A sending messages to endpoint B to C, etc.

So that we are going to configure in the endpoint configuration. So here where we've got use transport, learning transport, this actually returns a variable. And normally we'll call transport. And from there, you can do var routing equals transport dotrouting. And now we can say routing, and we can do route to endpoint. Now there's a bunch of options in here, you can take a whole assembly and say any message from the assembly we want to send to the destination.

If you are creating a bigger system, that you can make a system where every, single endpoint or every single logical service essentially has its own messages assembly. And that's probably actually a better thing to do than what I'm doing, now, we might talk about that later. And if so, this is what you use. But for now, we want to use just a specific message type, we want to say that only type of place order if we send that kind of a message that needs to go to the sales endpoint.

Okay. So now we're sending the message, we know where it's going to go, I guess we are still missing something. And that is something to handle that message. So in sales, we already have our program.cs, the starting NServiceBus endpoint, but we need a message handler. So I'm going to add a class. And I'm going to call it Place Order handler. This is a pretty good convention to use the name of the command plus handler.

And to market as a message handler, we implement I handle messages, which is one of my favorite interface names ever using NServiceBus of a type Place Order. And we will need to use messages. And then we'll need to implement. So this interface marks the class as a handler, and also provides the handle method. So in the handle method, we get the copy of the messages being handled. And we get an imessage handler context, this has all of the send, publish etc methods that you'll need to do all of your NServiceBus.

Let's also be a good citizen. And let's create a logger in here and do it the right way. I could do this with console.writeline. Let's do it right. So we're going to add a logger through NServiceBus. And when you are using the dotnet generic host then NServiceBus log manager dot get logger where we're type passing in this type so that we know that log messages come from this class. This interacts with the dotnet generic host. And so whatever you use for, for dotnet logging that you set up through the generic host, this will integrate with that and that will come out.

So in the handler, now, we want to say just for now we want to do is log.info, and we'll say, received Place Order, orderID equals and we'll do message.orderID. And this is currently a not an async method, we don't have anything async going on, I'm just going to go ahead and mark it async. Because we will be doing stuff like that eventually, I'm getting a thing here that says that I have marked this at async but everything we're doing is synchronous, but that's okay, we're just going to ignore that for now.

So now I think it's time for a demo. So we will want to create, we will want to run both the ClientUI and the sales endpoints simultaneously. And so in my experience, a lot of developers who are Newton's response have never actually run multiple endpoints at once, or never run multiple projects at once. They've always done just the startup project like this. So you right click the solution itself, you go to properties, and it comes up to startup project, you select multiple startup projects.

And so we want the ClientUI to start, and the sales endpoint to start and we hit apply. Okay, nothing's bold, because it's going to run multiple things. It says multiple startup projects here. So now we can start it and hopefully I haven't made any booboos. So here's the sales endpoint using the dotnet generic host. Here's the ClientUI. They're both started up this you can see the dotnet generic host stuff here. This is just console dot write lines, we've got the P to Place Order.

And so let's do it. Let's type P, and here, Sales, Place Order handler received Place Order, order ID equals this new one. And if we do this several times, we'll get it multiple times with multiple order IDs. Now, when I was starting out NServiceBus, this was one of the things that kind of blew me away, because I don't know how many people have done WCF before. But all of the XML configuration made me go a little bit nutty.

And being able to have two different processes, communicating to each other like this, with just a little bit of code that I've done so far, is kind of mind blowing to me. But now let's do some other stuff because let's do some stuff that WCF can't do. So what if we were to stop the sales endpoint, and then place a bunch more routers, with WCF, you would blow up, you'd get an HTTP exception, or depending on what your binding is, it would not work because you would not have that bi directional communication.

But I can place as many orders as I want. And I have no problems. So then, what happens when sales comes back online, let's debug it and start sales. When it comes online, it takes all those messages out of its queue, and it instantly processes them. So you get this concept of reliability, where even if an endpoint is down, you are still putting messages into its queue, you're not losing them, you're not throwing exceptions, it's just going to take a little longer to process them, you wait until the sales endpoint comes back up, and everything's fine.

Let's do something else fun. Let's go into the sales endpoint into the Place Order handler. And I'm going to actually add a static random, random equals new random. And what we're going to do is we are going to throw an exception sometimes. So let's say if random dot next number between zero and 100. Let's say if that's greater than, I don't know, 80. Let's just throw that let's not thread static attribute, let's throw a new exception, boom. Why are you red? Go away? Okay.

So roughly a fifth of the time, 20% of the time, this is going to throw an error. Let's run the system again. Okay, we're going to place an order. Okay, we got lucky it processed successfully. Let's keep doing it. Eventually, we're not going to be so lucky, there we go. So it says, immediate retries going to retry message and this long NServiceBus message ID because of an exception, boom. And what happened immediately after the immediate retries kicked in, and on the next chance, the order was placed successfully.

Now we didn't get a log. So what's happening here is that you get reliability for transient exceptions. So if you've ever run into a case where you have something that runs into a database deadlock, and you see that the sequel exception says, try running this transaction again, you get that for free NServiceBus because you send messages. And if it doesn't work immediately, it'll instantly do a retry, which might fix your error. And so you don't lose any data or anything like that Which is pretty cool.

So I'm going to stop it now. And I'm going to comment out the boom. And we can move on to publish subscribe. So if you recall back in the diagram, after the sales endpoint received Place Order, it was going to publish an order placed event. So what do we need? What do we need do, we need to have the event as a class just the command and then we need to publish it from here. So first, let's go to our messages. And let's add another class. And I'm calling this order placed, just commands it is pretty important to name your events correctly as well.

In this case, I'm using the predicate tense, which if you're not an English nerd, essentially means the past tense. So events are normally going to have an ED here at the end, they're going to announce that something has happened. You don't reject an event, you don't get to validate if an event is true and event means something that already happened, so be it, there's nothing you can do about it. So now that we have the class, just we marked commands with icommand, we will mark events with ievent and we need the NServiceBus using and we will also do the public string order ID. So we know What? Okay, that's our event.

Now we come back to the place order handler, and we want to publish that event, so we will do. Remember, we have our iMessage handler context here. And that's going to be an async operation, because almost everything NServiceBus is, so we're going to do a weight context dot publish, new order placed. And we're going to pass through the order ID, from the incoming message. That's really all there is to do there, we could run it, it won't do anything different.

Because this order placed event does not have any subscribers, but it's entirely okay for an event not to have any subscribers. In fact, you can create your systems so that you can go I know, this is an interesting thing that happened. And eventually something in the system is going to want to know about this. I'm not coding a subscriber for it yet. But I know at some point, we will. And if I know of, if I'm at least smart enough to figure out what should be in that order placed event, I can go ahead and create it.

And that enables me to later create a completely new endpoint and subscribe to order placed. And I can add new functionality that system without ever having to come back to my sales endpoint and change anything. That's an element of decoupling that you get with the publish subscribe pattern. So it won't do anything yet. But we're about to create our subscribers. So let's do that, save everything, come back over to PowerShell, we're going to do dotnet, new console application named billing.

And I'm going to copy that also create the shipping endpoint. And I'm going to add both to the solution. So dotnet, solution, add billing. And then with shipping. Okay, now we'll come back to Visual Studio, it will do its little reload song and dance, a whole bunch of new stuff. Now, I'm going to basically take a shortcut and copy and paste some of the references and stuff. So in sales with through PowerShell, we've already created the package references for NServiceBus and the hosting extensions.

And we've already created the project reference for messages, I'm just going to take that XML and copy it to both billing and shipping because they both need the same things. And also, I'm going to copy the program.cs file, I'm just going to modify it for each one. So starting with billing, paste in that and we will, let's see, we need to change sales to billing, essentially. So change the namespace, and the console title, and the endpoint name.

Did I not get all of the extent all of the dependencies in here? It seems to have corrected itself. Okay. And so I'll take that and copy it again, and come over to shipping, paste that in, change the namespace, and the console title, and the shipping. Now if you're wondering, hey, he's copying and pasting a lot of code, isn't it possible to be a little bit dry with this? Yes, it is. We see a lot of customers that take their endpoint configuration code specifically and they add it to an assembly that they kind of distribute through their entire project, maybe even they make a nougat package out of it.

I've heard it called a conventions assembly, essentially, so that you don't have to repeat a lot of this stuff. So yes, that's the thing that can happen. Okay, so now that we've got our billing and shipping endpoints created, we need handlers in them so in billing will create an order placed handler. Add class, order placed handler. This is just another message handler. It's just the same I handle messages, but this time I've typed order placed, and get all the usings and implement and can I steal by logging from somewhere. This comes from NServiceBus logging, so that should all go away.

Okay. And this is essentially the same thing I want in shipping as well. So I will add class to shipping, another order placed handler, I guess I could have copied the file as well, change this to shipping. Okay, so we now have order placed handlers in both billing and shipping. But let's concentrate on billing first. Because remember, billing is the one that needs to charge the credit card, or do whatever and then publish, order build. So we'll take out the throw.

And we'll do a log dot info, received Order, order placed, order ID equals and message dot order ID. And then we're going to turn around and await contexts publish, new, I don't have this class yet, order build, but I'll pretend as if I already have created it, you can probably guess what it's going to have by now. And async here. So now we'll go back to messages, we'll create order build, public class, or build, this is also an event NServiceBus. And the public string Order, order I just can't take. Okay, don't really need that anymore.

Now we go back and our publish is fine. We are receiving order placed, we're turning around and resend publishing, order build. So now let's go to our shipping handler. And it turns out, you can actually have multiple handlers per class. They're just some things you have to keep in mind, which I'll go over in a second. So you can do I handle messages border, build as well, and implement both interfaces. So now we've got handled methods for both order placed an order build. So here, I'll delete these.

And I'm just going to go ahead and make them async. Even though Visual Studio like that, with the green squiggly, that's fine. So order placed. Now what we need to think about is we have the shipping endpoint that is receiving both order placed and order build. But when do we ship it? We're not quite there yet. So I'm going to do log dot info. Here, I'm going to do received, order, placed for order ID. Message order ID, but should we ship I don't know yet.

I'm going to copy that into the order build handler, just change this to order build, should we ship. We'll think about that a little bit more in a minute. But first, we're going to do a demo and see this whole system running with pub/sub. So here's our ClientUI. Here's our sales service, or endpoint. I already messed up, I didn't reset the startup projects. So we also want billing to start, we also want shipping to start. This is where we get into console window craziness.

Okay, here's the shipping. Here's the billing. Here's the sales. Hopefully we can see all these. I'm going to place an order. So in sales we received Place Order, we published Order, order placed here in billing, we received order, placed order ID and then here in the shipping endpoint. We received order placed for order ID four, five blah, blah, blah, should be shipped and received order build, should we ship. This is the next problem we need to solve. And this is why we use NServiceBus sagas, but first of all try doing it without sagas and see how that works.

You might be thinking well I have two handlers in the same class clearly they must use the same stuff so why don't I just do a bool build and bool placed and then here I will do when we get order placed we will do placed equals true. And here we will do build equals true, and then let's just add these to our logging in here. So I'll just say, placed equals placed, build equals build. And I'll add that down here as well, and we'll just see what happens.

All of our windows come up again, in a completely different spot, of course, but we're concentrating on our shipping endpoints. So the other two you can just kind of ignore, so when I place an order, here we have received order placed, should we ship placed equals true build equals false, okay, we're getting somewhere received or build for order ID, same ID, should we ship this time place is false, although build is true.

So what's happening in here is, every time you receive a message, it is instantiating, a new copy of order placed handler. Now it's important to remember that these messages could actually come in through the queue, minutes apart technically, so there's no reason we can, but there's no way we can keep this around and keep this state we have to have a better way than in memory to keep the state. So that's why we do sagas. So we are going to start turning this order place handler into a saga.

And the first thing we need to talk about is the name because it's really not an order placed handler anymore, because it's handling order placed and order build. So what we really want to do is, I like to call them policies. So instead of order placed handler, this is the shipping policy, or the code that determines when it's actually time to publish order shipped. Next, we'll actually inherit from the saga base class. So saga is an NServiceBus class, but it's of type T, saga of type T saga data where saga data is a class and implements I contain saga data and is newable.

This is a place to store that data. So when we were creating build in place, what we were really doing is thinking about what saga data we needed. So I'm going to go ahead and order and implement that as an internal, not an internal I'm sorry, as a nested class. So public class, I'm just going to say data because it's inside the shipping policy. So the full name is shipping policy dot data. And it's going to inherit from contains saga data.

And I know that the thing appears that I contain saga data, but contains saga data, if you take a peek at it, it's just a class that implements contains saga data and provides the property at some of the properties that and NServiceBuss to do its thing that you really don't have to worry about. So we can move our build and placed into here. So public school build, and actually placed should be on top that normally comes in first. So we'll get rid of these. T

he other thing we're going to need in here is a way to tell this saga data instance from any other and what does that what I would do that well, that would be the order ID that's common between all these messages. That's what we call a correlation ID because it correlates all these messages together. So we do public string order ID. We're going to identify that as the correlation ID in just a second. But first, we'll change these, these are no longer but, I forgot I need to put shipping policy dot data in there.

So now not everything is happy yet, because we need to implement the abstract base class. We'll come around to that in just a second. But now that we're inheriting from the saga base class, we have something in our handlers called, if you look at this, this dot data, it's their data, okay. Let's not this must let's call this saga data instead. And now the intellisense will work for us, this dot data dot and here we see build and placed an order ID.

So from order place we want to do place equals true. And here we'll replace this with this dot data equals dot build equals true. Now the thing you don't have to do is set the order ID because we're going to use this method to identify the order ID is our correlation property. So you're going to have a message come in with an order ID on it. And we're going to know that, that's how you look up your saga data. So NServiceBus will set that property for you. So you don't actually have to do that.

So now let's Look at the mapping down here. And take this out. So the job of this method to configure how to find saga is to basically say from an incoming message, how do I find the correct instance of saga data that has the order ID that I want. And we do that using this mapper. And configure mapping is an older way of doing it. But we have a new one that I like a lot better because it's a lot less repetitive. So we say map saga, and then you provide a link expression. So from the saga data, is basically asking, what property are we looking for.

And we're looking for the order ID. And then from this, you have a fluent interface where you can map this to each message that comes in. So you can say, well, first of all, you can use message headers as well, that's a fairly new addition to enter response, you can correlate on message headers, but not most of the time we'll do to message and well what message order placed. And from order placed, we want to look at the order ID as well. And then we'll copy that for order build.

So what this is saying is when an order build message comes in, get the value of the order ID and let's say it's 1, 2, 3, 4, 5. And from your saga data, if we're talking in SQL terms, we want to do select from saga data, where order ID equals 1, 2, 3, 4, 5. And that's how it will find it. Okay, so now we only need a couple, a few more things. We aren't really doing anything about should we ship yet, although I notice I need to change these to data dot placed, and dot build, and then fix that down here as well.

We need to talk about what messages can actually start a saga. So I handle messages is the normal handler interface for regular message handlers. But within a saga, you have the additional consideration of what messages are capable of starting the saga. And most of the times you're probably default to almost all of them. And when you do you change this from I handle messages to I am started by messages of place of order placed an order build.

Now you might be asking yourself, well, that doesn't make any sense. Order build can't possibly can't become before order placed, because order build doesn't happen until the billing endpoint receives order placed. And the billing endpoint then has to publish order builds. So how could this possibly arrive before order placed? Well, it could be that the shipping part that the shipping endpoint has just a really long queue, or, and so it's taking it longer to process things, than billing, and billing is really quick.

Or it could be that there's an exception, and you run the whole process, and one of the messages goes to the error queue. And so order build is quick and comes in here and gets processed. And then it takes a lot longer for order placed to get successfully processed because it had to wait in an error queue for a while. So essentially, any message coming from outside the system is going to be generally an IM started by messages.

The only real time that it can't be is when ta message can only be created because of something that happened after the saga started in here. So if we came in here and we were to send a new some new command, then that would be just an iHandle messages. But we're not doing that. So, any message coming from outside the saga not directly caused by the sagas activities needs to be marked as I am started by messages.

Okay. Now we actually look at should we ship? What should we do when we ship. So I'm going to introduce another method called private async task, process order. And I'm going to pass in the iMessage handler context. So this is going to be called by both of these methods. So after it sets the state, it is going to call await process order and pass in the context. So what we're going to do in the process order method is we are going to say if data dot placed and data dot shipped, not shipped order data dot bill. That's when it's time to ship the order.

So do we have a logger in here yet? We do, okay, so I'm going to do log dot info, order is placed and build time to ship. And this is where we inside here is where we would take our context. And we would do a context dot publish, order shipped. But you've already seen me do publish subscribe. So in the interest of showing other more important things, we're going to skip that and just log the message right now. The other thing we're going to do is say at this point, according to the business requirements we know about now, this saga is completed, we don't need it anymore, we're not expecting any other messages.

So we're going to call mark as complete, which is a method on the saga base class. This is basically saying, hey, NServiceBus, we don't need this saga data anymore, when this is when this is complete. And when the transaction commits, you can delete the instance data for this and it goes away. Not every saga has to be completed. There are some styles of saga that are essentially never ending, but this one does have an end date. So we will mark it as complete.

Now it's time for another demo. So I will save everything and press start. So we're really watching or shipping endpoint here. So I'm going to place an order, I don't think it listened. Or maybe just everything isn't set up yet. I have an exception, what did I do? The selected persistence doesn't have support for saga storage. I did indeed forget something. When we have saga data, we have to store it somewhere. Normally, you will store this in some sort of database.

And we have a bunch of persistence libraries to help you store it in whatever persistence you are using for your business data. For this demo, since we're already using the learning transport, we're going to also use the learning persistence. So I'm just going to say endpoint, no endpoint configuration dot use persistence, learning persistence. And just the learning transport, which is storing messages as files on disk learning persistence, and storing visit as files on disk. Not suitable for production, but great for demos. So let's run all of this again.

That's a much happier startup. Okay, P to Place Order, okay. In shipping, we have received order placed should be shipped placed equals true, build equals false. Don't have an end parentheses, that kind of bugs me, but moving on, we have received order build for the same order ID should we ship, now it remembers that place equals true, and build equals true because it's storing that that state in a saga. So then we say orders placed in build, it's time to ship, which is great.

So essentially, what a saga is, is a message driven state machine where your state is this saga data that gets persisted for you between messages. And your messages come in and talk to the saga data and you make some decisions and then new messages go out. One thing I will mention it is not recommended to do any other sort of work within a saga handler, you should only have your message come in, consult your saga data, make decisions and have messages go out you shouldn't call web services, you shouldn't do database, work to persist business data, any of that.

Because your database is already keeping a transaction open to hold a lock on your saga data. You don't want to pollute that transaction and widen it by doing a whole bunch of other work that just makes it more likely that you will get basically deadlocks and have more retries and have to do stuff like that. So kind of the golden rule of sagas is kind of get in and get out as quickly as possible. Now that we've done basic sagas, I have one more topic and that is saga timeouts because sagas have another feature that is very useful.

So to do it as simply as possible. What we're now going to consider is what happens if there's something wrong with the billing procedure and the billing doesn't happen for a long time. What do we do then? Right now, we are just waiting for order build and order place to commence order placed will come in and it will do the first part of the saga. But we will wait for order build forever, and there's no way to check up on that. So we're going to introduce a timeout to provide a way to basically wake up the saga after a period of inactivity.

So in our order placed handler here, after we do the process order, we're going to come in here, and we're going to say if the data is not billed, because remember, it's possible for the billing message to come in first, then we're going to do await, this request timeout of a type, I'm going to call it order, not build timeout. And this is a thing I don't have yet we have to go create this. But we need to provide it with the iMessage handler context, we need to pass that through.

And we also need to give it some form of how long we need to wait. So if you look at the IntelliSense, this one, we can give it a date time, we can give it a time span, we also have ways that we can create a specific instance of the order not build timeout. So in this case, we're just going to use the timeout class that I'm going to create as kind of a marker. And I'm going to say, let's request the timeout for timespan from seconds, five.

In real life, you will probably have five second timeouts, especially for a business thing, this would normally be something like 48 hours, or a week, or something like that, you would want to have that process, come back and figure out what's going on. But for our purposes, we just want to see it quickly in the demo. So we can do five seconds. Now we need to define the order not build timeout. So I don't remember if I mentioned before, but I made the saga data and internal a nested class of shipping policy.

And the reason that is, is because there is tight coupling between shipping policy and the saga data for the shipping policy, that's not something you want to break apart. And it's also not something you really want anything else to access from outside the saga. The same is true of a timeout class. So I'm going to first copy the class name, so I don't get it wrong. And then I'm going to do public class order not build timeout, and it's just going to be an empty class, I don't have to mark it as an iMessage, or an iCommand or, an iEvent.

It's just a class and we handle it like that. So now it's basically a bundle of state. So you could put state into the future, you could include your own new order, not build timeout, and you could fill it with properties or do whatever you want. It's just going to get serialized to JSON. So now we've requested that timeout, but we also need to be able to handle it. So we will do iHandle timeouts in this case, or not build.

Did I get it wrong? No, it's a nested class. So I need the shipping policy dotspecify it. And I will implement the interface. So now here we have public task timeout. And when that happens, you get the order not build timeout state coming in. And an iMessage handler context looks very much like a handler, but it's just slightly different only happens in sagas. So now in here, let's do log dot info. And let's say, order data dot order ID.

Notice that any method going on inside the saga is going to have access to the data in the order ID so you don't have to send the order ID through with the state and get it automatically. You also don't have to do any mapping for timeout, timeouts already know what their parents saga is. So we're going to say, Order, order ID has not been build. Publishing order not built, which again, in the interest of time, because we are getting very close to the end of hour, I am not going to do, I'm just going to return tasks, completed tasks, so that this is happy.

Maybe, there it is. So now we have our nested timeout class, we have implemented iHandled timeouts. So now we need to actually test this in some way. So what we're testing is what happens if order build is slow, and we're testing with a five second timeout. So let's go back to order build, the place where order build is published, and that's in the billing, not the program.

In the billing, the order placed handler. So here where we're receiving order place, let's just we'll do this brute force, and I'm just going to do a weight task delay from seconds, time span from seconds, so this is going to delay the order build to endpoint where our saga will get concerned about it. Okay, I think that's it. So let's start it up and see what happens. So again, we have our shipping. And we also probably want to look at our billing. So let's kind of line it up so we can see billing and shipping at the same time. So P to Place Order. The billing is in its task delay right now. But we received order placed, should we ship?

And now it's going, order has not been built the publishing order not build. And then five seconds after that, we received order bill. And it turns out that the saga did complete successfully. So you could ask, well, why didn't it stop? What was the deal with order bills still going through, and it's still finishing? These are the types of questions that you really need to take to your business stakeholders, because I have no answer for you.

If you ask them the right questions, which usually involves asking, why, a whole bunch of times, you will get down to endpoint where they can tell you when an order hasn't build, we need to publish, and we need to do that kind of this workflow. And does that mean that the shipping policy should stop, or it should do something else. Only your business people can tell you that, I could make up requirements out of thin air, but the only way to do that in real life is to talk to them.

So that is the last thing I have. So going back to here, of course, it goes back to the beginning of this slide. So through this demo, you should have a decent idea, hopefully, if I've done my job of the nuts and bolts of how to send messages, publish events and build sagas, but you might be missing the bigger picture, which is how to kind of envision a distributed system that runs this way. And some of the best practices to use when you're building such a system.

For that we have the distributed systems design fundamentals course, which is free for a limited time. In this course, you'll learn why it's a good idea to design systems in this way, and how to think about modeling these types of systems, including much more information than I can provide you in one hour on modeling long running business processes as sagas. As I said, this course is absolutely free right now, so I encourage you to check it out. And with that I am done. So thank you.

Thank you, David, we have a couple of interesting questions that I wondered if you'd be so kind as to answer live.

Sure.

The first one is from Jeff Odell. It is, can you give an example of a use case for a never ending saga?

Sure. Thank you, Jeff. I know Jeff, Jeff's a good guy. I'm a use case for a never ending saga I believe we have a blog post on this topic called, Death to the batch job. So you can consider even staying within this retail demo, you could consider a kind of saga where you want to implement a customer loyalty program. So if I have, if I've bought $1,000 worth of goods over the last year with a sliding scale, then I get some sort of 500 or 5% preferred customer discount or something.

And so the way a saga looks like that, is whenever I buy something, I go into the saga data, and I basically increment the amount of my running total by whatever that purchases, but then I also add a timeout so that a year from now, that same amount will be decremented from my running balance. So when the saga says that my running balance goes over $1,000, then I publish customers now preferred when I go under $1,000 it publishes customer is no longer preferred.

But as long as I'm a customer if I never go away as a customer, then the correlation ID is my customer ID. And there's no reason to ever end that saga ever, and it lives forever.

Right. We are going to run a exit poll, so you can answer those questions if you feel like it. There is another question that's very interesting, David. Unfortunately, I don't know who asked the question. It's anonymous. But the question is, you mentioned that we should not do database transactions in a saga. Where should you do it?

It's a great question. So as I mentioned, sagas are essentially message driven state machines. So the way to do that is to send a message to somewhere else. So you either send a command, or publish, an event and another message handler will pick that up. This is a regular message handler that isn't a saga. So there's no database transaction going on with your saga data. So from that handler, you can then do, you can call your web service, or do your business data or, or whatever you need to do.

And then those sagas can also just do a context dot reply, which is I didn't show you. That's kind of the that's the request response pattern. So in that case, you do a context reply, and you give it a class that's not a command, or an event. It's just an iMessage. And then that message goes back to the saga kind of to report back, and then your saga can take further action. Is there anything else William?

I'm just busy. Sorry, give me a sec. I'm just scrolling through them to see which other ones would be interesting. So there's a couple questions that are asking how you would do database transactions NServiceBus. But so how you would update, or query a database within a handler, things like that? Is that something you could answer now?

I can give a very hand wavy answer to it. When you're using NServiceBus, usually you've got some sort of latent database transaction for your business data that NServiceBus probably manages. If you're using RabbitMQ, for example, you don't have RabbitMQ doesn't have any transactions to speak of. So it's a good idea to use our outbox feature, which gives you transactional atomic control between whatever messaging operations happen and whatever database transactions you do.

But that means that NServiceBus is managing your database connection before your handler even starts. So for each persister, for each storage library that we have, I think we've got an article about accessing data from within handlers. And it involves that iMessage handler context object again, but there's a property in there called synchronized storage that has extension methods so that if you're using SQL, for example, you can get access to the SQL connection and the SQL transaction that NServiceBus using, and then you can basically piggyback off that, and just use that connection and transaction directly.

Right. And then I think one last question before we end it. If you're hosting an endpoint within a web application, once that web application starts, how can you ensure that it has subscribed to messages? So I guess the question is, if you have?

Right, let me try to give an answer to that. So I guess, I have to make the assumption that you are having a web application that you want to subscribe to events. So first of all, there's only a very narrow use case where you should do that because your web application should probably not be a normal NServiceBus endpoint most, most web applications are send only endpoints, and do not process messages themselves, which means they cannot subscribe.

The reason for that is web applications, you're probably going to want to scale out and so you get into a problem of which web application is actually going to be responding to each subscribed message which can lead to weird things because the web applications all basically need to kind of agree. The one kind of case where you could maybe do that is if you need to provide feedback to your users and via some kind of SignalR type of thing.

Or if you need to drop some cache entries based on an event going out. In those cases, it can be a little complex to do that. In the case of SignalR, you need to provide a SignalR backplane, so no matter which web server the user happens to be connected to they get the message even no matter what NServiceBus endpoint processes it. So I guess my high level answer would be try to avoid that. Try not to do that. But if you do need to do the SignalR our use case, then absolutely reach out to our support, and we can help you with that kind of stuff.

Live coding your first NServiceBus system

🔗Why attend?

🔗In this webinar you’ll learn about:

🔗Transcription

About David Boike

Additional resources