I'm the creator of NServiceBus. Ask me anything.

00:04 Dennis van der Stelt

Hello, everyone. And thanks for joining us for another particular live webinar. I am Dennis van der Stelt and today I'm joined by Udi Dahan, creator of NServiceBus and one of the world's foremost experts on service oriented architecture and domain-driven design. And Udi will be answering your questions today during this Ask Me Anything webinar.

Just a quick note before we begin, please use the Q&A feature to ask any questions you may have during today's live Ask Me Anything. I will try to answer as many as possible. We'll follow up offline to answer all the questions we won't be able to answer during this Ask Me Anything. And we're also recording the webinar and everyone will receive a link to the recording via the email. All right, Udi, let's take it away.

00:54 Udi Dahan

Wonderful, thank you. Welcome, everybody. Got a good turnout for today's Ask Me Anything. And we got quite a lot of questions that were already submitted in advance. Dennis will be shepherding the questions as they're coming in and feeding them to me and I'll do my best to cover hopefully about 20, 25 of them. We'll see how it goes. And as Dennis said, whatever we won't get to, we'll follow up with afterwards. Dennis, what's the first question that we got for us?

All right. The first question is, "The description of the webinar asked if we still struggle with coupling and not having enough autonomy, even after applying domain-driven design, server-oriented architecture, SQS and so on." And the person asking the question says, "Aren't these just tools that we need to consider per project team company and so on? How can I understand the client needs and prevent, for example, over-engineering, choosing too many tools? Or if incorrect decisions were already made in the past, do you have some advice on how to roll them back?"

02:12 Udi Dahan

That's a tricky one. And essentially, it sounds like this person is a consultant and has been in the situation where things that have worked at one company don't necessarily work in another.

02:26 Udi Dahan

I'll try to give the generic consulting answer, which is start with the client where they are. I mean, if they have over-engineered things then it might be tempting to roll that back, but there are likely a bunch of people there that like they're over-engineered solution. They think they're getting a lot of value out of it.

02:50 Udi Dahan

I'd say pick your battles, try to understand of the tools that are available to you and of the place where the client organization is at, what changes they're more amenable to making, what changes are less. If there's one thing that I'd suggest as a general practice that is supportive of all the other things, if the organization is not doing retrospectives on a regular basis, try to incorporate that idea over there.

03:33 Udi Dahan

The fact that it is part of agile will likely make it be something that is accepted. They might say that, "We're already doing retrospectives," in which case I'd say the next step is to say, "Well, let's take action items from those retrospectives. Let's try to make changes based on the things that we've agreed to have not been working so well for us."

03:57 Udi Dahan

Get an organization to start getting comfortable with reflecting and making change. And then, that will enable you to layer in other gradual changes to whatever practices that they're doing. But the general top down architecture idea of we're going to come in, we're going to do DDD and CQRS and microservices and all that kind of thing, that often creates the over-engineering problems that consultants later on are brought in to help deal with. I hope that helps with that question and you can always follow up in the Q&A if you'd like more.

All right. One question came in from a person who attended your course or at least saw the videos. And the question is, "One aspect of your course that doesn't get much attention is front-end architecture. In particular, how one can create a software design so that teams can build and deploy independently from other teams, how the branding server complements this and how things like authentication is done for all these services. Can you share some best practices on how to set this up for the web, for example?"

05:15 Udi Dahan

Right. There are a lot of things to do that. We have an SOA workshop. Let me see if I can quickly find that. I think it's called... We have it underneath workshop here. If you go to githubparticular/workshop, you'll see a fairly comprehensive workshop with all sorts of samples that does a lot of the composition. It does front-end side composition as well. You'll see that in the exercises over here. Exercise number one is composite UI.

05:56 Udi Dahan

It doesn't go into things like the authentication and authorization stuff, but it's a fairly good place to, if you're looking for a large code sample with solutions and references on all those things set up, that you can take inspiration from, jump off of, this is a great place to start. Again, that link is github.com/particular/workshop over here.

06:27 Udi Dahan

With regards to the other questions about authentication, that usually falls under the service that I call on my course IT Ops, which is more of a generic type of service that almost all SOA implementations will have something like that.

06:47 Udi Dahan

Authentication usually a cross-cutting concern. It's plugging into some sort of infrastructure pipeline, looking at the request and checking to see whether there is a valid token for that request coming in. At that point, you're getting into regular authentication type of territory depending on whatever tooling that you're using, identity server or otherwise.

07:12 Udi Dahan

But to go deeper into the front end architecture, I'd say the question was about web, right, Dennis? They didn't ask about mobile?

It also included mobile. Yes.

07:26 Udi Dahan

Oh, okay. I'd say when you're getting into mobile app territory, then you're a lot more limited by ultimately what the environment for mobile enables in terms of componentization and more specifically, packaging and deployment. But on the web it's a lot more friendly, especially if you're doing things by inside, that you can load things up from sub-domains. And again, more information we'll get to maybe in a later question. But for a starting point, start here. Further questions, reach out to us and we'll go into the details with you on that.

All right. Another question is, "I've designed my services around business capabilities. In order to avoid data duplication, I reference code from multiple services using NuGet packages. And I now have code in one service that depends on code that's inside these NuGet packages from other services. And if one of those NuGet packages are changed, I need to redeploy multiple services because they all depend on these NuGet packages. Is this the trade-off I have to make in favor of having service boundaries or is there a better way?"

08:53 Udi Dahan

Maybe let me pull up some of the slides from the course so that we have a picture. This is the general idea, top to bottom services that we're talking about. They all have front-end components, back-end components, data, et cetera. The situation about coupling, I think I'm going to scroll a little bit ahead. This is when we're looking across multiple systems in the organization.

09:31 Udi Dahan

Just reiterating from the previous question, a given system which would be seen here as a horizontal slice, okay, will be deploying components from services one, two, and three in this case. Now, the question that was raised is essentially referring to a component from service one, referencing a component from service two directly via NuGet package. Where for example, doing that instead of, for example, having them communicate over HTTP. Component in service one calling via HTTP API something in service two.

10:16 Udi Dahan

Now again, the question that was raised is they're doing that instead of doing data... Did they say duplication, replication? I don't remember the exact word that they used.

Duplication.

10:29 Udi Dahan

Duplication. Right. Correct. By going back to the picture that we had over here, each service essentially owns its data. You don't want to be sharing data between services, but that also includes not via replication, meaning not necessarily doing pub/sub exposing all of the data to other services, but also not via request response.

11:00 Udi Dahan

Now to be clear, when we're talking about data here, we're talking about the raw business type data. We look at this picture, these things like the product catalog or the prices or the actual inventory. The raw business data, not necessarily the identifiers. A product catalog ID is fine to share between services.

11:21 Udi Dahan

All right. In the case where somebody's doing request response between different things in different services, then I'd say that that's a bad of coupling. And that potentially, the fact that they feel that they need to do this, where a component is calling another component, again, whether that's as a NuGet package or as an HTTP request response or specifically raw business data purposes, then usually that's indicative of the service boundaries being incorrect.

12:03 Udi Dahan

One of the causes of that could be that the services are not going far enough up into the UI. Meaning they probably have some top level UI thing and then it's calling one services API. And then that service is orchestrating calls to an multiple backends. Let's say if you're doing something like that, that's an indication that you don't actually have top to bottom vertical slice type service decomposition and it can be challenging to get there.

12:45 Udi Dahan

But the places, to circle back to the question, where it is appropriate to be using NuGet packages would be, let me give an example from... Let's say we're doing a more advanced type of composition case. We're needing to do customer specific pricing or discounting that takes into account customers order history and their demographics and any special offers.

13:20 Udi Dahan

You'll notice that there is this interface here called an IProvideListPrice, or I ProvidePriceFactor. Any type of framework level or IT ops level interface or package is a NuGet package that will be shared, will be referenced by multiple services. And then that is the one that you end up taking the dependency on.

13:50 Udi Dahan

But for the most part, these interfaces are meant to be used as essentially a plugin model. To be able to say, "In order for my services code to take part in the pricing engine, I need to implement this interface." Usually, there'll be very little raw business data that gets shared in that manner and it's really purely a plugin model.

14:19 Udi Dahan

If you're doing that, if you're using IT ops packages with plug-in type interfaces and those are the things that you're referencing, then you're good. If on the other hand, you're having code from the product catalog, which is calling inventory and pricing and cross-selling in those level of NuGet package references, then essentially you're doing an orchestration and you don't really have your loosely coupled services. You need to look back at the decomposition. All right. Next question.

All right. Some people have been asking about Kafka. One person writes, "We currently use Apache Kafka and the producer subscriber model with them for our order fulfillment process. What benefits do we get to move into something like NServiceBus?"

15:14 Udi Dahan

Okay. When talking about pub/sub, let me see if I can just put a reference type picture here, where you've got these types of discreet business events from this type of example, to maybe described order processing. They, sorry, described order processing. Order received or order shipped, or those types of discrete business events.

15:44 Udi Dahan

I think to clarify, we need to first talk about Kafka as a technology versus the architectural style, which is let's call that event stream processing. Which Apache Kafka was originally designed around. Event streaming came about from a domain or from domains where we weren't so much interested in discreet business events, but moreso as a type of, again, stream of things that are happening.

16:25 Udi Dahan

This is very common, for example, in the telemetry space where you've got, it could be IoT sensors or whatever that are periodically reporting temperature, pressure or whatever. And the processing usually, there's no real need for it to be discreet. We'll say essentially roll up the last five or however many events that we got, these type of telemetry events. That's what Kafka was designed for. It's great at that.

17:00 Udi Dahan

And when we're talking about microservices, service oriented architecture, domain driven design, that type of domain of discreet business events, that's where historically speaking, queuing systems, message oriented middleware that RabbitMQ, Azure, NServiceBus, Amazon SQS, IBM MQ, all of those types of technologies, they were designed around that use case.

17:28 Udi Dahan

Discrete business events is queuing territory, where you're processing one at a time. And you care very much about the reliability, durability, what if something goes wrong type of thing. You can make Kafka do that. You can figure out ways to create queuing semantics around Kafka, but as a technology or an architectural style, that wasn't what it was initially designed for.

17:58 Udi Dahan

Now that being said, you've got things like NServiceBus, which are a layer on top of an underlying queuing system. And like I said, you could fiddle with Kafka to make it behave like a cue. I wouldn't say that there's anything wrong with doing that, but I'd say that you want to be careful because the tooling in many ways leads your architecture in certain directions.

18:34 Udi Dahan

One of the things that I see discussed a lot in the Kafka circles is that the events that are being moved around are essentially CRUD type of events. Something was created, something was updated, and now we've got another update, another update, another update, that kind of thing. And they're very entity centric type of events. They also tend to be, let me use the term fat events in the sense that they include a whole lot of business data.

19:06 Udi Dahan

Here, what I'll do is I'll try to find an example. I think I've got code somewhere. Where's the best way that I can show this? When we're looking at, let's say the Amazon checkout flow, where somebody's making an order and there's a shopping basket and a shipping address and credit card information and all of that stuff in a certain order. The way that, let's call it non-service oriented modeling would do something like that is they create a one large order entity and essentially put lots and lots of data into it.

19:50 Udi Dahan

And then when the user clicks submit, then an event will be published saying, "Hey, order received." And that will include all of the information about the order. And then finance service will pick out the credit card information in the order total and it'll charge that and shipping. When it gets those events, it'll pick out the relevant items and the shipping address and it'll ship that.

20:15 Udi Dahan

You end up sharing a lot of raw business data between services. And I'd say that that's common practice in the Kafka world because the assumption is that we're essentially using Kafka as a data distribution mechanism moreso than let's call it a workflow discrete business event type of scenario.

20:45 Udi Dahan

When applying the principles of service oriented architecture, we change the division of data from the beginning. Instead of having, let's say the sales service collect up all the information that the user is putting in, what we'll do, this is give this example in my class, that even before the user starts the order really, when they click proceed to check out, what we can do is we can pre-generate a UUID, universally unique identifier for the order.

21:23 Udi Dahan

And remember that UI composition thing that we were talking about before? Each of the services as shipping, billing, whatever. Remember that picture vertical slices going all the way to the UI? The shipping service would collect the shipping address rather than the sales service, and it would index that by that order ID that was pre-generated. And when you go to the billing screen to type in your credit card information, the billing services UI would collect that information indexed by that ID.

22:01 Udi Dahan

Such that when the event order received is published, it only needs to include the order ID. When finance gets that order ID, it can look up in its own data store, what are the payment choices that were previously stored by it. And then, you don't need to transfer that information in the event. Your events get a lot slimmer.

22:34 Udi Dahan

Let's say that's the primary distinction is that NServiceBus and the service oriented architecture styles that I've been preaching about for the last 15, 20 years are all meant to encourage a much greater degree of loose coupling, a whole lot less data sharing via events or via request response.

22:56 Udi Dahan

And that once you're talking about these type of discreet business events that don't include a whole lot of information, then I'd say that the benefit of something like Kafka and the whole, "Oh, it's going to keep your events streamed in order and you can replay things from the beginning of time." I mean, that's another discussion.

23:18 Udi Dahan

But I'd say that it's, again, Kafka is a fine technology, but when it comes to discrete business events, it's lacking a whole lot of features to make supporting that be done well. And unfortunately, it encourages too much data sharing, which I think causes people logical architectural problems down the road. Right. Next up.

All right. Quite a few NServiceBus questions also came in.

23:52 Udi Dahan

Great.

One is a little bit more genetic. This person is on an old version of NServiceBus and MSMQ and SQL Server and is planning to update, amongst others, containers Kubernetes. The problem, however, is currently transactions in MSMQ and they're afraid, if in my own words, to lose those or to lose the distributed transactions. And the question is, "What patterns can we follow to ensure eventual consistency without transactions? And is there a transport that is showing the most promise given our desired end state?"

24:38 Udi Dahan

Okay. I think you probably need to get on a call with that person to understand more about that their specific circumstances. It sounds like there's a transition in place going from one transport technology to another. In this case, MSMQ to SQL.

24:57 Udi Dahan

There is the ability to forward messages from existing endpoints. What you can do, you can set it up so that... Let me see if I have a slide on that. I don't remember off the top of my head if we've got forwarding included in there. No, not off the top of my head. No. But we've got this documentation online, Dennis, maybe you can find it and share it afterwards.

25:36 Udi Dahan

In your legacy endpoints for MSMQ, what you can do is you can set up a forwarding address. Essentially, the client code doesn't change, continue sending messages to the same queue as before. The processing code that you have there, essentially what it does is it takes those incoming messages and it can reroute them to another queue. And over there you can use this thing that we've got. It's called the transport bridge.

26:14 Udi Dahan

The transport bridge allows you to essentially take messages that are coming in from MSMQ and then pop them over to SQL in a relatively simple manner. And then, you can have essentially the same code that you were running before behind the MSMQ endpoint that is now running behind your SQL transport. That can allow you to essentially do a zero downtime shift where you're taking all of your existing MSMQ environment. And endpoint by endpoint, switching that from running off of MSMQ to running off of SQL, where the client code remains the same as before. You'd probably want that code still talking to the same backend database as it was before.

27:07 Udi Dahan

And then after doing that for a while, then you can start changing the client code to talk via the new transport that you're on. Both MSMQ and both SQL are essentially fully transactional. ACID transaction compliant. I don't think you need to worry at all about eventual consistency if you're making that shift from an MSMQ transport to SQL transport, again, provided that the database that you have behind that is let's say SQL or some other database technology that can participate in distributed transactions.

27:46 Udi Dahan

I'm assuming if you're going to SQL that you're using a SQL database. And then at that, point you shouldn't have any eventual consistency concerns at the level of each endpoint by itself. But as I said at the beginning, if you need more detailed guidance, please ping us and we'll get on a call and figure out how to handle that with you. Okay. Next up.

Another question from someone who attended your course before and mentioned it's brilliant. And he asks if, "My current project using SOA, and it is fine. Is there any motivation to upgrade it to microservices? Or if it's not working, don't touch it?"

28:35 Udi Dahan

I'd say that when people... If people are doing SOA as I've just been showing over here, they've got these larger vertical slices that are then being composed into systems, mobile, backend, whatever.

29:00 Udi Dahan

Then for the most part you've achieved, I'd say the vast majority of the benefits that microservices could potentially bring you. There are other levels of decomposition that you can go within the service boundaries that I also talk about in the course, that you can think of them as kinds of microservices. For example, let me say, I think we have it here.

29:36 Udi Dahan

Inside this shipping service that I was showing, we have code that is processing two events, the order received event, and the customer build event. Okay. And essentially, it needs to correlate, needs to say, "I received event number one, now I need to wait for event number two to arrive." And then do some processing and then finish that.

29:59 Udi Dahan

That type of code, if you're using NServiceBus, we have a feature called sagas that allows you to implement that. Essentially, it's a long-running process or event stream processor, if to use the Kafka terminology. A component like that, a saga can be deployed by itself, separate from the rest of the system, which enables it to be independently monitored, deployed, scaled, et cetera.

30:40 Udi Dahan

A lot of times when you're using, you're building message driven type systems, your backend can be turned into these independently deployed and scaled components. The fact that you're doing SOA already takes you, I'd say probably a good 70, 80% of the way there. Layering in messaging, you're probably 95% of the way there. At that point, it's just looking at your components and saying, "Would I benefit from taking this message handler, this saga, and deploying it to a separate process and scaling that separately?"

31:19 Udi Dahan

If you have a very high load that that component is processing, then the answer is yes, probably. And then at that point it becomes an independently deployed microservice. But the benefit becomes more of a deployment time scaling type of thing in terms of the diversioning that people talk about. Say, "Oh, by having things as separate microservices, I conversion them separately."

31:55 Udi Dahan

If you have just like... Let's see, where's this picture again? All right. If you've got some backend, all right. This is this purple thing going horizontally. It's got three components inside it. Now those components, they don't actually talk to each other, not request response or anything directly. They're hosted together for let's say a matter of convenience because we don't need to scale them out.

32:23 Udi Dahan

At that point, they're largely independently deployable. It's like, "Oh, there's a new version of this S3 component? Great." Just pull that down and deploy it, recycle the processes and you're done. You don't have to touch the other things because they're not referencing it.

32:40 Udi Dahan

The loose coupling at the SOA component level, like I said, it'll give you probably 90% of the benefits that you imagine you're going to get from microservices, the last 10% for scaling and whatever, you can do that if and when you need it. But other than that, I'd say if what you're doing is working, don't go chasing fads, microservices or otherwise. All right.

All right. Another question. "Dapr seems to be quite a popular framework. How does it compare to NServiceBus and what are some of the pitfalls of Dapr?

33:18 Udi Dahan

Dapr for those of you don't know, is this thing over here. They call it an API for building portable and reliable microservices. Now, there's that word microservices again, which is a tricky overloaded word.

33:35 Udi Dahan

Just a little bit of history. Dapr was originally created by a couple of folks within Microsoft. It was however, never part of the Azure organization. It was never part of DevDiv. It's not officially supported by Microsoft. Technology is now part of the CNCF, if I'm not mistaken. Yes, it's a cloud native computing foundation project. It's out there in the open. They're lots of people interested in it.

34:07 Udi Dahan

I think the best way to describe it is more as a virtual actor framework. It is more similar to if you're familiar with Akka.Net or Orleans, that would be the better example to connect it to. And then by that token, it's a, what is it good for? I'd say it is good for domains where actors are good.

34:32 Udi Dahan

For those of you don't know, there are a couple of attributes to that. It is usually places where the speed of the response is paramount. Most of the actor models essentially keep everything in memory. While Dapr and others allow you to have things in a durable store, again, I'd say the architectural style, the history of it was for low latency types of domains.

35:12 Udi Dahan

Think of this presence. For example, the social networking, think about Slack or teams or any type of chat type application to be able to know is somebody online or not? Is somebody typing right now or not? Those types of events are things that are great to use actors for. You could have a user object, an account object, and that you're propagating these types of events where the latency of showing somebody online or offline or that they're typing or that they're not typing or those types of things, very important.

35:52 Udi Dahan

You see this in online gaming, chat as well. Other domains is online ads. When somebody's going to a website and you need to show them ads. Communicating with a number of ad providers to say, "Hey, we're running a bid for this page. Here's the user. Who's going to bid how much you're going to get your ads shown?"

36:16 Udi Dahan

Low latency, they usually tend to be request response in nature of the interaction because of the latency domain. That's I'd say where virtual actors shine as a whole, again, Akka.Net, Orleans and Dapr would fit in there.

36:37 Udi Dahan

Where does that compare to something NServiceBus? NServiceBus is more interested, as I mentioned before, in the discrete business events type of state. I'd say things that are more interesting in the transactional backend processing side of things. I gave this example over here of saying, processing an order that was submitted, billing a customer, shipping an order, integrating with third parties where you want high degrees of reliability and fault tolerance and you're willing to sacrifice some amount of latency for that.

37:14 Udi Dahan

That's where NServiceBus has a framework and the underlying technologies that it uses in Azure, that would be Azure, NServiceBus, CosmosDB, et cetera. In Amazon. That would be SQS, SNS, DynamoDB. On-prem, that would be RabbitMQ, MSMQ, SQL, server, et cetera. Back-end transactional processing is the sweet spot, if you will, for what NServiceBus is trying to do.

37:46 Udi Dahan

Less about propagating these relatively lightweight, I'm going to use the term notifications. Somebody is typing, somebody's online, offline, somebody's moving around on a map in an online game, those types of things that's where your actor frameworks, low latency stuff works great. Again, that's as a broad, overarching type of statement. Of course, every one of those technologies can be banged to fit and has all sorts of additional features.

38:17 Udi Dahan

One of the things that I'd say, I think there was a question of what to be careful with Dapr. Dapr has this functionality. I don't remember exactly where it is in the docs, they have a transactions feature. It's not going into that right now. I'd say be very careful about using the Dapr cross-entity transactions feature because that's essentially the bad form of distributed transactions/core buff from back in the day.

38:56 Udi Dahan

If you got actors that are enlisting other actors in a larger distributed transaction, you're trying to do a workflow thing, but you're trying to do it transactionally. That's not a good place for doing actors. You're essentially creating almost the worst form of distributed monoliths at that point. I'd say that's your big no-no. All right.

Lots of questions on how to convince an organization from certain things. I picked one which is right up our alley and probably interesting to a lot of people. "I believe our organization would benefit from a message-based architecture, but I'm having difficult time making a case to the people making decisions. I think it could reduce cost, et cetera. The primary concerns seems to be that a reply is not immediate. What can I do to make a stronger case?"

40:03 Udi Dahan

Right. I'd say it's a process and start with the easy stuff. Let me see if I got some examples of the easy stuff in my messaging patterns place. I'd say generally speaking, people are afraid of change. Meet them when they are. I'd say at a first start level, don't start with a message broker like RabbitMQ or something like that if they're, again, you have this freaking out nature.

40:46 Udi Dahan

One of the things that we've done with NServiceBus is that we support using a regular SQL database as the underlying queuing system. I'd say that's step number one is don't introduce new infrastructure right away. Another thing that I'd suggest is start with processes that, from a business perspective, are naturally asynchronous.

41:19 Udi Dahan

For example, many systems have the need to generate a PDF. Okay. Or sending an email. Those types of things can take a while and HTTP request response can time out over there. I'd say those types of things, I'd say I talked to the organization saying, "You know how we're calling Mailchimp or some other third party thing in order to send emails for us? If we put a queue in there that could make our email sending a lot more reliable."

41:54 Udi Dahan

If our Mailchimp, Twilio, whatever, the thing that we're talking to for sending out emails or notifications might be temporarily down or overloaded, by introducing messaging at that integration point, we're going to get a whole lot of benefits. And for the most part, we're not even waiting for a response really beyond them to acknowledge that yes, they are going to be sending those emails or that PDF is going to be generated. I'd say those would be good places to start.

42:28 Udi Dahan

Another place to look at are batch jobs. We've got this blog post that you can forward it on. Death to the batch job. A lot of organizations have batch jobs. Batch jobs, just to define it, usually you have things that run middle of the night that are going to some database table, extracting data from there and doing something with it.

43:02 Udi Dahan

Now essentially, the database at that point is essentially being used as a queue. You got a front-end system that's putting stuff in a database and you got a batch job that's pulling stuff out of the database and processing that. One of the downside of batch jobs is they're running in batch, meaning they're processing lots of things at once. That often means if one of those things fails, the batch job might need to restart again. And then you need flags to be able to see, have I processed this thing before? Batch jobs get to be quite complex.

43:36 Udi Dahan

The other problem is that if you end up having too many things building up, the batch job might take longer to run than just the middle of the night. And then it starts running into the daytime hours and overloading your database and then that makes the rest of your online system really slow. I'd say talk to the organization, say, "Do we have any batch jobs that are painful? That's a great place for us to introduce some messaging. We can make that, we can preserve the same level of reliability, we can simplify that, we can process it in more realtime. We can offload the database by introducing an actual message broker so that if the batch jobs are a little slow, then they won't slow down the rest of the online system."

44:26 Udi Dahan

To the question, I'd say those are the places to start. Have the organization experience some of those benefits and say, "Hey, yeah, this messaging stuff is really quite powerful." And do more and more and more of that. And eventually over time, you may get into the places where before, it was synchronous request response type of interaction over HTTP. But I'd say that's not the place that you should start with the more async type of places that I've just mentioned.

NServiceBus specific question, what are some of the reasons behind the decision not to have an outbox in NServiceBus in a send only end point?

45:12 Udi Dahan

Oh. There was a detailed design discussion by the task force that built the transactional session feature. For those of you don't know, just going to open that up for you. The transactional session feature is where you've got, for example, a web front end that gets requests, goes, puts them in a database, and then publishes or emits some message to a message broker.

45:44 Udi Dahan

And you want to make sure that you've got a strong degree of consistency between all of these actions. There were a couple of ways that that feature could have been implemented. The choice was to leverage the outbox that we already had. And it was tested, I don't want to say bulletproof, but it was used by many companies in production scenarios under very severe levels of load and had proven itself, battle tested and hardened and all that kind of stuff.

46:21 Udi Dahan

The thinking was it will be the fastest time to market and the highest reliability choice to have this transactional session feature make use of the preexisting outbox that we already had built. But what that required was the outbox works using a queue on the incoming side. That's essentially the conflict is that, in order for transactional session to work by using the underlying outbox, it needed a queue. And that's the conflict with the send only endpoints.

46:59 Udi Dahan

Again, for those you don't know, send only endpoints in NServiceBus are endpoints that emit messages but don't process messages, meaning they don't have an incoming queue. That's the distinction there. That's essentially how the decisions were made, why they were made that way. For more detailed discussions on that, we'll happy to take that up with you afterwards, or you can pop to the NServiceBus discussion group and we can discuss it more there also out in the open. Okay?

"Thanks for providing access to the video course during the lockdown," someone says. "And I still struggle to grasp IT ops logically. Can you get a bit more concrete on how, for example, code or a project structure would look like?"

47:50 Udi Dahan

Right. This is where we're back to that SOA workshop that we're talking about. If we look at some of the exercises that we've got here in terms of composite UI. There's a bunch of things that exist here. But for example, what you'd want to be looking at is let's say the composition gateway.

48:24 Udi Dahan

Essentially, it's all of the framework code that supports the composition over the top of it. If I start opening up code and walking through it's going to take quite some time. Again, what I suggest, if you haven't looked at this workshop, open up the workshop. If you have questions, get in touch with us afterwards.

48:47 Udi Dahan

But it will be in terms of how concrete do you want to get. Once you go into that, you'll say, "Oh, okay, but that composition gateway was running server side. What if I want to do it in a spa on the client side?" Then that also exists in the workshop, but shown in a different exercise.

49:06 Udi Dahan

And then we get into the nitty-gritty detail of saying, "Right, but I don't use Angular. What would that look like in React?" Every single framework choice and environment is going to look different. We don't have the time to get into that degree of concreteness on this call. But follow up with us and we'll be happy to chat.

"What is your recommended first step for a system that is a monolith, that it doesn't have a complex domain model?" And an example is given, "Like a DevOps software as a service. And wants to achieve the scalability deployability on all the other benefits of microservices. How can we separate and find the right boundaries inside this system, which has not a very specific complex domain?"

50:00 Udi Dahan

Okay. If I'm trying to read between the lines of the question, to me this sounds almost like a startup type of domain. Where you have, now startup could be that this is indeed a startup as a company, or it could be that this is a relatively new business unit, relatively new business system, where the first or earlier versions of that thing are less complex.

50:33 Udi Dahan

My recommendation is don't microservices, don't SOA don't DDD, don't nothing. Right. You've got a simple code base. I'd say continue to keep that as simple as possible and try to focus on the things that your users care about. Try to get more users, try to have them be more successful using that system.

51:01 Udi Dahan

Worry about those things. As you do that, users will start getting in touch with you, say, "Can you build this feature for us? Can you add that feature?" You're going to start getting some more complexity that is being driven by your users. I think this is a general thing that, as engineers, we try to anticipate future complexity and try to make architectural choices to mitigate that future complexity.

51:30 Udi Dahan

I'd say as an industry, we have an absolutely atrocious, horrible record on anticipating complexity and having the architecture reduce the cost of future complexity. I'd say don't try and try to keep things as simple as possible. But using certain patterns, as I mentioned before, batch jobs, I try to stay away from them, making things async where they can be.

52:06 Udi Dahan

As I mentioned, integrating with some other system that isn't always available, use a little bit of messaging there. I'd say those kinds of patterns can be helpful. They're let's call it tactical patterns that reduce some of the complexity, improve some of the scaling behavior and reliability of the system. But I'd say don't cause problems for yourself by trying to get ahead of anticipated complexity that isn't currently there. Last question and then we have our surprise type of giveaway thing that we'll let you all in on.

The last question is one that I hear myself, hear others ask a lot as well. "What kind of books or other resources would you recommend to learn about event-driven architecture?"

53:07 Udi Dahan

Oh, so I'd say it's not just about event-driven architecture. I think that one of the problems with people in the event-driven architecture space is that the term event is hugely overloaded and used far too much.

53:28 Udi Dahan

Let's say before getting all excited about event-driven architecture, there's this book, Enterprise Integration Patterns by my friend, Gregor Hohpe and Bobby Woolf. There's also a website for it. This, sorry, is the book. It also talks about events, but not only about them.

53:55 Udi Dahan

It'll give you a much larger repertoire of patterns to use from, so message routing and enrichment. You see some of these patterns over here, the routing slip, process manager, envelope wrapper, claim check. Anytime that you need to have a larger amount of data. Again, to be passing along a PDF, you don't want want to be putting the PDF itself inside the body of the message. Essentially, you put in what's known as a claim check, an identifier that links to that thing later on.

54:39 Udi Dahan

Dead letter channel. Oh, this is one of those things that can for all of the folks that are doing Kafka as well, end up having to build something like this themselves. And then to say, "Well, what do we do with the things that end up in the dead letter channel?" And then how to reprocess them.

54:59 Udi Dahan

There are lots of elements that are in here. This is an older book before everything got really popular around event-driven architecture and serverless and all of that. I think it's hugely important to understand. It also discusses the difference between a command message and an event message. None of that create customer event or delete product event. If it's a command, it should be a command, it's not an event.

55:29 Udi Dahan

And a lot of times, sometimes people make the mistake of doing document messaging or document-oriented events, where essentially they're passing a whole product as a message or a whole patient record as a message. All of these things are patterns. They have names, there's a book about them. I'd say this is a great place to start if you're unfamiliar with it. And the website is also very useful.

I'm the creator of NServiceBus. Ask me anything.

🔗Why watch?

🔗See Udi respond to questions such as:

🔗Transcription

About Udi Dahan