NServiceBus past, present, and future

00:00:12 Udi Dahan

All right. Well, I guess this is the moment that I'm supposed to dance around on stage screaming developers, developers, developers, developers. I mean, everybody has to have their moment. I'm sure you've all watched Steve Ballmer doing his thing inside. Yeah. I'd never do something like that. But I am really excited. This is a cornerstone moment for me, having started NServiceBus as really just a little project on the side. That's really the story that I want to talk to you about today.

00:00:55 Udi Dahan

In talking to developers around the world about NServiceBus, a lot of times, they're just like, "Yeah. I found it. I heard from a colleague. I downloaded it. It's really cool." But they don't really know much of the backstory. I wanted to make sure that you knew where it came from before talking about where we're at today. I don't want to call it prognostication, but some hints at what we've got coming for the future.

00:01:26 Udi Dahan

Let me just give you guys a moment if you haven't looked around, because a lot of times, people ask me, "Well, who else is using NServiceBus?" We always have this tension with customers saying, "Can we put your name on the site?" Then the developers are like, "Sure. Yeah. That'll be awesome." Then legal comes along and says, "No." It's really hard for us to tell this story of who's using NServiceBus. It's always this really long legal wrangling type of thing.

00:02:01 Udi Dahan

Just take a minute, look around, get a sense of the people that are showing up at the conference, talk to them. Hear about what they're doing with NServiceBus. One of the things that surprised me very early on with NServiceBus was I had a certain idea as a consultant of what I was using it for. I was ultimately scratching my own itch. I think that that's how a lot of open-source projects start before looking at some big, gigantic story around it, it all starts pretty small.

00:02:37 Udi Dahan

With just scratching my own itch, I need this for this project and wrapping up this little library and carrying that from one place to the next. I did not intend at the beginning to even create an open-source project. It was just a, "Well, this is what I need to solve my own problems." After going through three, four, five projects using the same libraries over and over I said, "Oh my god. This is a thing. It's not just a temporary little itch that needs to be scratched." Then going on and making it open-source.

00:03:16 Udi Dahan

For those of you who are feeling a little bit nostalgic, it's always nice to reminisce back. But let me show you just how terrible things looked in the past. For those who don't know, or didn't see it, this was the first website that came for NServiceBus. It was pretty much not very much more than a blog with an nServiceBus word on it, lowercase n at the time, because that was the thing for open-source projects in the .NET space, lowercase n at the front.

00:03:57 Udi Dahan

There is the nice, "Oh, yeah. It was so much fun." The great thing as I remember back was that I could actually spend my time coding on NServiceBus quite a lot. Unfortunately, nowadays I don't get to actually spend my time in the guts of the code base because otherwise the guys over there would take away my keyboard.

00:04:17 Udi Dahan

Yes. It was great Udi at the time, NServiceBus was awesome. Nowadays, we've got much bigger problems to handle. We have much stricter processes in place. Andreas will be talking about all of the big complex things that we do internally to make sure that everything is 100% solid. But part of it in terms of the history or the journey was over time we said "Oh, okay." Well, moving from a blog and then we started doing training at Skills Matter.

00:04:50 Udi Dahan

Skills Matter for those who don't know, very longtime partner of NServiceBus. The first NServiceBus course was delivered here. They've been with us throughout this entire journey. We're really happy to have this event here with them. From there, we got this idea of maybe we should have a Download Now button, maybe people should actually have a decent way to find how to download it.

00:05:17 Udi Dahan

But for those who don't remember or weren't using at the time, a Download Now you just got a zip file, and you extracted it to some directory. It was really just a bunch of DLLs. That was it. Here you go, NServiceBus. Documentation-wise, there really wasn't very much up there. Over time, we gradually grew and improve the website and started having to deal with the whole issue of commercial licensing. Commercial licensing, for those of you don't know NServiceBus is currently commercially licensed.

00:05:52 Udi Dahan

I got to tell you, I was sweating bullets the day that I announced that NServiceBus would be going from free open-source to commercial. But the reality was that it had been probably somewhere in the area of seven or eight months that I had not written a single line of code of NServiceBus. I was just spending my time doing training and consulting. It was great for me personally, as a consultant, being able to work on with clients on the project on my baby.

00:06:30 Udi Dahan

But realizing that months and months had gone by, and no improvements had been made. I began to extrapolate going forward. I'm only getting more busy in terms of professional services. But if another year goes by, and there's no new features, that's going to become a problem. Everybody who adopted NServiceBus is going to start wondering, "Is this project?" Well, on the Microsoft platform, we never call things dead.

00:06:55 Udi Dahan

You get me wondering if NServiceBus is done, not dead. That was something that worried me, because here I was as a consultant, and I take my role as a consultant very seriously. If I recommend a client use a piece of technology, then that's my reputation on the line. If this project would not continue evolving and developing and have a future, then I could not with full integrity, actually recommend to anybody to use this technology.

00:07:27 Udi Dahan

What I realized was, unless this thing becomes commercial, unless there is actual money coming in, that will fund the future development, the project will die. For those of you who have been around the open-source space, you tend to see that happening. Your project runs pretty well for a year, two years, three years, as long as the lead developer is working at a consulting company, where they can actually get away with using it on their projects.

00:07:57 Udi Dahan

Then they move to a different company. That's at the project step. Everybody who used it is now scrambling, saying, "Well, how do we replace this with something else?" That was over three years ago. I'm glad to say that since then, we've really ramped up not just in terms of improving the website and adding more types of information online and then presenting ourselves as particular software and all that stuff.

00:08:30 Udi Dahan

The fundamental thing is that NServiceBus has a guaranteed future now. If you look around more towards your right, you'll see a growing cadre of people wearing these white shirts. That from year to year, it's always a couple more, a couple more or five more. It's been growing all the time. This is our first NServiceBus conference. We're looking forwards to having quite a few more.

00:09:01 Udi Dahan

But before jumping directly into the present, I want to get a show of hands to get a sense of where you guys got on the train? Who started using NServiceBus in the 1.x days? Okay. Wow. Oh, my god. Look, there are a bunch of hands up. Thank you very much and I'm so sorry. There were so many things that seemed like a good idea at the time, but in retrospect, we're extremely dumb. Like ILMerging, every single third party dependency into the NServiceBus DLL.

00:09:43 Udi Dahan

I'm really hand on heart. We wanted to make it easier to get started with. But it was an extraordinarily dumb idea. 2.x, who started in the 2.x timeframe? Wow. Look around for a second. Keep your hands up. Wow. Look at all of you. Guys, I'm still sorry. We did different dumb stuff in the 2.5 days. But it definitely was better than 1.9.

00:10:18 Udi Dahan

Then 3.x? Okay. Some of the younger guys that had the more reasonable experience, the fact that there was actually some documentation online beyond a random blog post somewhere. That was a huge step for us. Not to say that we don't have more to do in that area. Anybody just started just now with 4.x? Okay. Wow. A bunch of hands coming up. Good. I hope a year from now I will not be standing up here saying, "Oh, I am so sorry."

00:10:59 Udi Dahan

But it has gotten much, much better. For those of you who have lived through the history of the 1 point, 2 point, 3 point, et cetera, we have been hearing from you guys. That's one of the things that if anything, we try to remain true to our open-source heritage. We don't believe that we're the smartest guys in town. We were constantly learning from you guys. It just boggles my mind in talking to customers, the number of different types of systems that are being built with NServiceBus.

00:11:34 Udi Dahan

That's one of the things that either myself or the team when we interact with customer and say, "Well, why don't you just do x?" Some idea was like, "Well, that sounds like a great idea if you're, say, an online software as a service company." But we actually have companies, can you imagine it that actually don't run on the web. Okay. They actually install their software on client machines.

00:11:57 Udi Dahan

There are a bunch of these customers in the healthcare domain. They write software that is installed at hospitals and doctors' offices and pharmacies and is just in a micro little chasm and managing those environments. What makes a huge amount of sense for online software as a service doesn't necessarily make as much sense over there.

00:12:22 Udi Dahan

We've got companies that are using NServiceBus to track trains and trucks and barges and cranes and heavy equipment and collecting sensor information from oil rigs. When I started building NServiceBus, that was not foremost in my mind, people in doing stuff on oil rigs with NServiceBus. Media companies that while you have this, "Oh, yeah. That's very online." There's a huge back office story for media companies.

00:12:56 Udi Dahan

Then there's just, let's call it, the boring enterprise customers. If you're working in those boring enterprise customers, believe me, that's where I come from. I love the big boring enterprise customers. It sucks as a consultant, because you can't ... your next gig point to I built that website, and I build that website. But working internally with people in solving problems for employees, the thing that I like most about these types of internal organization projects is, a lot of times the users are using these systems every single day, all day.

00:13:34 Udi Dahan

It's a huge deal for me to build a system that I know this is going to be taking up a massive chunk of somebody's everyday experience. That's where I came from with NServiceBus. That's why it was such a big deal for me to make sure that this thing was bulletproof, that it was solid, that it was robust. Because if somebody is using this constantly, little types of problems, it drives me crazy as a user of software. I'm curious.

00:14:05 Udi Dahan

For those of you who started with version 1.x, who still has a version 1.x system in production? Okay. We got one hand in the back over there. It's a double-edged sword now that I'm in the commercial space. Talking to customers like, "You should upgrade." They're like, "Why? It just works." It's really hard to have a conversation with somebody that controls a budget. He says, "But why would I upgrade? I mean, it's great that from version 1.9, you're now in version 4, and you're getting ready to roll out version 5. But we have the system built on 1.9. Ever since we put it there, it just keeps on ticking. Nothing bad happens ever."

00:14:56 Udi Dahan

It's really hard to get people to want to upgrade when you build reliable software. But I am very glad that it's been able to stand the test of time. Sorry. Who's still on 2.x? Wow. I see the more hands towards the back then towards the front. Afterwards, we'll need to do some analytics on human psychology as to why that got set up. It's one of those areas, for those of you that are still on 2.x. This is not purely technical, but it is on there.

00:15:34 Udi Dahan

We, in recent years from say, from version two to three, we invested more in creating a solid upgrade path, meaning that if you have a version two system in production, you want to move to version three. It's going to be easier than say, doing a 1.9 to a 2.5. Okay. But this bit is still not great. However, from this point, going forwards, to say that we are applying SemVer is really superficial.

00:16:06 Udi Dahan

Andreas is going to talk lots and lots about this. But our biggest story that we want to take forward for you guys, and for you to know is that when you put an NServiceBus system into production, you will be able to run it side by side with other NServiceBus systems on adjacent versions, wire level compatibility to synchro sig. We really take a lot of care in making sure that not only is a given system in and of itself bulletproof in terms of reliability.

00:16:37 Udi Dahan

But if you continue rolling out NServiceBus across your organization, that it will work end-to-end constantly. That's really quickly in terms of history. Where are we today around the world? We got all of these little stars. We are not surprisingly, a distributed company. It was one of those things early on that I didn't really think a lot about. But it makes sense in retrospect. Got people in the US, West Coast, in Midwest, and Sweden, and Northern Europe, and Italy, and Australia ended up being a major hub for us.

00:17:20 Udi Dahan

Of course, the Aussies that flew all the way over here are somewhat complaining like, next NSBCon down under. We've been expanding all the time, as I've said before. Part of this, it's not just in terms of being a distributed company that we can hire the best people wherever they are. We are still hiring, by the way. But also from a support perspective that when we say that we provide 24/7 support, it is because we have people all around the world that know NServiceBus deeply and have built systems with NServiceBus.

00:18:01 Udi Dahan

If you do have a problem, and whatever it's 3:00 a.m. for you, it's not going to be 3:00 a.m. for somebody else. They're going to be bright and fresh, and you're going to know how to handle your problem. If you have those problems, I'm happy to say that we have the infrastructure to make sure that you will never be on your own facing a production problem.

00:18:23 Udi Dahan

Now, the other part of where we are today is that it's not just NServiceBus anymore. I know that's our heritage. But now as particular software, we rolled out a bunch of other apps around NServiceBus. Primarily, and sometimes people, "Why not just keep focusing on NServiceBus?" Believe me, NServiceBus-wise, we have a roadmap and a backlog a mile long. There are so many things that we want to do.

00:19:06 Udi Dahan

There just purely NServiceBus that we could spend the next five years and not get it done. But the big thing for us is ultimately in talking to people like you is hearing stories about monitoring. Who wished that NServiceBus came with a better monitoring solution when they adopted it the first time? Okay. Yeah. Most of the hands are going. It was one of the things. Yeah. NServiceBus is great. But we really want some monitoring.

00:19:40 Udi Dahan

We created this tool called ServicePulse. I don't want to say that it's beautiful, that it solves all of your needs, but the baseline that we were coming from was, well, it's great that NServiceBus fails messages into an error queue. But it's hard for administrators to be looking at an error queue or to pull out a performance counter that says, "The number of messages in an error queue and to be looking at that as a part of a dashboard."

00:20:18 Udi Dahan

That's something that we did, just because ultimately, everybody needed it. Also, for those of you who don't know, when a message fails, it takes over the stack trace of the exception, which caused it to fail. All of that is available in this ServicePulse environment. It makes sure that, let's say, all of the currently, if I'm aware where we are today, not where we're going.

00:20:43 Udi Dahan

That everything that was available to administrators before, but required a lot of manual work to say, "Oh, okay. Well, how do I find out what the stack trace is of a message that failed in their queue?" "Oh, very simple. You open up queue explorer. You go to the message. You look at the headers of the message. In there, you will see a header that says, 'Stack trace.' There, you've got the stack trace. What could be simpler than that?"

00:21:07 Udi Dahan

After you've had those conversations with administrators, begin to realize that may be a little bit more graphical tooling is required there. That's this element of failed messages that make sure that if something goes wrong, an administrator will get notified about it. Now, in terms of notification, I know if you've been looking at this tool, if you've tried this out, well, a little dancing red envelope.

00:21:34 Udi Dahan

While that's some level of notification, we'd like email, we'd like SMS, we'd like all these other things, that's coming. Okay. Like I said, there's a million things for us to do. We are getting through them as quickly as we can. But as the rest of the guys over the course of the conference will start telling you, "There's a much larger story around this. We want to make sure that you understand what we're doing, why we're doing it, how we're doing it."

00:22:03 Udi Dahan

As I said before, to get your feedback. If we're doing something dumb, or that there are ways that we can improve it, let us know. Okay. We're very open. One of the things that I appreciated most about NServiceBus from the early days was its community. All of the wonderful ideas. Pull requests, by the way, absolutely 100% welcome. Please do send them over. Code-wise, we've also done quite a bit to make it easier and more accessible to get into the NServiceBus code base.

00:22:37 Udi Dahan

If you really want to learn how it works under the hood, it's possible as well as all of these other tools. In addition to this ServicePulse failed messages thing, we added a little bit of heartbeats, meaning that endpoints that are instrumented with the ServicePulse and the service control plugins will be reporting that they are online. If they stop reporting that they're online, you'll get a little notification here saying, "Okay. The sales endpoint stopped sending me heartbeats." Then administrator can note to take action.

00:23:13 Udi Dahan

The Custom Check thing is currently our main point of extensibility that allows you guys to hook in any check that you want. Meaning, if you're integrating with a third party, you could just in your custom check, invoke a get a query against the third party. If you get a response, they say, "Oh, okay. It looks like the third party is there and answering my questions."

00:23:40 Udi Dahan

What often happens for people that are doing production monitoring is that you'll see all of a sudden, the number of failed messages just goes way up and just a couple of minutes. Administrators, they're looking at the system, everything's great. All of a sudden, 100 messages; all of a sudden, 500 failed messages, 10,000. There's this, "Oh, my God, what happened? How do I even go about tracking this down?"

00:24:02 Udi Dahan

The custom check thing, if we cannot contact a third party, just put in that custom check. If an administrator sees 10,000 failed messages, at the same time that a single custom check goes wrong, it's going to make their life quite a bit easier, say, "Oh, duh. Third party went down. Pick up the phone. Call them." Once their system is back online, and the custom check clears again, replay all of the messages, and you're golden.

00:24:33 Udi Dahan

It's a very simple extension mechanism can help administrators manage the failed messages quite a bit better. You can plug in any number of custom checks that you want. Very useful. But like I said before, there's still a whole lot of work for us to do in that area. Then there's the debugging parts of it. Who has had a hard time debugging an NServiceBus system? Please be honest.

00:25:04 Udi Dahan

Okay. I apologize for that. But this is something that's inherent to a lot of message-driven systems. This is not new with NServiceBus. When I was originally starting NServiceBus, meaning ultimately working with MSMQ at a relatively raw level, I had these exact same problems with debugging these types of systems.

00:25:27 Udi Dahan

Other people that were using WebSphere MQ or typical rendezvous, in the messaging space, the good news is your system is very loosely coupled, which means you can change one part of the system and know that that code is not going to break another part. The bad news is with all of this loose coupling, it's really hard to get a sense of what on earth is going on across my system.

00:25:49 Udi Dahan

What we've started to do, and if you've already seen ServiceInsight, that that's where it's coming from. But the idea was how can we take something that NServiceBus already knew how to do, which was auditing? NServiceBus does auditing out-of-the-box, meaning every message that is processed by an endpoint is put into an audit queue. Well, that's nice. It's not very usable.

00:26:18 Udi Dahan

Again, all of the queuing systems have this. They had MSMQ calls of journaling. RabbitMQ, they don't really have a specific name for it. But you can bind any queue to an exchange, and then you now have built-in auditing that way. All of the queue support this. But the challenge was, how am I able to connect a message flow across multiple endpoints when I have 10,000 submit order messages, and 10,000 order accepted messages, and 10,000 order build messages, how do I line all of these things up?

00:26:56 Udi Dahan

Well, clearly, you can do some custom thing where you are querying the audit log based on a common order ID that's in there. But there might be a case where there's some messages that go downstream from that, that say, "As a result of an order being accepted, a customer became preferred." Then you no longer have the order ID in the customer became preferred message.

00:27:21 Udi Dahan

Then once again, it's difficult to know the causation relationship between what flowed in your system. Yes. I've got an audit log that tells me everything that happened, except for knowing what caused what. That was the area that I was starting from with auditing in message based systems. What we've done since then in the NServiceBus core is that every time that a message is emitted, we put in through the headers a little breadcrumb that allows us to be able to trace in the audit log, which message caused which other message caused which other message.

00:27:57 Udi Dahan

That allows us after the fact to be able to build visual graphs like this that can tell you, "Oh, okay. This was the order of causation. It makes the audit log all of a sudden useful." That was one of the things, especially in production environments when user saying, "Hey, the system behaved weird. It didn't blow up." That's the story when we're talking about over here.

00:28:26 Udi Dahan

When things blow up, they blow up and make a noise. You can see what the story is. But sometimes it's just as simple as, "Hey, the system is working weird. I'd like to know if maybe an order wasn't shipped." Just being able to visually track for a specific order, say, "Did this happen or not by an audit log?" This becomes quite a bit easier. Now on the production side of things, this is useful.

00:28:50 Udi Dahan

Hand on heart. The biggest thing for me was actually building a system. When I have 5, 10 different endpoints that I'm coding on at the same time, having something being able to show me what the cross endpoint message flow was, and where was an event supposed to be published, but didn't get published and all that stuff. It's been a lifesaver for me in working with clients, definitely the other people on the team that have been talking to clients.

00:29:19 Udi Dahan

I'm curious who's played with ServiceInsights so far? Okay. Some hands going up. Please leave your hands up. Okay. Who likes it? I'm looking for hands going down. Anybody not like it? Okay. Good. You guys saw who left their hand up? Take him out back afterwards. Make sure there are no cameras around. There are still improvements to do on this tool and everything else.

00:29:53 Udi Dahan

But what we are trying to do is to take those kinds of things that that have been difficult to do in message centric environments, quite frankly, for years, and move the ball forwards without creating a great deal of complexity. This is where, again, in terms of where we are today, if we're going to be talking about complexity. I have to talk about our drag and drop.

00:30:28 Udi Dahan

It's a question on how you look at it, whether you read it left to right or right to left. Is it drag and drop for the win? Or drag and drop, what the fuck? Again, honestly, for years, as a developer, every time somebody showed me a drag and drop type tool, usually I gave it a chance, said, "Okay. I'll play with this for five minutes." But invariably, they ended up falling short. The goals are always laudable in terms of having some graphical tooling type of thing.

00:31:11 Udi Dahan

But there's a very big difference between, let's call it analytics of the system that you already built versus using graphical notation to build the system to begin with. Most developers like, "Yes. We like monitoring that has lots of graphs and icons and whatever." That's good. Do that. "Yes. We like visualization, debugging type of tools." Great. But you go and tell us to actually use a drag and drop type thing to build the system? We've been bitten too many times in the past to really expect that to work out very well.

00:31:57 Udi Dahan

Just so you know, I've always been skeptical of this. But I saw a demo, I think it was about five or six years ago, of some tooling. I said, "Wow. That is extremely powerful." The fact that it can be done inside Visual Studio and be fully synced up with your code was a huge deal. Ultimately, what we tried to do with ServiceMatrix, and for those of you who don't know it, ServiceMatrix, we've been on the sidelines for over two years now, been building up this tooling.

00:32:39 Udi Dahan

We feel that it's getting to that point that it's ready for primetime. Who's seen ServiceMatrix, so far? Okay. A bunch of you. Who hasn't seen it yet? Okay. Oh, also quite a large chunk of you. In ServiceMatrix, ultimately, it's an extension to Visual Studio that gives you this canvas environment that you can use to create endpoints. You can have these endpoints, send messages, and publish events, and subscribe to them all with fairly simple point and click type of model.

00:33:14 Udi Dahan

Where ultimately what gets generated is a bunch of placeholders. For example, if you have an endpoint that subscribe to the order accepted event, there's a handler class over here, it's your code. You do whatever business logic you want in there. The thing that we're trying to solve, at least in this early phase, is the problem of developers. Not really sure why a message that was published from this endpoint did not arrive at that endpoint. Has that ever happened to you?

00:33:54 Udi Dahan

You set up publish subscribe and the messages didn't go across? Yeah. I see a bunch of hands up. You're scratching your head and you're looking at the config. I think that's how to do it. Then you run it again, and the message doesn't go across. Then you look at the documentation, and then you copy and paste a bunch of config from our documentation into your app config. Then you run it. Then it works.

00:34:21 Udi Dahan

Then you switch back to what you had before and you're like, "But they look almost exactly the same. Why is this working in that one not working." It could be things like the message types that you specified in the routing config just ... the message type, it was not an assembly qualified name. But because you had very long names, you didn't notice that you were missing the assembly qualified element of it, or some such other stupid things that nobody really understood.

00:34:57 Udi Dahan

It felt a little bit like voodoo. You do the right dance and you sacrifice the right animals and it works. Then you tell developers, "Don't ask me why it works. It just works. Leave it alone. We'll just take that to production." In talking to customers and hearing that story, I'm like, "Oh, my God. Are you serious? Is that what you're doing? I'm so sorry."

00:35:26 Udi Dahan

What we've tried to do is say you want to set up publish subscribe, how about we just say, "Okay. This thing over here, just say 'publish an event.' That thing over there, say, 'subscribe to that event.' We'll generate the config." Because quite frankly, up until now, I have not met a single person that actually gives a damn about how the config is set up. They just want it to work, preferably as soon as possible.

00:35:54 Udi Dahan

Just being able to say, "Okay. Here's a config that is generated by this tool, because it understands what your endpoints are, what the message flows are, you go from a whiteboard sketch of this is what we want the system to do to something working on your machine in just about five minutes." Just being able to get new developers up to speed.

00:36:15 Udi Dahan

For those of you who are in a team lead, or architect type positions where you have a new developer joining your team and saying, "Okay. We've got this NServiceBus thing that we're using, and it does pub/sub." "Oh, okay. What's pub/sub?" "Oh, it's like .NET event, it just distributed." "Cool. How do I make it work?" "Well, you don't. You just copy and paste this config, and sloth or something, offer up a prayer and maybe it'll work for you."

00:36:43 Udi Dahan

We want to put that behind and get to a model where you can say, "This is what I want to have run and just to have it run." Other than that, our main goal is to ... as much as possible, get out of your way. Okay. It's sometimes a fine line to walk of how much support do we want to give versus, let's call it the level of intrusiveness in your code. Beyond just showing you PowerPoint, and later on this, there's going to be more complete demos.

00:37:19 Udi Dahan

I want to take them in and actually show you this ServiceMatrix thing as it is today. For that, I have a fairly trivial solution set up. Give it a minute. The problem is almost always when doing this on a projector that I've got an entire 1024 pixels available to me, which is maybe half of what you'd use on a regular development rig. It's silly to really give somebody a sense of what is the capability of this tool?

00:38:04 Udi Dahan

Oh, wait, I need to auto hide it. Great. I've got 24 more pixels. In this environment, like I said before, we've got endpoints, and we've got events, and we've got commands, and we've got all that stuff. One of the things that we've ... and the team practically died on the fences trying to build this in time when we were launching the platform.

00:38:38 Udi Dahan

One of the things that I really pushed for, because I believe in its power very strongly is saga support in ServiceMatrix. Who's built in NServiceBus saga before? Okay. Good. Who found it easy the first time to build an NServiceBus saga? Okay. I'm seeing maybe 5% of the hands that went up of people use. It's one of those things that kind of like the NServiceBus routing. There is a certain set of magic incantations that you, like, "Okay. I need to handle the message." "Oh, right. The configure a mapping. Yeah."

00:39:16 Udi Dahan

Okay. Do the configure mapping thing. Then wait a minute. Again, this is a whole versioning problem. Does the saga automatically subscribe to the events or not? Because if it's a handler in the past, if you were using version three, a handler would automatically subscribe to events, if you had it there. But if all you had was a saga, the saga wouldn't automatically subscribe to events. You just change a class from a handler to a saga.

00:39:51 Udi Dahan

All of a sudden, the messages don't necessarily go across anymore, and you're looking at your saga and you're scratching your head saying, "What on earth?" That's one of the reasons in version four is that we made sagas automatically subscribe. You might be wondering, "Well, why did we do it in version three that sagas didn't automatically subscribe?"

00:40:07 Udi Dahan

Very long, complex architectural story that makes sense for a company that has been using NServiceBus for five or six years and has been doing service-oriented architecture and has all of the infrastructure and patterns and processes and methods. It makes sense for them. It turns out that there's maybe just a handful of companies that are really at that level.

00:40:31 Udi Dahan

For those people, well, if they want, they can actually not subscribe to events. One line of extra code for them, as opposed to confusion for 99% of all of the other users. In retrospect, it's a stupid, stupid, stupid type of moment. But that's the way that it is. One of the things that we've had in here is that you have this thing over here. You see that? Convert to Saga. Really, that's where it starts it. I've got a handler. Let's make this thing into a saga. Done. It is now a saga.

00:41:12 Udi Dahan

All of the infrastructure, all the crap is behind the things. One of the things that says right away is "Configure how to find the saga." Why? Because most people they don't know to do that. I said, "Okay. What's the number one thing people get mixed up on?" They forget that thing. All right. There you go. All of the rest of the plumbing is there. Now you just need to describe that element of message to saga and off you go.

00:41:42 Udi Dahan

There's so much more that we're looking to do in this canvas environment. We're constantly looking at the sitting down and watching people using our tools and saying, "Where are people struggling?" There's nothing about this that forces you to use it. I mean, if you want to manually go into the class and write all of your regular NServiceBus code, that is totally entirely up to you. We're not going to get in the way of that.

00:42:08 Udi Dahan

But what we are trying to do is to take those things that we've just seen people trip over time and time again, say, "How can we make that smoother? How can we take the user's intent? Can't I just make this thing into a saga and then just shorten the number of steps so that it's as easy as possible to do that?" As I said before, step out of the way.

00:42:33 Udi Dahan

If you haven't played with ServiceMatrix yet, please, please go to our website, Particular .NET, click the Download Now button. You're going to get the installer together with ServiceMatrix, together with ServiceControl, all that kind of stuff. The other thing that we've done with ServiceMatrix, is that when you run a ServiceMatrix solution, and hopefully it won't blow up on me now, because I made a change, is that automatically ServiceInsight, that debugging tool will pop up. Okay.

00:43:06 Udi Dahan

Now, again, try to imagine this not on a single 1024 by 768 environment. Okay. But rather, as you are debugging your system, this ServiceInsight view that you're looking at is on your second monitor. Okay. When messages are flowing through your system, and let's have some messages flow through the system. That ServiceInsight on your second monitor, hooked into the audit log of whatever is currently being debugged.

00:43:43 Udi Dahan

Said, "This is what just happened in the backend of your system." You've got all of the nice little consoles open up in front of you with all of the little log outputs. These things over here, it's saying, "Oh, yes. Sales received, submit order, and billing received, order accepted." All of the little green screens that ... I'm sorry. I have a thing for green screens.

00:44:08 Udi Dahan

But that on your second monitor, you have something that's sitting there fully in sync with your current debug session that when you put a breakpoint in Visual Studio, that on your second monitor, you have something saying, "Well, this is where we are in the message flow." It tells you, "Okay. This came from the frontend, endpoint to the backend. This one from backend number one to backend number two."

00:44:29 Udi Dahan

Of course, if you want to peer a little bit more deeply into the message, you can see the body of the message in terms of the raw XML. You have performance information about at the time that it was sent, the processing time, all of that information collected and available for you automatically. Of course, fully in sync with your current debug session. Every time that you debug, in essence, what it's doing is it's hiding the history and keeping in sync with what you're currently seeing.

00:44:56 Udi Dahan

I got to tell you, it's a whole other level of building and debugging message based systems when you have all of these things set up and hooked up together, where we've been pushing really hard to get all this to work. We really want more feedback from the community to say, "How can we make this even better? Which edges can we polish off a little bit more?" Really think that you're going to have lots of fun using it.

00:45:24 Udi Dahan

With that, let's go back to PowerPoint. What my team actually trusts me with. In terms of documentation, who when they got started with NServiceBus felt like they had enough documentation? Not even one hand. That's one of the things I said, "It's a work in process." Recently, we've released our new doc site where it's backed up by GitHub.

00:46:07 Udi Dahan

All of the documentation that's on there is on GitHub as well. Then you can ... If there's something in there that doesn't make sense, then open up an issue on that page, say, "Doesn't make sense." If there's a typo, I mean, yes, you can send us an email, there's a typo. Or just go in there and edit the page and typos gone, submit a pull request. Okay.

00:46:33 Udi Dahan

We're constantly writing more documentation. There's a whole lot more for us to do. It's the bane of every open-source project. We're still paying down some of that history. But we are trying to do more in that regard. There's a book out on NServiceBus written by one of our champs, David Boike. If you haven't ... Who's read the book? Let's actually ... Oh, Cool. Bunch of hands have gone up.

00:46:59 Udi Dahan

If you haven't read the book yet, and you want to get access to it, on our website, we've got three chapters free. Because you were so great, and actually signed up for the conference, then just send us an email if you haven't gotten the book and we'll send it to you. Ultimately, we want you guys to have this documentation. If you don't got it, contact us. There's the big Contact Us button on the website. Hopefully you can't miss it.

00:47:24 Udi Dahan

Get in touch with us. Tell us that you want the book. We'll get you a copy of the eBook. Again, if there's more that we can do in terms of documentation, talk to us, let us know, we're doing as much as we can in that area. What's the future hold for us here at NServiceBus? The problem is that every time that we've made promises in the past, we haven't been very timely in keeping them.

00:47:57 Udi Dahan

It's one of those things that our best intentions around doing something ultimately smash up against the reality of bug fixes, and critical patches and all the millions of other things that we have to do. The things I'm going to talk about are not distant future type stuff, but rather more immediate type future. Promises that I can make that I'm have a very high degree of certainty that we're going to be able to keep for you.

00:48:23 Udi Dahan

The first thing is NServiceBus version 5.0, which is going to be coming out very soon, with the whole Distributed Transaction Coordinator thing. For those of you who have been in the NServiceBus environment, they've known that in the past, we've said, "We are going to get rid of our reliance on the Distributed Transaction Coordinator." Then we said that again in version three, and then we said that again with version four.

00:48:52 Udi Dahan

This time we mean it. We are moving away from the Distributed Transaction Coordinator. As you probably know, the devils in the details, but we've come up with a solution that is extremely solid. When I say extremely solid, I mean, we've tested it a million different ways. It is holding up. It is reliable. There are some ... I'm not sure if I call it snags.

00:49:24 Udi Dahan

But in order to provide the same level of deduplication both on the way in and on the way out of your endpoints. It's something that we wanted to be careful in how we built it not because can we make it work for a single system? But as I said earlier, one of our big concerns is how will, say, an NServiceBus five system interact and run alongside and NServiceBus four, NServiceBus three system?

00:49:58 Udi Dahan

We can make sure that an NServiceBus five system will deduplicate and everything will be correct. If the way that it stands now, you take an MSMQ based system on say NServiceBus 338 or 442 or something like that, and you hook it up with an NServiceBus 5.0 no DTC endpoint, while this one will guarantee that it does deduplication. The way that we have to do deduplication without the DTC results in duplicate messages arriving rarely on the DTC enabled old endpoints. Okay.

00:50:35 Udi Dahan

It's a tricky situation for us to stand up at a conference and say, "Yes. It's ready. Go use it everywhere." I mean, it's fairly ... Yeah. Like I said, it's extremely solid. More importantly, it's much faster than the regular DTC implementation. We're talking about 200%, 300% faster in terms of messaging throughput than an equivalent DTC endpoint. There are lots of things to like in our implementation.

00:51:06 Udi Dahan

But in terms of integrating a new type of no DTC endpoint with an existing DTC endpoint, there are still some snags over there, which is why for now, in terms of defaults, what we're doing is that we're only turning this on by default for RabbitMQ. Okay. Who's on RabbitMQ? Okay. We got maybe two hands out of the crowd. Okay. One of the reasons why we're doing this for RabbitMQ is because RabbitMQ never supported DTC to begin with.

00:51:43 Udi Dahan

Anybody who is running on RabbitMQ needed to deal with duplicates anyway. If you are on RabbitMQ in the past, you already had to write business logic that dealt with duplicates. In which case, turning it on by default over here means that it's going to decrease effectively the number of duplicates that hit your business logic. It's going to improve performance of your code, and a whole bunch of other things. But it's not going to cause any legacy problems for you.

00:52:11 Udi Dahan

Those of you that are on MSMQ, for example, we're not turning it on by default. You can turn it on yourselves. That's fine. But be aware that if you are hooking this new endpoint in with an existing endpoint, that that existing endpoint may start getting some duplicate messages. Okay. Going forwards we're going to provide you with APIs that allow you to fine tune that behavior and control it a little bit more and get notifications about this in ServicePlus and see the status of the endpoints in ServiceInsight.

00:52:41 Udi Dahan

We're going to be providing cross platform support so that when you are starting to use this, everything will be a whole lot smoother. Sometimes people ask, like, "Why do we even need this? I mean, we've got the DTC set up, and it's not really a problem?" Try to remember back, if you will, to introducing NServiceBus to your organization, and going to production or going to staging for the first time, and then your endpoint's crashing.

00:53:10 Udi Dahan

Then the administrator saying, "Well, what's the problem?" Then you run this DTC ping tool. Who's heard of DTC ping? Okay. A lot of you have. To find out that your message processing server cannot DTC ping to your database server, and then saying to your DBA, "Yes. I need you to open up ports 5,000 to 6,000 on the database server in order for my endpoint to be able to connect to the database." Your DBA says, "I'm sorry. What? No. A thousand ports? You must be out of your mind."

00:53:48 Udi Dahan

The security guys backing up the SQL Server guys, it's not easy to push through this type of resistance in an organization. A lot of times, you guys have already pushed through it, and you're like, "Well, okay. That's spilled milk. From now on, we don't care." But in looking at easing the work for future adoption of NServiceBus just removing that hurdle of going into production is going to be a big help.

00:54:19 Udi Dahan

We've always wanted to be production ready by default. Having this element that we're not in conflict with your corporate governance, security guidance, principle checklist thing, that's probably going to make life easier on you. Of course, you might be saying, "Well, in your company now everything's set up. But if you go to another company and you want to bring NServiceBus along with you, it's going to simplify things over there as well."

00:54:45 Udi Dahan

The no DTC thing is a big thing. We've been promising it. It is now at that level of production readiness. We're, like I said, on by default for RabbitMQ. Those of you that are on MSMQ and, say, building a brand new system that can be NServiceBus five end-to-end, absolutely use this. Okay. But I want you to be aware of that element of if you have a subscriber from an old endpoint or something like that, want to be totally upfront and clear, and there's going to be lots of guidance on this to make sure that you don't accidentally end up with problems in this area.

00:55:23 Udi Dahan

Second thing, coming very soon, ServiceInsight sequence diagrams. While we think it's nice to have the message flow diagram, being able to see physical endpoints and saying that message is going from here to there, and then that message goes back. Then we've got a timeout event for a saga and off of that the saga publishes this event over here and getting all of the traditional information that we can see, performance and processing time and delivery time, et cetera, just being able to get some more meaningful documentation and visualization.

00:55:59 Udi Dahan

Ultimately, it's the same data that ServiceInsight is currently using just set up to look a little bit nicer. We found, again, in our original work, it just makes a world of difference and being able to track down problems of being able to see what's going in and out of every endpoint. We think it's going to make life quite a bit easier on you.

00:56:26 Udi Dahan

Again, if you haven't been playing with ServiceInsight, please do. The other thing about ServiceInsight with regards to sagas, if you haven't played with it, we have some pretty awesome saga visualizations in there. If you are using saga, ServiceInsight is going to just be absolutely amazing for you, even as it is today. Go download it. Start playing with it. You're going to have lots of fun.

00:56:47 Udi Dahan

In general, looking forwards, everything is going to be not harder, better, faster, stronger, but easier, better, faster, stronger. We've broaden things out. We've got all of these new tools. I hope I'm giving you a sense of where we're going with them. A lot of it is at this point in time incremental, it's adding this and polishing that bit and removing friction over here and adding that little feature over there. There's a million things for us to do.

00:57:18 Udi Dahan

The best way to move that forwards is to get feedback from you guys. Please do use them. Give us your feedback. Ultimately, that's what we're here for. We love hearing your feedback, and making the product even better. Just wrapping up and giving you a sense of what's coming up today. There are, of course, a whole bunch of speakers at this conference coming from NServiceBus itself.

00:57:43 Udi Dahan

But in addition to the good folks at particular software, we have Greg Young that's going to be speaking with Szymon over here about how they have taken the event store and in essence, ripped out all of the infrastructure of NServiceBus. All of the RavenDB, SQL Server, MSMQ, all that stuff, and put everything top to bottom on Event Store. Greg's going to be talking about that story. Szymon is going to be demoing a lot of the amazing work that he's done in getting that to work.

00:58:20 Udi Dahan

It is very impressive, and it's very exciting. You're going to be seeing that tomorrow. We have James Lewis, from ThoughtWorks speaking about microservices. If you haven't heard about microservices now is going to be a great time to get an idea about it. NServiceBus is a really great way to get started building these microservices. It's going to give you some architectural insights together with the technological stuff that you already know about.

00:58:46 Udi Dahan

We have Yves, the man in the Azure who has been bringing the cloud to NServiceBus for years, talking about how he's done that, what are the plans going forwards? What things you need to keep in mind when you are targeting Azure? Is anybody using Azure today with NServiceBus? Okay. Bunch of hands going up. Anybody thinking about going to Azure with NServicesBus? Whoo, look at that, even more hands going up. Very nice, very nice.

00:59:18 Udi Dahan

Yves is the man, hound him during the breaks. Get as much information out of him as possible, just an absolute incredible source for that. We have Charlie Baker from our good friends at Wonga, the guy that raised his hand up in the back with still version 1.9 in production. Wonga one of the first companies here in the UK that I'm aware of that adopted NServiceBus and is still running with it today. Quite a bit larger than they were before, talking about the story of Wonga.

00:59:52 Udi Dahan

But how they rolled NServiceBus and the challenges that they faced, clearly more challenges with version 1.9 and version 3. We have Roy Cornelissen and Mark Taling from our good sponsors at Info Support. Thank you very much Info Support for sponsoring this conference along with us. They're going to be talking about their journey. Roy as well has been using NServiceBus for many years and has been a very active champion with his clients and helping in the community.

01:00:20 Udi Dahan

Mark had a really great blog post about putting the NServiceBus story together with multi-tenant software as a service systems. Strongly recommend reading the stuff that they have got. Anybody doing multi-tenant software as a service stuff out there? Okay. A bunch of hands clustered in the back, some over here. Lots of interesting extensibility stories.

01:00:41 Udi Dahan

We have Jan Ove from Norway, WebStep, who will be talking about their story about introducing NServiceBus to a client and the whole service-oriented architecture philosophy. It's one of the things that we want you to hear about is not just the technology itself, but the challenges people faced, getting it into an organization, how they overcame developer objections to this new, loosely coupled event driven model, how they did monitoring, all that stuff.

01:01:10 Udi Dahan

We have Dylan Beattie from Spotlight who will be helping tell the story about how NServiceBuses helping starving actors and actresses around the world get a job. Yes. There is a positive spin on this whole commercial side. Szymon as I mentioned before, with the Event Store. We have a really great lineup. I think you're going to enjoy hearing from them a lot.

01:01:33 Udi Dahan

We've got plenty of breaks throughout the day. If you do have questions, please do not feel free to ... please feel free and do not hesitate. Feel free to hound the speakers. That's what we're here for to make sure that your questions get answered. Just to round this off, we're relieving you with a little gift.

01:01:57 Udi Dahan

One of the things that you received on your way in is this little disk on key thing with a Particular Software thing on it. You might not have given it much thought because you always go to conferences, and they give you a discount key thing. What we've done with this discount key is we've preloaded two days of my advanced distributed systems design course on it.

01:02:17 Udi Dahan

That covers the whole story about the fallacies of distributed systems, coupling in distributed system, bus and broker patterns, messaging patterns, Introduction to service-oriented architecture. It's about 15 hours, because this is just not enough for you. You just want more Udi speaking at you all the time.

01:02:36 Udi Dahan

Fifteen hours of training videos from ADSD on this. In addition to all the great material that you got just from sitting here that you'll be able to take a lot of ideas from SOA and start applying them. Just to give you a sense, the regular price for the two days videos is $1,000. We really hope that you're going to enjoy and feel free. Pass it around your organization. The idea here is for you to share the knowledge as broadly as possible. I know that it's just going to come out.

01:03:12 Udi Dahan

When do we get the rest of the three days? Okay. Can't give away everything right on the first day. Give us some time. With that, I want to thank you all for coming to our conference and being longtime NServiceBus users. Yeah. Have a great conference. Thank you so much. Thank you.

NServiceBus past, present, and future

🔗Transcription