Getting NServiceBus ready for the cloud

More and more companies are realizing that it’s in their best interest to move to the cloud. As software consultants at Headspring, we’re often tasked with helping organizations migrate their highly complex legacy systems to modern cloud environments. For clients using NServiceBus to facilitate messaging, moving to the cloud without it seems untenable. So how do we prepare to take NServiceBus with us?

For complex software systems like the ones we’re used to working with, moving to the cloud often involves more than a simple “lift and shift.” NServiceBus proves to be just as valuable in a cloud environment, but various aspects of its configuration and integration into the system need to be modified in order to take full advantage.

One of our recent clients—a government agency—was already using NServiceBus and wanted to migrate their system to Azure. The steps we took to get NServiceBus ready to run in Azure may be helpful to you as you map your own cloud moves. I’ll walk you through exactly what we had to do differently in the cloud, explain the hybrid-cloud solution we built with NServiceBus for reporting, and demonstrate how we handled logging in the cloud, as well as our client’s need to run locally on occasion.

🔗Making the move to cloud computing

We were looking to move a software system that had been running successfully on premises for a decade. The system uses NServiceBus to facilitate heavy auditing required by government regulations, without slowing down the site, and has run without a hitch since deployment.

In order to get up to date with changing regulatory requirements in their space, embrace new efficiencies, and set themselves up for the future, the client decided to move to the cloud. They went with Azure because the Azure Government Cloud environment is compliant with new hosting regulations that they needed to meet. The client’s existing servers had, over time, become an elaborate house of cards that people were somewhat afraid to touch. They loved the idea of a simple, configured app service instance that could be spun up or down, grown or shrunk, or even destroyed as needed without losing anything valuable.

🔗A focused approach to getting cloud-ready

This was a legacy system, and there was a lot that had remained unchanged for far too long. In deciding the level of change we wanted to introduce as part of our cloud migration, we had to decide what needed to change in order to get them to the cloud and leave the nice-to-haves to future efforts.

We were on a limited timeline and budget, and the client was looking for the final switchover to be smooth. To keep the surface area of change minimal, we decided to make only the modifications necessary to support the migration—no unrelated library updates that could introduce behavior changes, or unnecessary UI improvements. We’ve found these projects tend to go better if you touch one thing at a time, and the first thing to do was support the cloud migration.

Even with this policy in place, as we’ll see, there would still be a few areas to address to get it ready to run in the cloud.

🔗Cloud-ready message transport

The first aspect we needed to upgrade was the old application’s use of MSMQ. MSMQ is not a cloud-friendly technology—it only provides local queues. Any queue traffic between applications on different machines would be between MSMQ instances, using MSMQ’s specific communication. There is no single queue server, just queues that live on each application server—and that violates the idea of instances being able to be spun up or down with nothing lost. Plus, MSMQ is pretty much dead anyway.

The natural replacement for MSMQ in Azure is Azure Service Bus. Unlike MSMQ, Azure Service Bus provides a single, centralized broker that is separate from any individual instance of the application. The next step was to wire in this new message broker, and the first thing we’d have to do was upgrade our existing version of NServiceBus.

🔗Getting versions up to speed

NServiceBus has made a lot of great improvements over the last few years—not only in supporting Azure, but also in embracing async/await, configuration improvements, supporting .NET Core, and many others. However, we had some changes to make in the system to support the latest version of NServiceBus. The application was also on a very old version of .NET, so, driven by our guiding principle of only making changes that would support the cloud, we bumped the application up to .NET 4.6.2 (the minimum version we could get away with) and snapped in a modern version of NServiceBus.

The specific updates we had to make as part of our NServiceBus upgrade were related to the bus metaphor and the configuration. Since NServiceBus was added to the system, the product has moved away from the catch-all IBus to more granular, intention-revealing interfaces. Swapping in these new interfaces meant looking at how we actually used NServiceBus in each location and matching it to one of the new interfaces. That took work, but resulted in message handlers that better reflected their actual use. The configuration of NServiceBus changed as well: We moved from the older, specialized XML-Based configuration to the modern code-based configuration that is common to newer tools.

🔗Creating a hybrid cloud solution for reports

Of course, unexpected things always come up in big projects like this, and in this case, our surprise “gotcha” was SQL Server Reporting Services (SSRS) reports.

The Azure environment we were in didn’t have a great story for migrating the existing SSRS reports over—other than upgrading them all to Power BI. We just didn’t have time to tackle that kind of upgrade, so we ended up tying together our all-cloud infrastructure with a client-owned server to run the actual reports.

We used NServiceBus from our cloud environment to send a message to a small application on the client-owned server, had that server generate an SSRS report for us, and then send the report back to our main cloud-hosted application. Since this was a government-specific system, outbound traffic was quite locked down—we were unable to reach anything hosted on premises. However, we were able to access the cloud transport (the queue in Azure) from anywhere, including from this on-premises server. This hybrid cloud solution quickly got us past what could have been a huge blocker for a critical part of the project.

🔗Centralized logging

Like message queues, application logging requires special consideration in the cloud. NServiceBus comes with logging out of the box, so it logs quickly and easily to the filesystem without any configuration. This is fantastic when you’re getting started, and it works fine on a traditional server.

As we saw with MSMQ, the cloud is fundamentally different, so appropriate configuration is required. In the cloud, the filesystem is part of the whole system that can be spun up, spun down, or thrown away. We want to keep our log somewhere different in order to be sure to preserve them even if a particular instance of the site were to disappear.

Luckily, NServiceBus has no problem working in this environment, and there are several options for implementation. The open source tool, Serilog, for example, includes the concept of “sinks,” which are destinations for logs. We added a sink for Azure Monitor, which provides modern, centralized, searchable structured logs in Azure. Since NServiceBus has an adapter to log into Serilog, we were good to go to log in the cloud. For those on current versions of NServiceBus, you can now take advantage of the NServiceBus.Extensions.Logging package, which lets you use just about anything you want. If we had been able to more fully modernize the code, we would have likely used this  to avoid any direct ties into a particular logging framework.

🔗Running locally

NServiceBus is critical to our application, but it is only one piece of the whole. Sometimes, it is useful to have the option of running the application locally and disconnected while developing functions unrelated to messaging, such as when working on business logic or UI.

Certain scenarios make a local option attractive: When traveling, connecting via a restrictive VPN, or working from a location with an unreliable internet connection such as hotels—or even home when there is a lot of neighborhood internet traffic. In our client’s case, we added an alternate configuration to run the Learning Transport provided by Particular, and we can switch between these modes using application configuration.

While using a local transport can come in handy, I would not recommend it as a way to test or validate your NServiceBus system. Different transports have very different properties and failure modes, so confirming that a system works with one transport says almost nothing about how that system will perform when configured with a different transport. Removing blockers to working on other parts of the system is the only reason I’d use this kind of alternate configuration—in this case, testing your system in a staging environment equivalent to production before launch is even more critical.

If you plan to develop with a consistent internet connection, you should consider a simpler development configuration that does without any switching. Instead, try using developer-specific namespaces in Azure Service Bus to develop locally—you will get to see how your code will do in Azure while you are developing, with no extra effort.

🔗So, how’d we do?

We launched the updated system in December, and it’s been running successfully ever since. The system has been running on its more updated version of .NET, Azure Service Bus has been receiving and distributing messages, Azure Monitor has been receiving NServiceBus logging messages, and we’ve even run locally on occasion as well. We’re a long way away from fully modernizing the entire system, but the move to Azure has set us up for success.

This example scenario isn’t a comprehensive guide for moving your own software system to the cloud with NServiceBus. However, it does prove that it is possible, doable, and not too hard after all! I hope this story, at minimum, helps you navigate some of the challenges you may run into if you decide to make a similar move. In the end, you’ll end up with all the benefits of NServiceBus working for you in the cloud—which these days, is the place to be.