Azure Storage Queues learns some new tricks

Azure Storage Queues is a basic yet robust queueing service available on the Azure platform. In contrast to other messaging services in Azure, it has very few features out of the box. For example, it lacks concepts like topics and subscriptions to build publisher and subscriber topologies—these are capabilities NServiceBus adds on top of Azure Storage Queues.

In our latest enhancement release, we’ve added native publish/subscribe to our Azure Storage Queues transport, creating feature parity between Azure Storage Queues and all the other transports provided by NServiceBus, and created a new sample to show you how to integrate with 3rd-party systems sending messages to NServiceBus using native SDKs.

Let’s take a look.

🔗Native publish/subscribe

We have added native publish/subscribe support directly into the Azure Storage Queues transport. This means you no longer have to configure a “subscription storage” to do pub/sub, and you’ll no longer need to use the routing.RegisterPublisher(…) method to tell NServiceBus where to send subscription messages.

Here’s how it works.

The native subscription model uses storage tables, which are already used by native delayed delivery. Because Azure Table storage is already part of the storage account that contains your queues, this will work without setting up anything else in Azure or providing any additional connection strings. However, if you want to use the Azure Cosmos DB Table API instead, we support that too.

In this new model, the subscriber will register its subscriptions directly into a shared storage table without sending a subscription message to the publisher. These subscriptions are partitioned by subscribed event type, ensuring optimal performance when the publisher retrieves them.

The publisher reads the subscribed endpoint queues from the shared table by retrieving the partitions matching any base type in the event type hierarchy. It then deduplicates the subscribed endpoint queues and sends the message to the resulting set of queues.

To make the transition as smooth as possible, a compatibility mode allows you to have both the new native publish and subscribe mechanism and the message-driven publish/subscribe mechanism running side-by-side until every endpoint is migrated.

🔗Native integration

Most systems need to integrate with other systems; these systems may even be created using different programming languages and runtimes. We now support consuming Azure Storage Queues messages that weren’t created by NServiceBus, or even in .NET, as long as there’s a way to specify what the message body content of incoming messages is.

While the transport has always been able to support this, it’s a little trickier on Azure Storage Queues than on most message brokers, so we built a new sample to show you how.

Most message brokers support message headers, which provide a convenient way to pass the message type information in. But not Azure Storage Queues. Originally it was designed as a task queue service, passing tasks between different Azure services, which didn’t need the sophistication messaging services require today. As a result, Azure Storage Queues only allows arbitrary body content.

To determine the message type from the body content, the transport exposes a method, UnwrapMessagesWith that provides access to the message body, which you could use to derive the native message type.

For example, if an external system were sending one type of message, you could set up an endpoint as a sort of adapter, and configure the UnwrapMessagesWith method to assume a single hardcoded message type:

transport.UnwrapMessagesWith(message => new MessageWrapper
{
    Id = message.MessageId,
    Body = message.Body.ToArray(),
    Headers = new Dictionary<string, string>
    {
        { Headers.EnclosedMessageTypes, typeof(SingleMessageType).FullName }
    }
});

In more complex scenarios, you could deserialize the native message and dynamically determine the message type as well.

Check out the new Native Integration with Azure Storage Queues Transport sample for more details.

🔗Summary

With the addition of native publish/subscribe, you no longer need external subscription storage or to call the RegisterPublisher() method when you’re using Azure Storage Queues. And with the ability to do native integration, you can send a message from an external system written in Java, Python, Node.js, PHP, Ruby, or even C++ and have it understood by your NServiceBus endpoints.

The new version of the Azure Storage Queues transport can be downloaded from NuGet. For more information, check out the detailed release notes for this version, the Azure Storage Queues transport documentation, or the documentation article about native publish/subscribe.