Transactions with Azure Table Persistence

One of the major changes that have permeated organizations across the globe in the last decade is the use of cloud providers and the variety of services they offer. For data storage alone, Azure offers: Table Storage, CosmosDB, SQL Server, Blob Storage, and more. There are, of course, many advantages to these cloud services, like minimizing infrastructure management, and higher performance at a reasonable cost and it’s easy to accept these advantages in the face of the trade-offs, like giving up transactionality to achieve data consistency.

With the latest release of Azure Table Persistence for NServiceBus, we offer full transactionality across Outbox, Synchronized Storage Session as well as Sagas. We are bringing you the full power of transactionality on top of a native cloud solution while bringing down the operational and data transfer costs, making operations much faster without compromising backward compatibility with all versions of the previous persistence version - no more trade-offs.

🔗Transactionality for peace of mind

How does transactionality translate into a business scenario? Let’s consider an example.

Every time a customer places an order, we want to keep track of all the order information and see if the customer is eligible for special discounts on future orders. That privilege is determined by the total order amount placed over the last six months. To achieve this, we’ll use a long-running saga to keep track of the customer’s accumulated order total, notifying additional stakeholders when the total reaches a certain level. (Incidentally, this is an ideal way to avoid batch jobs).

In a non-transactional scenario with Azure Storage Persistence, there are a few concerns to consider given that there are no transactional guarantees:

• The storage of your business data might succeed, while updating the running total in the saga fails
• Both the order information and the running total may be successfully updated, but a timeout could cause the message to be reprocessed
• Messages sent or published to stakeholders might not have been delivered or delivered multiple times during retries

This forces us to consider custom message deduplication and idempotency when storing information and processing messages.

With the new release of Azure Table Persistence for NServiceBus, it is now possible for transactions to span saga data, business data, and the outbox (if activated), which brings the full power of NServiceBus to Azure Table Storage and Azure Cosmos DB Table API. By default, the persistence does not attempt to commit saga data or business data atomically and instead uses the saga ID as a partition key. Using the TableBatchOperation API, saga data and business data can be committed in a single transaction if everything is stored in the same partition within the same table. No more worrying about consistency.

🔗One connection string to rule them all

The persister’s connection string can point to either Azure Table Storage or Azure Cosmos DB Table API. Both services will work out of the box without additional configuration.

endpointConfiguration.UsePersistence<AzureTablePersistence>()
   .UseConnectionstring("ConnectionString");

When a connection string is not enough to fulfill advanced authentication scenarios, you can provide a fully preconfigured CloudTableClient:

endpointConfiguration.UsePersistence<AzureTablePersistence>()
   .UseCloudTableClient(new CloudTableClient(...));

This behavior has been aligned across Azure Table Persistence, Azure Storage Queues and Azure Storage Blob Databus.

🔗Need for speed

Based on our internal testing, saga loading and updating is up to 170% faster than the previous version of the persister for complex saga data entities, and over 220% faster for simple saga data entities that contain only data types supported by Azure Table Storage without requiring additional serialization. In addition, newly stored sagas have been optimized to reduce the number of storage operations required to load sagas by their correlation properties. These changes improve the throughput of the NServiceBus endpoints using Azure Table Storage and reduce operational and data transfer costs, since fewer operations are executed against table storage.

For new endpoints, which do not require backward compatiblility with sagas stored with NServiceBus.Persistence.AzureStorage, you can squeeze out even more speed by disabling compatibility mode.

🔗TL;DR - Faster, easier, and cheaper

To recap, Azure Table Persistence is cheaper, has better performance, and is easier to configure than NServiceBus.Persistence.AzureStorage You get all the benefits that come from using cloud storage along with the benefits of transactions from a more traditional storage mechanism. What’s not to like?

You can find the NServiceBus.Persistence.AzureTableStorage package on NuGet. Be sure to check the documentation and samples for more information.