Azure Storage Persistence now faster in NServiceBus 6
If you're using Azure Storage Persistence and haven't upgraded to NServiceBus 6 yet, get ready for a tremendous performance boost for your application when you do---especially if you make use of sagas. In the previous version of Azure Storage Persistence, looking up a saga by a correlation property was not as fast as looking it up by
SagaId. In the new version, both the correlation property and
SagaId are indexed, so retrieving a saga is much quicker regardless of whether it's looked up by
SagaId or the correlation property.
If you're the TL;DR type, you can stop here and head over to the NServiceBus 6 page to try it out. But if you're interested in how we did it, read on!
Azure Table Storage, where saga data is stored, is limited to indexing on two columns: the Partition Key and the Row Key. To boost the performance of saga retrieval, we now create two rows in the table for each saga entry rather than one. One row is indexed on the
SagaId and contains all of the saga data. The second row is indexed on the correlation property and contains only the
SagaId. To retrieve saga data using the correlation property, we first look up the row based on the correlation property, which gives us the
SagaId. Then we look up the saga data directly, based on the retrieved
Here's how it might look in practice:
In this example, the first two rows represent a saga with a SagaId of
9bc38..., and the correlation property is the telephone number. To look up the saga based on a specific phone number, we search for the row with
Partition Key = 555.1234, which gives us a SagaId of
9bc38.... Now we can look up the full saga data using this ID and querying where
Partition Key = 9bc38.... Both lookups are very fast since they are retrieved using the natively indexed Partition Key values.
This secondary index pattern provides great performance for our saga persistence. The amount of improvement depends on how much data your persister is managing at one time, but suffice it to say that the improvement was so good that we implemented it for Timeout lookups too.
So if you want to experience these performance boosts in your own system, come download NServiceBus 6.