AWS Enhancements

Check out the latest enhancements to NServiceBus support for AWS. All of these updates aim to give you more control, fewer surprises, and a smoother experience when building distributed message-based systems on AWS.

For SQS, we’ve added automatic message visibility renewal to prevent premature redelivery of long-running messages. For saga persistence with DynamoDB, we’ve enabled optional eventual consistency to help reduce read costs. Plus, with native support for AWS Lambda Annotations, it’s now easier to build serverless message handlers with less boilerplate and better tooling integration. These are just some of the improvements.

Let’s get into the details.

🔗Message visibility renewal

What we did: NServiceBus now automatically prevents duplicate message processing in Amazon Simple Queue Service (SQS) by extending message visibility timeouts during long-running operations. This update eliminates a common source of race conditions and data corruption in distributed systems.

Why we did it: In SQS, each received message becomes invisible for a limited time ¹. If processing takes longer, the visibility timeout can expire, and SQS may redeliver the same message to another consumer, even while the first attempt is still in progress. The extra message delivery leads to multiple issues:

• Non-idempotent handlers can create business or data errors
• The original consumer can’t complete because its receipt becomes invalid
• Message delivery metrics become inaccurate
• A cycle of repeated failures can develop

To solve this, we introduced automatic message visibility renewal. During processing, NServiceBus extends the visibility timeout in 5-minute increments, with configuration options available to customize the renewal duration:

transport.MaxAutoMessageVisibilityRenewalDuration = TimeSpan.FromMinutes(10);

🔗Reserving payload space

What we did: You can now reserve space for third-party tracing headers (like DataDog) to prevent Amazon Simple Queue Service (SQS) message size calculation failures. This new configuration option eliminates intermittent send failures that were previously difficult to diagnose and resolve.

Why we did it: Because SQS messages have a 256 KB limit, NServiceBus has a feature that will offload large messages to S3 to avoid hitting the limit. However, third-party tracing tools like DataDog inject headers just before message dispatch, after NServiceBus has already checked the 256 KB SQS limit. As a result, messages could pass validation but fail during send when the extra headers pushed them over the limit.

To address this, we introduced a configuration option that lets you reserve space for these headers:

transport.ReserveBytesInMessageSizeCalculation = 512; // Size in bytes

This setting allows you to proactively reserve space for third-party headers, such as those commonly used by tracing solutions like DataDog, or diagnostic headers from OpenTelemetry, ensuring the actual message size remains safely below the threshold. You can now tailor your SQS payload calculations to your environment’s specific needs.

The reserved space decreases the maximum allowable message size. That means the transport will trigger the S3 fallback for smaller message sizes, providing a reliable and controlled safety margin for tools that inject headers during send operations.

🔗Support for alternate storage solutions

What we did: You can now turn off S3 payload signing in the Amazon Simple Queue Service (SQS) transport, making it possible to use alternative storage providers like Cloudflare R2 that offer S3-compatible APIs but don’t support signed payloads. This option enables lower-cost providers without being locked into AWS storage.

Why we did it: Until now, large message bodies in SQS were always offloaded to Amazon S3, which requires payload signing. That created a problem for services like Cloudflare R2, which does not support the Streaming SigV4 signing implementation used by the S3 SDK. Because signing was hardcoded, R2 wasn’t usable with the transport.

We’ve introduced a new configuration option:

transport.DisablePayloadSigning = true;

When set, the transport skips the signing step, allowing use of S3-compatible storage providers that don’t support signed payloads. For example, Cloudflare R2 provides zero egress costs, which can significantly reduce expenses for systems handling a high volume of large messages.

If you choose to turn off payload signing, weigh the trade-offs carefully: it unlocks alternative backends but reduces the security guarantees of signed requests.

🔗Promoting SQS message attributes

What we did: SQS message attributes are now automatically promoted to NServiceBus headers, giving direct access to external system metadata without extra integration code.

Why we did it: Previously, these attributes were only available by accessing the native message on the extension context, which limited their usefulness for scenarios like routing, monitoring, or enrichment.

With automatic promotion of SQS message attributes, any custom or third-party attributes on an incoming SQS message are now available as headers throughout the NServiceBus pipeline. This header promotion allows you to inspect and use them from NServiceBus pipeline behaviors ² to enable all sorts of cross-cutting infrastructure concerns.

🔗Improved poison message handling

What we did: The SQS transport now tracks message receive counts (the number of processing attempts before sending a message to the error queue) more accurately and handles poison messages more reliably, reducing duplicate processing, false retries, and wasted compute in production systems.

Why we did it: Previously, message receive counts were based only on an in-memory cache, which had drawbacks. In-memory counts can’t be shared across endpoint instances, so in a scaled-out environment with many nodes, it would take many processing attempts for any one node to observe the message enough times to move a message to the error queue. Additionally, in-memory counts were lost during endpoint restarts, resulting in the loss of historical context.

However, AWS’s ApproximateReceiveCount is not accurate either and has been observed under-reporting actual counts.

The transport now combines both values to determine the actual receive count:

ActualReceiveCount = Max(LocalCacheValue, ApproximateReceiveCount)

This hybrid approach improves accuracy by:

• Buffering against AWS under-reporting
• Recovering from cache loss on restarts
• Auto-correcting in competing consumer scenarios
• Remaining compatible with the existing cache mechanism

Additionally, this improved approach causes poison messages (those that consistently fail) to move to the error queue without invoking business logic. This optimization prevents unnecessary reprocessing, duplicate deliveries, and wasted resources.

Together, these improvements make retry behavior more predictable, simplify operational diagnostics, and facilitate a smoother transition from other transports, such as SQL Server.

🔗AWS Lambda Annotations

What we did: NServiceBus now supports the AWS Lambda Annotations model for .NET, making it easier to build SQS-triggered Lambda functions with less boilerplate and better integration with modern AWS tooling.

Why we did it: Lambda Annotations use C# source generators to reduce repetitive glue code and automatically synchronize CloudFormation templates with annotated methods. With this update, you can register the NServiceBus AWS Lambda integration directly into the .NET service collection provided by the annotation model.

Here’s what a fully functional Lambda handler now looks like:

public class SqsLambda(IAwsLambdaSQSEndpoint serverlessEndpoint)
{
    [LambdaFunction]
    [SQSEvent("ServerlessEndpoint")]
    public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
    {
        using var cancellationTokenSource =
            new CancellationTokenSource(context.RemainingTime.Subtract(DefaultRemainingTimeGracePeriod));

        await serverlessEndpoint.Process(evnt, context, cancellationTokenSource.Token);
    }

    static readonly TimeSpan DefaultRemainingTimeGracePeriod = TimeSpan.FromSeconds(10);
}

This integration unlocks a more modern and developer-friendly way to build serverless applications using NServiceBus:

• Eliminates boilerplate code needed to wire up Lambda functions manually
• Leverages compile-time source generation for CloudFormation synchronization
• Reduces cognitive load and setup time for teams adopting AWS Lambda and NServiceBus together

The NServiceBus documentation for AWS Lambda and SQS contains details on the new features, including a ready-to-use sample. If you’re building .NET applications on AWS Lambda and want first-class integration with modern tooling, this update helps you get there with less code.

🔗DynamoDB custom JSON serialization

What we did: To enable advanced scenarios like DynamoDB-specific attribute handling and schema-aware transformations, we introduced support for injecting custom JsonSerializerOptions, JsonTypeInfo, and context resolvers into the saga persistence and mapping APIs.

Why we did it: Previously, the mapping pipeline relied on internal, preconfigured JSON options. These defaults worked for simple objects, but made it difficult (or impossible) to:

• Respect custom serialization attributes like DynamoDBProperty or DynamoDBIgnore
• Inject custom converters or behaviors
• Support schema-specific transformations required by DynamoDB or other systems

With this update, you can override the default options at both the persistence and mapper levels. For example, to support DynamoDB-specific attributes:

readonly JsonSerializerOptions serializerOptions = new(Mapper.Default)
{
    TypeInfoResolver = new DefaultJsonTypeInfoResolver
    {
        Modifiers = { SupportObjectModelAttributes }
    }
};

The modifier inspects and manipulates JsonTypeInfo at runtime to recognize and apply DynamoDB semantics:

public static void SupportObjectModelAttributes(JsonTypeInfo typeInfo)
{
    if (typeInfo.Kind != JsonTypeInfoKind.Object)
    {
        return;
    }

    foreach (JsonPropertyInfo property in typeInfo.Properties)
    {
        if (property.AttributeProvider?.GetCustomAttributes(typeof(DynamoDBRenamableAttribute), true)
            .SingleOrDefault() is DynamoDBRenamableAttribute renamable &&
            !string.IsNullOrEmpty(renamable.AttributeName))
        {
            property.Name = renamable.AttributeName;
        }
        else if (property.AttributeProvider?.GetCustomAttributes(typeof(DynamoDBIgnoreAttribute), true)
            .SingleOrDefault() is DynamoDBIgnoreAttribute)
        {
            property.ShouldSerialize = (_, __) => false;
        }
        else
        {
            property.ShouldSerialize = (_, __) => false;
        }
    }
}

Now you can serialize complex domain objects using existing DynamoDB conventions without redundantly decorating them with JsonPropertyName or JsonIgnore attributes. For example, you can now serialize a complex domain object like the following Customer type using DynamoDB conventions:

class Customer
{
    [DynamoDBHashKey("PK")]
    public string PartitionKey { get; set; }

    [DynamoDBRangeKey("SK")]
    public string SortKey { get; set; }

    public string CustomerId
    {
        get => PartitionKey;
        set
        {
            PartitionKey = value;
            SortKey = value;
        }
    }

    [DynamoDBProperty]
    public bool CustomerPreferred { get; set; }

    [DynamoDBIgnore]
    public string IgnoredProperty { get; set; }
}

Here’s the corresponding message handler, which operates in the same DynamoDB transaction due to the use of the synchronized storage session:

public async Task Handle(MakeCustomerPreferred message, IMessageHandlerContext context)
{
    var session = context.SynchronizedStorageSession.DynamoPersistenceSession();

    var customer = await dynamoContext.LoadAsync<Customer>(message.CustomerId, message.CustomerId, context.CancellationToken);

    customer.CustomerPreferred = true;
    customer.IgnoredProperty = "IgnoredProperty";

    // Thanks to the customized serializer options, the mapper understands the context attributes
    var customerMap = Mapper.ToMap(customer, serializerOptions);

    session.Add(new TransactWriteItem
    {
        Put = new()
        {
            Item = customerMap,
            ...
        }
    });
}

This change streamlines DynamoDB integration, providing fine-grained control with custom converters or source-generated type resolvers.

🔗Eventually consistent saga reads

What we did: You can now reduce DynamoDB read costs by opting into eventual consistent reads for saga data when using optimistic concurrency. This consistency option provides flexibility in high-throughput or cost-sensitive environments where occasional retries are acceptable.

Why we did it: By default, the NServiceBus DynamoDB saga persister has always used consistent reads to guarantee the most up-to-date saga data and prevent version mismatches or conflicting writes. While safer, this approach consumes twice the read capacity units compared to eventual consistency.

With the new global configuration option, you can enable eventual consistent reads:

var sagas = persistence.Sagas();
sagas.UseEventuallyConsistentReads = true;

When enabled, saga reads use DynamoDB’s eventual consistency model, lowering read costs while still supporting optimistic concurrency control.

🔗Summary

If you’ve run into any of the friction points above, we hope these changes make your life easier. If you spot other issues we could improve—or have ideas to make NServiceBus and the Particular Platform better—please open a GitHub issue in the relevant repository.