Skip to main content

NServiceBus on AWS Lambda

The NServiceBus.AwsLambda.Sqs preview package makes it easier than ever to host NServiceBus endpoints in Amazon Web Services (AWS). AWS Lambda is Amazon’s serverless technology for running your code in the cloud with zero administration. You can run code for just about anything and Amazon takes care of running and scaling your code with high availability. You only pay for the compute time you consume.

As foundations go, this is as strong as bedrock. However, to build a complex distributed system on serverless technology that implements best practices from the Enterprise Integration Patterns book, you need something more that will bridge the gap between the “invisible” server infrastructure and your business code.

That is where our new AWS Lambda SQS package comes in. NServiceBus provides the infrastructure code not provided by Lambda so that you can write business code and execute it in the Lambda environment. You can even use Lambda, with its cheap hosting model for rapid prototyping, and only later take advantage of the NServiceBus abstraction to easily shift hosting to container-based deployment or even to a competing cloud platform.

Let’s take a closer look at what our AWS Lambda preview makes possible.

Basic Lambda

Receiving a message from Amazon SQS and processing it is a fairly easy task. You have to create an incoming queue and bind a Lambda to messages from that queue.

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]
namespace CheckoutLambdaFunction
{
    public class CheckoutLambdaFunction
    {
        public string HandleSQSEvent(SQSEvent messageBatch, ILambdaContext context)
        {
            foreach (var record in messageBatch.Records)
            {
                var checkoutMessage = DeserializeMessage(record.Body);
                ProcessCheckout(checkoutMessage);
            }

            return $"Processed {sqsEvent.Records.Count} records.";
        }
    }
}

This is simple enough, but if you look at this code closely, you’ll notice many infrastructure tasks that you’ll have to do yourself:

  • You need to deserialize the message body yourself.
  • If there are multiple types of messages (or different versions of the messages) arriving in one queue, you need to understand how that affects deserialization as well as routing to the correct method to process each one.
  • If you want to send a message, you’re also on your own. You need to new up an SQS client or SNS client to send or publish a message, respectively.
  • SQS messages are delivered in a batch. What happens if one message in a batch fails to process and the others succeed? What happens if you send a message while processing the first message, and then a subsequent failure causes the entire batch to fail? The message you sent is already gone. This is something that NServiceBus handles with the Outbox feature.

These are problems NServiceBus has consistently solved well. Now it can help you with Lambda as well.

NServiceBus on Lambda

Hosting an NServiceBus endpoint within AWS Lambda is easy. The first thing to do is create a Lambda function as well as the SQS queues required (a queue for the incoming messages and a queue for any error messages) and bind the Lambda to be triggered by messages from the incoming queue. You can use a CloudFormation template to do this.

Now you can create your endpoint and configure it. AWS Lambda endpoints are created by instantiating a new instance of an AwsLambdaSQSEndpoint class and configuring it using a Func<AwsLambdaSQSEndpoint>.

static readonly AwsLambdaSQSEndpoint serverlessEndpoint = new AwsLambdaSQSEndpoint(context =>
{
    var endpointConfiguration = new AwsLambdaSQSEndpointConfiguration("NServiceBusAwsLambdaEndpoint");

    var advanced = endpointConfiguration.AdvancedConfiguration;
    advanced.SendFailedMessagesTo("error");

    return endpointConfiguration;
});

This static instance can now be used to process the SQSEvent that will be triggered as part of your AWS Lambda configuration.

public class NServiceBusLambdaFunction
{
    public async Task HandleSQSEvent(SQSEvent messageBatch, ILambdaContext context)
    {
        await serverlessEndpoint.Process(messageBatch, context);
    }
}

With the tiny bit of boilerplate above in place, you can now use ordinary NServiceBus message handlers to process messages coming in over SQS.

public async Task Handle(CheckoutMessage message, IMessageHandlerContext context)
{
    Console.WriteLine("Processing CheckoutMessage");
}

You can even send commands or publish events using the NServiceBus API. NServiceBus takes care of creating the necessary SQSClient or SNSClient for you. NServiceBus will also manage the cases where one message in a batch fails, making sure that only failed messages are retried and that outgoing messages are only dispatched if the individual message processing succeeds.

public async Task Handle(CheckoutMessage message, IMessageHandlerContext context)
{
    Console.WriteLine("Processing CheckoutMessage");

    await context.Send(new DoAnotherThingCommand());
    await context.Publish(new SomethingHappenedEvent());
}

By using NServiceBus with Lambda, you get publish/subscribe, recoverability, delayed delivery, and long-running workflows with NServiceBus sagas. You write the business logic, NServiceBus handles the plumbing, while Lambda provides the serverless execution environment.

Rapid prototyping

One of the biggest advantages to AWS Lambda is the ability to easily and cheaply deploy a proof of concept solution and then scale it up to production, especially when you’re unsure of what scale the solution will ultimately need to fulfill.

Eventually, solutions that start on Lambda may reach a level of maturity where the serverless hosting model is no longer preferable. The cost of Lambda, while flexible, may be higher than running the system on dedicated resources.

In this case, using NServiceBus as an abstraction layer over Lambda really pays off. The message transport (SQS) stays the same, and the message handlers (classes that implement NServiceBus’s IHandleMessages interface) all stay the same. Meanwhile, hosting can easily be shifted to containers running on Amazon Elastic Container Service or Elastic Kubernetes Service. Only the minimal boilerplate code that creates endpoints out of SQS queues is coupled to the Lambda stack.

Other benefits

A few other benefits of integrating NServiceBus with AWS Lambda include:

About our preview program

Support for AWS Lambda is being released as a result of our Preview Program.

But make no mistake: Preview does not mean substandard quality. Our goal is to be enterprise-grade, which means stability. All our previews are production-supported capabilities, offered with a more permissive license.

We get a lot of requests to support new technologies with the Particular Service Platform. As you can imagine, it is hard to keep up with them all. Deciding what to add and when to add it has been hard. We strive to be innovative while maintaining the high standards we hold ourselves to. This is why we have created the preview program.

Previews are new software that adds new ideas to our platform or expands the technologies that we integrate with. Since it is new, it comes with some risk. While we do our research, we are still not sure if our customers are willing to adopt the software we release. The Preview program is designed to be the final test of any innovation we have been exploring. By releasing a minimally featured version, with support in your production environments, we can validate that the solution we are bringing forward is the correct one and that it will be well received by our current and future customers.

The current Preview software we are offering and the status of each one can be found on our Platform previews page. Let us know if you are using any of our Preview software and keep an eye out here and on our Twitter feed as we launch additional Previews.

Summary

With support for AWS Lambda, you can bring the capabilities of the Particular Service Platform to a cost-efficient serverless environment running in AWS. NServiceBus enables simple message handling while taking care of serialization, routing, error handling, recoverability, and other complexities of messaging in distributed systems.

To get started with some code yourself, check out our Using NServiceBus in AWS Lambda with SQS sample or check out the Lambda documentation.

About the authors

William Brander

William Brander is a developer at Particular Software who knows the only real serverless environment is the one that runs in your browser.

David Boike

David Boike is a developer at Particular Software who doesn't care whose computer it is, but just doesn't want a hefty bill for 10 requests per day.

Don't miss a thing. Sign up today and we'll send you an email when new posts come out.
Thank you for subscribing. We'll be in touch soon.
 
We collect and use this information in accordance with our privacy policy.
Need help getting started?