Redesigning the NServiceBus pipeline yet again: The nerd sniping of Daniel Marbach

Daniel and I ¹ have worked on improvements to the NServiceBus pipeline multiple times before: a 10x speed-up in 2017, and another 5x improvement in 2022. We thought we were all done with that.

So, quite unexpectedly, this is the story of how I accidentally nerd sniped Daniel into redesigning the NServiceBus pipeline yet again.

One little comment from me, and Daniel made the pipeline start 330x faster and run 25-29% faster on the success path, while still producing zero allocations along that path.

Do I feel a little bad about it? Yes. Will I get over it? Also probably yes.

🔗Setting the trap

During one team sync, we were discussing all of the changes that would be required to make NServiceBus fully support assembly trimming and ahead-of-time (AOT) compilation, as described in the previous post in this series.

There are many logical categories of trimming warnings. Each requires a different strategy to achieve trimming compliance. For example, XML serialization was never designed with trimming in mind, so we marked it as 100% not trimmable, knowing that users who one day wanted to enable trimming would need to use a serializer like System.Text.Json that was designed for it. In other cases, uses of Reflection code can be replaced using a source generator that creates code at compile time.

The pipeline poses a much different challenge. As described in 10x faster execution with compiled expression trees, the 2017 redesign of the pipeline uses Reflection.Emit to literally compile expressions into new code that includes all the pipeline’s behaviors in one super-fast delegate type, which is executed for every message that comes through.

Except Reflection.Emit is a big no-no for trimming. You can’t cut out a bunch of the .NET runtime and then compile new code that might need all the stuff you trimmed away! ²

But the standard “use a source generator for that” solution to many trimming problems isn’t very helpful here. Compile-time source generators are a great solution for problems you can statically analyze at compile time. Still, the decision about which behaviors to include in the pipeline happens at runtime.

Did we really want to redesign how behaviors are defined and registered completely, and all the breaking changes that would entail?

That’s when I said it:

“Gee Daniel, it’d be super nice if you could just magically redesign the pipeline again, you know, make it go 10 times faster and be trim-capable too?”

No way that’s possible. We had already worked on pipeline performance so many times. Surely any additional performance enhancement would be like squeezing water from a stone. Right? ³

Although I don’t remember him saying anything that day besides maybe “Oh yeah, good one, David,” apparently, something deep in Daniel’s mind said, “Hold my beer.”

🔗Back to the drawing board

The 2017 expression tree approach underwent at least 7 design iterations before we landed on that solution. At the time, we were convinced we had taken the pipeline as far as it would go.

But the design space evolves. At that time, we were working with .NET Framework 4.7 and had just begun work on the first NServiceBus version supporting .NET Core. In the years since, we’ve gotten a lot more language tools added to our utility belt. Perhaps roads that were once dead ends were now worth re-exploring?

One of the pipeline design options we evaluated back in 2017 was functional trampolining, a technique that uses immutable continuation passing to avoid deep call stacks. A small mutable frame in the current message context tracks the current position within the behavior chain, with the “trampoline” incrementing the state index and handling dispatching duties to effectively “bounce” execution to the next behavior.

The problem with functional trampolining is that it requires an orchestrator to keep track of which step the pipeline is currently running, and then to handle the next() delegate and marshal execution into the next behavior in the chain. That all requires additional overhead. In 2017, compiled expression trees proved faster, so we went with that.

But Daniel knew that the basic idea behind functional trampolining would be much more friendly to code trimming. He decided to see whether the new tools C# provides today could provide a mechanism to remove the extra overhead.

🔗The breakthrough

Default interface members ⁴ (or DIMs) make it possible to have a static method of invoking the next node in a chain without having to compile anything dynamically. DIMs allow interfaces to provide a default implementation that all interface implementations inherit automatically. In this case, Daniel used the default interface method to allow each behavior to create its own invoker.

The result is prewired invokers – a statically typed continuation chain built from invoker nodes that call the interface method that each behavior inherits by default.

Instead of generating code at runtime, we build the chain using generics. Each behavior wires the next behavior directly into a typed continuation.

While we are simplifying away some of the more esoteric runtime optimizations and other details in this example, ⁵ an invoker node looks something like this:

abstract class InvokerNode
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public abstract Task Invoke(IBehaviorContext context);
}

sealed class InvokerNode<TInContext, TOutContext>(
    IBehavior<TInContext, TOutContext> behavior,
    Func<TOutContext, Task> next) : InvokerNode
{
    public override Task Invoke(IBehaviorContext context)
    {
        return behavior.Invoke(context as TInContext, next);
    }
}

Each NServiceBus behavior already implements IBehavior, but internal default interface members provide the machinery to create invoker nodes for each behavior:

public interface IBehavior
{
    internal InvokerNode CreateInvokerNode(InvokerNode? next);
}

public interface IBehavior<in TInContext, out TOutContext> : IBehavior
{
    Task Invoke(TInContext context, Func<TOutContext, Task> next);

    InvokerNode IBehavior.CreateInvokerNode(InvokerNode? next)
    {
        Func<TOutContext, Task> nextFunc = next is not null
            ? next.Invoke as Func<TOutContext, Task>
            : CompletePipelineDelegate;
        return new InvokerNode<TInContext, TOutContext>(this, nextFunc);
    }
}

The pipeline builder iterates behaviors in reverse, calling CreateInvokerNode() on each. Each behavior creates its own typed node and links it to the next. The result is an immutable chain where each node knows exactly how to invoke its behavior and pass control onward.

We still have to build an invocation chain at startup, but now we don’t have to do any expensive (and disallowed when trimming) dynamic compilation to wire it all up. No expression trees. No code generation step. No central factory that needs manual updates for every new stage. ⁶

Our production code has a few more optimizations, such as reducing allocations, than shown here. The real breakthrough, though, is default interface implementations. They make it possible to express this in a way the compiler can understand at build time — something that was impossible to do just a few years ago.

🔗Why it’s awesome

The performance gains are the visible win. Here’s what our benchmarks reveal:

• 330x faster startup - No more expression tree compilation at startup
• 25-29% faster execution on the success path
• Zero allocations on the success path
• Same allocation profile on exception paths (~1.3 KB)

But as impressive as those are, the architectural gains may matter even more over time. The new approach is compatible with trimming and AOT scenarios because it uses no dynamic code generation nor Reflection.Emit. That gives us smaller binary sizes, better trimming support, and removes one of the longstanding obstacles on the path toward future .NET deployment models.

That means less time waiting for endpoints to become productive and more time doing the work they’re intended to do.

If you want to take a look at the implementation yourself, you can see the evolution in these pull requests:

• #7625 Replace expression tree pipeline execution with prewired invokers (Exploratory, not merged)
• #7631 Replace expression tree pipeline execution with prewired invokers
• #7636 Refactor PipelineInvoker to decouple behavior invocation logic into dedicated InvokerNode and simplify pipeline assembly

🔗What changes for existing behaviors?

If you have written custom behaviors, nothing changes. Existing implementations continue to use the same public interfaces and automatically benefit from the new pipeline wiring.

The new wiring stays inside the framework. Your behaviors continue to implement the same Invoke method they always have. NServiceBus handles the rest.

If you’ve never created an NServiceBus behavior, perhaps it’s time to learn how to build your own NServiceBus behavior, or for more general information, read our blog post Infrastructure soup to see the kinds of cross-cutting concerns that the NServiceBus pipeline was designed to solve.

🔗Summary

The pipeline has been made impossibly faster once again and is ready for future assembly trimming and ahead-of-time compilation scenarios. Yet, most users will never need to think about this change because all existing behaviors will experience benefits from this improvement with no changes necessary.

That combination matters to us. NServiceBus should stay easy to extend while we steadily remove runtime costs and architectural blockers that matter more and more in modern .NET applications.

This is a huge victory for the good guys! But at what cost?

So… I nerd-sniped Daniel into a pipeline upgrade that, against all odds, delivered trimming compatibility and was also a lot faster than the previous implementation.

But the most damning thing of all… I think I can live with it. And if I had to do it all over again, I would. A guilty conscience is a small price to pay for NServiceBus performance.

So I will learn to live with it.

Because I can live with it. I can live with it… ⁷