Resistance is futile...unless you have ServicePulse

We are the Borg. Your messages have failed and will be assimilated. Resistance is futile.

Distributed systems are a lot like Star Trek’s infamous Borg Collective: massively parallel, highly interconnected, and sometimes, drones services go down or are inaccessible. After all, even with a galaxy-class architecture, things sometimes go wrong. Messages can fail due to transient errors, unexpected exceptions, or configuration hiccups. However, well-designed distributed systems take a stronger cue from the Borg; they are built to be resilient. But how? Let’s explore a more peaceful quadrant of the galaxy to illustrate how this works.

🔗The Starship Enterprise vs. distributed failure

In the world of Star Trek, many things can be replicated with a simple command, and whole universes can be generated with a few phrases in the Holodeck. Still, there are plenty of situations where people want the real thing, whether it’s a bottle of Château Picard, a Klingon bat’leth, or an authentic Bajoran earring.

Imagine you are an operations officer on a Federation starship running the Starfleet Supply Chain system in LCARS. Naturally, it is a distributed system. Orders made on the ship are routed through an NServiceBus-powered system.¹ One of the commands looks like this:

public class PlaceOrderCommand : ICommand
{
    public string OrderId { get; set; }
    public OrderItem[] Items { get; set; }
    public string Destination { get; set; }
}

public class OrderItem
{
    public string Sku { get; set; }
    public int Quantity { get; set; }
}

Orders are handled via messages like this:

public class PlaceOrderHandler : IHandleMessages<PlaceOrderCommand>
{
    private readonly IFederationSupplyApi _supplyApi;

    public PlaceOrderHandler(IFederationSupplyApi supplyApi)
    {
        _supplyApi = supplyApi;
    }

    public async Task Handle(PlaceOrderCommand message, IMessageHandlerContext context)
    {
        // This call to the Federation Supply Service will fail when shields are up
        await _supplyApi.SubmitOrderRequest(message.OrderId, message.Items);
        
        Console.WriteLine($"Submitted order {message.OrderId} with {message.Items.Length} items to {message.Destination}.");
    }
}

But oh no! Just as you are placing your own personal order with the computer for a bona fide Klingon d’k tahg, a Romulan ship decloaks off the starboard bow, and the ship’s shields go up. The shields interfere with subspace communications to the Federation Supply Service, causing the external API calls to fail with a ServiceUnavailableException. Did your order go through?

🔗Retry protocols engaged

NServiceBus detects the failure and applies its first-level and second-level retry strategies. But if retries are exhausted and the issue remains, the message is moved to the error queue—not dead-lettered, not deleted, just safely quarantined.

This is where ServicePulse beams in.

🔗ServicePulse: Your command center for failed messages

Unlike many systems that silently drop or bury failed messages in a dead-letter queue that might as well be beamed into space, NServiceBus gives you a mission control interface—ServicePulse.

As shown in the image below, ServicePulse groups failed messages by exception type, and originating endpoint:

In this case, 174 messages failed with a ServiceUnavailableException inside the PlaceOrderHandler handler. These messages were all of type PlaceOrderCommand.

It turns out the Romulan ship is actually part of training exercises that are now complete. With shields down, subspace communications to the Federation Supply Service are restored. We can now perform a level 3 diagnostic. From LCARS, we open ServicePulse and see that there is a list of failed messages.

🔗Manual intervention isn’t futile

Sometimes, you want to investigate specific errors more deeply. With ServicePulse, you can drill down into individual messages, inspect the exception stack-trace, message headers, even the message body, and choose to retry a single message, or even a whole group of messages. This level of control means your system is both resilient and humanoid-friendly.

There is no need to SSH into remote systems to access queues, ² write ad-hoc scripts, or run a recursive algorithm through the main deflector dish—ServicePulse makes distributed systems self-heal like a Zalkonian undergoing transformation.

In this case, the cause for the failures is clear and the solution straightforward. With a single click on Retry all, ServicePulse sends those messages back to the original endpoint’s input queue. With things back to normal, those same messages can now be replayed successfully and your own order is on its way.

🔗NServiceBus vs. the Collective

In any world, message failure is inevitable, but NServiceBus and ServicePulse give your distributed system the tools to bounce back. With smart retry logic, UI-driven error recovery, and grouped failure analysis, you don’t have to wonder if your failed messages got assimilated by the Borg. In other words, resistance is not futile.

🔗Looking to improve failure recovery?

• Try recovering from failure using ServicePulse in one of our tutorials
• Learn about NServiceBus recoverability and retries
• Read the related blog post I caught an exception. Now what?