Reliable and resilient financial systems - with NServiceBus

With financial systems, your applications need to be rock solid and should just work. But you still need to handle the increasingly complex landscape of regulations in an industry that can be local, regional, or global, often all at the same time. In short, you need an approach to system architecture that is reliable, resilient, and flexible. After all, you're dealing with the world's money.

Building a system with NServiceBus and the Particular Service Platform will let you:

• Reliably process requests ensuring full data consistency.
• Integrate easily with multiple external systems.
• React in real time to a demanding client base.
• Handle a complex domain that is ever-changing.

Preventing data loss

NServiceBus stores every request for work as a message in a durable queue. If processing fails, the message goes back to the queue and gets retried, so a crashed server or a network timeout won't cause data loss.

There's an obvious, often unspoken rule in the financial industry: don't lose people's money.

In financial systems, that translates to: don't lose data. But if you build a distributed system based on HTTP calls over remote procedure calls (RPC, i.e., REST), you run this risk with every call. If the call fails, the data is gone.

Even worse, what if an unreliable network causes an HTTP request timeout? Did the server successfully process the request, but then time out while sending the response? Did the server never receive the request in the first place? There’s no way to know.

In a financial system built with NServiceBus, your data is recorded immediately to a message in a queue, where it's safely stored to disk. If anything fails while it's being processed—if a server crashes, if the database is down, or if a third-party service fails—the message goes back to the queue and waits to be processed again.

When your data is kept safe, it can save money, prevent lawsuits, and keep you focused on what matters: delivering value.

Retries are safe, too. The outbox ensures that a retried message doesn't update your database or send outgoing messages twice. Idempotency matters in distributed systems and the outbox helps you achieve it.

Designing your processes with the Particular Service Platform means when one step in the process fails, it will automatically get retried. No more late-night phone calls about failed batch jobs. If a message still can't be processed after all retries, it's moved to an error queue, where you can inspect it and retry it in ServicePulse once the problem is fixed.

Watch this video for a better picture of how failures can lead to losing financial data or other critical customer data, and how using NServiceBus will keep that data safe.

Coordinating workflows with external services

NServiceBus Sagas track which steps of a workflow are done, send messages to the services that still need to act, and set timeouts for partners who don't respond in time.

Financial systems don't operate in a vacuum. Often, a workflow requires interaction with several external services, such as tax authorities, insurance companies, credit bureaus, property inspectors, and regulatory offices, and often with other internal systems. Coordinating a complex workflow between these systems can be a lot to wrap your head around, especially when you need to worry about service level agreements (SLAs) and what to do when a service is down in the middle of a process.

With NServiceBus sagas, you can easily manage complex workflows that involve integration with third-party services. Think of a saga as a message-driven state machine that takes messages in, makes decisions, and sends messages out to other handlers for processing. It's a simple but powerful concept.

Consider a mortgage application where you need to contact a credit bureau, a property inspector, and an insurance company. Here's how the saga handles it:

1. The saga receives a request to process a mortgage application.
2. It sends messages to two different handlers: one to do a credit check, another to arrange a property inspection.
3. The credit check handler responds with an approval. The saga checks whether the property inspection has been done. It hasn't, so the saga goes back into a holding pattern until it has.
4. A message arrives saying the property inspection is done. The saga wakes up and checks whether both the property inspection and the credit check are done. At this point, they are.
5. All conditions are met for the next stage, which is to find an insurance company that will underwrite the mortgage. The saga sends another message, and the workflow continues.

This is a simple example, but it gives you an idea of how a saga can coordinate multiple services. Sagas can also handle any deviation from the normal, expected sequence of events:

• What if there's no response from the property inspector within the required SLA? Saga timeouts work like an alarm clock, so the workflow can continue even without an external trigger.
• Did the credit check come back with a rejection? Publish an event, and another process can send an appropriate message to the sales department so they can decide how to proceed with the mortgage application.

With NServiceBus sagas, these concepts are expressed as code. That means the complex business logic can be verified with unit tests, even when modeling the passage of time. You're in total control, every step of the way. The third-party integration tutorial shows how it works in code.

Replacing batch jobs with real-time processing

With NServiceBus, each step in a process starts as soon as the previous step publishes an event, not at the next scheduled batch run. Saga timeouts handle work that has to wait, one account or mortgage at a time.

In the "old days", people would make payments with their checkbook and track their current balance in their check register. Then, once a month, they would receive a statement from their bank in the mail, and balance their checkbook by reconciling all of the purchases made with line items from the bank statement.

Not so much anymore.

Today, customers expect instant access to their account data from the web and a mobile app, with up-to-the-minute updates on recent transactions and account balances.

So it is across the finance industry. Real-time services like Fedwire and TARGET have been around for decades, but only recently have customers started demanding the same across all products and services. The push to be real time in personal banking, mortgages, lending, and securities trading is ever-present. A few examples:

• Often, the company that wins a mortgage is the first to reach out to a prospective borrower with a rate quote.
• Trading platforms need to react quickly to changing market data to make the most of invested capital.
• Customers want wire transfers to be executed quickly, but regulations require due diligence and internal approvals.

Making things truly real time in an industry still reliant on mainframes and flat-file integration is a challenge, but message-based processing with NServiceBus can close those time gaps to near zero.

Using the publish/subscribe pattern, events that occur in one part of the system can trigger other processes immediately, rather than at the next execution of a batch job. This makes loosely related processes execute on demand, without suffering from the coupling that comes from tightly orchestrating the steps in the process. This makes it easier to evolve the business processes later.

NServiceBus sagas can make it possible to model time by requesting a timeout message that acts as an alarm clock to take an action at some time in the future. This lets you take scheduling to a very granular level, down to each entity like an account or mortgage. Instead of checking every entity for some condition every night, each entity sets its own timeout for when the next action needs to occur. When the timeout is due, the process wakes up and makes decisions for that specific entity in real time.

By breaking processes down into discrete messages and message handlers, the wait time between processes shrinks, so that everything that used to involve a lot of waiting happens instantly.

For more about this scenario, see our blog post Death to the batch job.

Designing systems for complex financial regulations

NServiceBus lets you build each compliance step, like Know Your Customer (KYC) checks or enhanced due diligence, as its own message handler with clear boundaries. When regulations change, you add or update one piece of the system instead of rewriting the whole process.

Finance is one of the most complex industries in existence. Ever since the invention of fiat currency, the things humans can do with money have been limited only by our imagination. Unfortunately, that also means virtually unlimited possibilities for exploitation by bad actors. As a result, finance is also one of the most regulated industries in existence, to combat fraud and money laundering.

In financial systems, complexity starts with the various ways in which money can be manipulated. It grows due to regulation added to prevent abuse, and is compounded by the number of integrations, both standard and proprietary, that must be implemented to ensure compliance with those regulations.

Software systems in the financial services industry must overcome regulatory challenges such as:

• Know Your Customer (KYC)
• The Customer Identification Program (CIP) provision of the USA PATRIOT Act
• Customer due diligence
• Enhanced due diligence

Of course, there are external vendors for all of these services, but integrations still need to be built that can reliably reach out to those vendors and then orchestrate the results.

NServiceBus sagas help by defining complex business processes as message-driven state machines. Sagas passively wait for incoming messages and then take action when conditions are right, or they can take control and orchestrate processes by stepping through a workflow. Regardless, the logic defined in an NServiceBus saga can be easily tested, so that when business logic needs to change, you can be sure the business process won't fail.