The Transactional Outbox Pattern implementation in .Net Core

You could find the Sample Project code here :

https://github.com/mogh64/OutboxPatternSample

A service command typically needs to update the database and send messages/events. For example, a service that participates in a saga needs to atomically update the database and sends messages/events. Similarly, a service that publishes a domain event must atomically update an aggregate and publish an event. The database update and sending of the message must be atomic in order to avoid data inconsistencies and bugs. However, it is not viable to use a distributed transaction that spans the database and the message broker to atomically update the database and publish messages/events.

Suppose that in a microservice implemented architecture, you change some data in one of your services that has a relational database such as MSSQL Server as an underlying database, so you would want to propagate this change to another service with the elastic-search database as a search engine.

So the problem here is that how to reliably/atomically update the database and publish messages/events?

The solution is, a service that uses a relational database inserts messages/events into an outbox table (e.g. MESSAGE) as part of the local transaction. A service that uses a NoSQL database appends the messages/events to attribute of the record (e.g. document or item) being updated. A separate Message Relay process publishes the events inserted into the database to a message broker.

I have implemented a sample project to describe it in practice, so you could find it here: https://github.com/mogh64/OutboxPatternSample

So, for implementing this idea we should add below architectural components to our basic design:

Let’s go into the code!

At first, I will define IMessagePublisher Interface that has two separate implementations: OutboxPublisher which publishes messages into the Outbox table in the database, and RabbitMessagePublisher that publish messages from the outbox table into RabbitMQ message broker in the Worker service.

So the OutboxPublisher would be :

OutboxPublisher creates an OutboxMessage from the received event and stores it in the database using a repository.

The OutboxMessage could be defined like the below code:

Data in OutboxMessage is a JSON structure of the event which is stored in the database, so by RecreateMessage, the event object is reshaped using the stored JSON data.

OutboxRepository Implementation could be like this:

Afterward, on the worker side, we have an implementation of IMessagePublisher that publishes messages to RabbitMQ. For abstracting communication with RabbitMQ message broker I have used Masstransit message bus:

So the worker has a job scheduler like Quartz that in a specified period reads from the outbox table and publish events to the message broker :

The next step is consuming messages from the message broker and store them in the elastic-search database. So it is the responsibility of the Consumer component to do this. MassTransit give you an abstraction to consume messages by implementing a specific Consumer for each message.

GenericSyncRepository has implemented to store documents into elastic-search :

Also, we could have a GenericReadRepository for Search API:

With this Implementation to connect to elastic-search with Nest library:

Finally, just we will call it in the API:

Resources:

https://microservices.io/patterns/data/transactional-outbox.html

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mortaza Ghahremani

A Passionate software developer & engineer with many enthusiasm to learn new technologies and concepts to solve big challenges