MassTransit: Configure queue for multiple API instances

I’m introducing myself into Event-driven architecture using MassTransit with RabbitMQ on localhost and AWS on live environments, using .NET 5 Web APIs.

I have an API (#API-1) which produces events, such as EntityCreated, EntityUpdated, EntityDeleted, etc.

Then I have another API (#API-2) which stores an in-memory copy of entities. This #API-2 must subscribe to those events in order to update its local cache.
The #API-2 will have multiple running instances, let’s say 3.

• How should I configure #API-2 so that all of the instances receive the same message?

Producer Configuration (#API-1):

services.AddMassTransit(x =>
{
    x.UsingRabbitMq();
});
services.AddMassTransitHostedService();

Consumer Configuration (#API-2):

services.AddMassTransit(x =>
{
    x.AddConsumer<EntityCreatedConsumer>();
    x.AddConsumer<EntityUpdatedConsumer>();
    x.AddConsumer<EntityDeletedConsumer>();

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.ConfigureEndpoints(context);
    });
});
services.AddMassTransitHostedService();

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You need to use different endpoint names. Each endpoint gets its own queue and binding, so if you use the same endpoint name, you get one queue and all instances started to compete for messages.

It is a common mistake, that’s why it’s even mentioned in the documentation.

Method 2

To properly configure instance-specific endpoints for your consumers, you can use the Endpoint configuration when adding the consumer. I’ve updated your code to show how to use it.

string instanceId = "SomeUniqueValue";

services.AddMassTransit(x =>
{
    x.AddConsumer<EntityCreatedConsumer>()        
        .Endpoint(e => e.InstanceId = instanceId);
    x.AddConsumer<EntityUpdatedConsumer>()
        .Endpoint(e => e.InstanceId = instanceId);
    x.AddConsumer<EntityDeletedConsumer>()
        .Endpoint(e => e.InstanceId = instanceId);

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.ConfigureEndpoints(context);
    });
});
services.AddMassTransitHostedService();

The instanceId can be pulled from the environment, or even random. If you want the queues to be removed when the process exits, you can specify:

.Endpoint(e =>
{
    e.InstanceId = instanceId;
    e.Temporary = true;
});

This was added in v7.0.4

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0