What is a servicebus?
The first time I read about the concept of a servicebus and saw the images, I could not figure out how it was supposed to work. The abstract perspective is like any component could just send messages to something that looked like a pipeline. Anyone interested could pick up random messages as it would seem fit. So what is a servicebus?
Abstract servicebus
The abstract concept shows like the image on the right. There are sources and sinks, which are respectively senders and receivers of messages. The idea is that sources and sinks are completely decoupled. This is achieved by making use of the publish/subscribe pattern. A source, the publisher, publishes a message and has no idea about the sinks, the subscribers. Any sink can then subscribe to a message and receive the message. This isn’t magic though.
A publisher has no clue who will subscribe to the message, especially during design-time. It depends on the queuing technology used, how subscribers tell a publisher they are interested in a certain magic. But there definitely is some knowledge needed with the subscriber which messages are published. Let’s get a little bit more concrete.
Federated queues
In the Microsoft world, an old but very stable and reliable queueing technology is MSMQ. When you want to send a message to a component on another machine, MSMQ will first locally persist the message to disk. This way it guarantees the message doesn’t get lost when the node crashes. The executing code can continue, while MSMQ tries to send the message to the other machine. The other machine will receive the message and immediately persist the message to disk. Only then will the code on the other machine be able to read the message from the queue.
Even when a machine is down, the message is persisted. MSMQ will keep trying to send the message as long as the machine is down. This all assumes that you’ve set up transactional queues to achieve this kind of reliability. You can add a SAN for additional storage reliability.
Brokered queues
A lot of queueing technologies are not running locally on every machine, but rather centrally located. Think of a relational database as being a central component to a lot of applications, that have the need to persist information. Examples of a brokered queue is RabbitMQ, Azure Service Bus or Amazon SQS.
The drawback is that, because it’s centrally located, it can be a single point of failure. For this reason you might want to set up a cluster. This can turn out to be very complex. Especially when networking issues cause a partition within the cluster and you have to solve the issues that a brokered queue can’t solve for you. Another option is to turn to the cloud with Azure Service Bus or Amazon SQS. Although that can lead to availability issues due to network issues. Too many times I’ve seen companies unable to send messages because they could not reach the cloud. Hosting your entire system in the cloud might help. Most of the time the connectivity within a single datacenter isn’t compromised. And when your brokered queue is offline, your website likely is offline as well.
Scaling out
A benefit of brokered queues is that it is very easy to scale out, using the competing consumer pattern. When designed and developed properly, it’s just a matter of adding another instance of an endpoint and you’ve basically scaled out your system. With federated queues you’ll need a message dispatcher or another way to distribute messages over workers, for example using round-robin.
Broker vs buses
There’s a difference between brokered queues and message brokers. It is important to recognize that brokered queues are basically just infrastructure and contain no logic. Whereas message brokers differ from a servicebus in the fact that they can contain (content based) routing, transformations and logic and can aggregate and decompose messages. This can be very handy for integrating with third-party systems, like SAP, SalesForce, etc. The issue is that it is too easy to have too many responsibilities in a broker. Especially when business logic is centralized, which is a fallacy.
Summary
When choosing a message transport for your system, be aware of the drawbacks of each of them. Think about connectivity, availability and the ability to scale. And especially be aware of adding any logical centrally, because you shouldn’t.
Dennis is an independent consultant and software architect who loves building distributed systems and the challenges they bring. What he learns he tries to share via numerous articles, blogposts, presentations and workshops.
So Biztalk is a message broker and NServiceBus is a servicebus that can use different queueing technologies (messaging?) like federated and brokered queues? So the word ‘broker’ in brokered queues and message brokers have no relation with each other?
The whole vocabulary is confusing as hell 🙂
Also, when looking specifically at NServiceBus:
NServiceBus is a servicebus right?
NServiceBus is built on top of existing queuing technologies. In NServiceBus the choice of queuing technology is referred to as a “Transport”.
NServiceBus transports can be divided into several categories.
Bus transports. Check. Service *bus*
Bus transports include:
MSMQ
Also possible, Broker transports. Uhhhh… does this mean that it is not a service *bus* anymore? Like now it is a service *broker*? Is a servicebus using a brokered transport a servicebroker?
That’s a lot of questions! And I agree the vocabulary is confusing. Don’t get me started on the word “service”. Or about the difference on a “servicebus” and an “enterprise servicebus”. I seriously have no clue what the difference is between the two. Try searching the internet for it. Or better, don’t. 🙂
BizTalk is a message broker. It was originally sold as such and later an additional download was offered, which magically made it an “enterprise servicebus”. Like when Word is a text editor, but when you download an additional package, it suddenly becomes a website designer or a spreadsheet. It doesn’t work like that. It’s likely they did this because “servicebus” became a buzzword back in the days.
Just remember that tools like MassTransit, NServiceBus, etc. can use different transports like MSMQ, Azure Service Bus, RabbitMQ, etc. There’s a difference between MSMQ and ASB & RabbitMQ in that the latter two are centrally located and MSMQ is on every machine. That’s where I made the difference.
It gets really, really complicated, because everyone uses the terms differently. This is from Wikipedia about RabbitMQ:
> RabbitMQ is open source message broker software (sometimes called message-oriented middleware)
And then remember that RabbitMQ also has support for federated exchanges : https://www.rabbitmq.com/federated-exchanges.html
So why does RabbitMQ calls itself a message broker and Azure Service Bus calls itself a servicebus? That’s why we at Particular Software call them transports and differentiate between federated and brokered transports. Some transports have more features like others. MSMQ & Azure Storage Queues don’t support PubSub natively, ASB & RabbitMQ do. NServiceBus abstracts away these issues and makes sure you can use PubSub on every transport and adds a lot of additional features. Competing Consumer is ‘easy’ for brokered transports, you don’t specifically need a framework like MassTransit or NServiceBus for that .But what about concurrent message processing on a single node? Or retries for transient & non-transient errors? Or reliability for when you process messages from the queue and insert business data into a database, but can’t or won’t use distributed transactions?
You can all build this yourself and it’s immense fun. But it’ll take an insane amount of time to do it and do it right. I can promise you it’s cheaper to take a servicebus from the shelf.