3 posts tagged with "Technical Deep Dive"

Native MQTT released in RabbitMQ 3.12 has delivered substantial scalability and performance improvements for IoT use cases.

RabbitMQ 3.13 will support MQTT 5.0 and will therefore be the next big step in our journey to make RabbitMQ one of the leading MQTT brokers.

This blog post explains how the new MQTT 5.0 features are used in RabbitMQ.

RabbitMQ's core protocol has been AMQP 0.9.1. To support MQTT, STOMP, and AMQP 1.0, the broker transparently proxies via its core protocol. While this is a simple way to extend RabbitMQ with support for more messaging protocols, it degrades scalability and performance.

In the last 9 months, we re-wrote the MQTT plugin to not proxy via AMQP 0.9.1 anymore. Instead, the MQTT plugin parses MQTT messages and sends them directly to queues. This is what we call Native MQTT.

The results are spectacular:

1. Memory usage drops by up to 95% and hundreds of GBs with many connections.
2. For the first time ever, RabbitMQ is able to handle millions of connections.
3. End-to-end latency drops by 50% - 70%.
4. Throughput increases by 30% - 40%.

Native MQTT turns RabbitMQ into an MQTT broker opening the door for a broader set of IoT use cases.

Native MQTT ships in RabbitMQ 3.12.

The team was recently asked about whether and how quorum queues can offer the same message ordering guarantees as classic queues given that they will deliver messages from a local queue replica (leader or follower) when possible. Mirrored queues always deliver from the master (the leader), so delivering from any queue replica sounds like it could impact those guarantees. 

That is the subject of this post. Be warned, this post is a technical deep dive for the curious and the distributed systems enthusiast. We’ll take a look at how quorum queues can deliver messages from any queue replica, leader or follower, without additional coordination (extra to Raft) but maintaining message ordering guarantees.