Vicedomini Softworks

AI & Automation

Kafka vs RabbitMQ: How to Choose the Right Broker

8 November 2026

Architects compare message broker architectures

Apache Kafka and RabbitMQ are both used to move messages between services, but they are built on different premises. Kafka is a distributed log that retains events and lets any number of consumers re-read them. RabbitMQ is a broker that routes messages into queues and removes them once they are acknowledged. The right choice depends on whether the workload is event streaming with replay or task distribution with flexible routing, and many production systems use both.

Vicedomini Softworks
Design Reliable Message Integrations
Vicedomini Softworks builds custom backend systems and integrations, helping businesses connect complex technology ecosystems with maintainable software.
Ask for a consultation

TL;DR

  • Kafka stores events in partitioned, replicated logs that are retained independently of consumption, so multiple consumer groups can read and replay the same data.
  • RabbitMQ routes messages through exchanges to queues, with priorities, per-message TTL, dead-lettering and multi-protocol support, which suits task queues, request/reply and device connectivity.
  • The two have converged: RabbitMQ has offered Streams since version 3.9, and Kafka added queue-style share groups in version 4.2.
  • Benchmarks disagree because results depend on message size, durability settings and hardware, so the workload should be measured rather than assumed.
  • Combining them, with Kafka as the event backbone and RabbitMQ as the task-routing layer, is a common and sound pattern.

Table of Contents

The core difference: a log versus a queue

The distinction that determines most downstream behaviour is what happens to a message after it has been read. In Kafka, records are written to named topics and stored in partitioned, replicated logs on broker disk. They remain there according to retention rules, whether or not anyone has consumed them. Each consumer group tracks its own position, called an offset, in each partition, so several groups can independently read the same topic.

RabbitMQ implements AMQP 0-9-1 as its original model: producers publish to exchanges, the exchanges route messages to queues, and the broker pushes messages to consumers. Once a consumer acknowledges a message, the broker deletes it. The Vercel engineering team summarises the consequence well: queues are mutable and topics are immutable logs, and ordering, replay, consumer scaling and operational burden all follow from that single choice.

Both are open source. Apache Kafka is described by its project as a distributed event streaming platform used for data pipelines, streaming analytics and data integration, while RabbitMQ is released under the Mozilla Public License 2.0 as a messaging and streaming broker. Both are actively maintained: Kafka 4.4.0 was released on 22 October 2026, and RabbitMQ’s latest release is also 4.4.0.

The core difference: a log versus a queue

The gap is narrowing

A comparison written five years ago would draw a sharper line than is now accurate. According to the RabbitMQ documentation on its comparison with Kafka, RabbitMQ has had an append-only, replicated, disk-based log, Streams, since version 3.9 in 2021. Kafka gained queue semantics through share groups (KIP-932) in version 4.2 in 2026. High-level capabilities therefore overlap substantially, although each system still carries the assumptions of its origin: Kafka is a log first, RabbitMQ a queue-and-routing broker first.

Delivery, ordering and replay

The practical differences appear in how consumers receive data, how ordering is preserved and whether history can be revisited.

Criterion Apache Kafka RabbitMQ
Storage model Partitioned, replicated log retained by time or size Queues that empty on acknowledgement; Streams as an append-only log
Consumption Pull; consumer tracks its own offset Push to subscribed consumers for queues; offset-based reading for Streams
Replay Native; rewind to any retained offset Via Streams only; classic and quorum queues delete on acknowledgement
Ordering Strict within a partition FIFO within a queue while a single consumer reads it
Per-message controls None natively (priorities, TTL, delayed delivery) 32 priorities, per-message TTL, delayed delivery, dead-letter routing
Protocols Kafka protocol only AMQP 1.0, AMQP 0-9-1, MQTT, STOMP, JMS, stream protocol

The table draws on the RabbitMQ comparison documentation, which covers delivery, retention, ordering and per-message controls for both systems.

Ordering in practice

Kafka guarantees order within a partition. A partition key therefore gives per-entity ordering, for example all events for one customer or one order, while different entities are processed in parallel. RabbitMQ’s first-in-first-out ordering holds only while a single consumer reads a queue. As Vercel’s analysis notes, competing consumers break that order unless Single Active Consumer is used to serialise processing, which in turn limits parallelism on that queue.

Replay and the common misconception

A frequent assumption is that RabbitMQ always deletes a message as soon as it is read. That was true of classic queues, but RabbitMQ Streams behave like Kafka: consumers attach at an offset and read forward, and the data stays available after consumption. The practical rule is that replay and independent consumers moving through history are Kafka’s native territory, while RabbitMQ reaches the same ground through Streams and its older queue types do not.

Kafka share groups

Kafka’s share groups allow consumers to acquire individual messages under a time-limited lock and acknowledge them one at a time, and consumers can outnumber the partitions they read from. The RabbitMQ documentation points out a trade-off: because the state of which consumer holds a message lives in the share-partition leader’s memory, a broker failover produces a burst of redeliveries. Consumers should therefore be idempotent regardless of the broker chosen.

Reliability and durability

Both systems replicate data across nodes, but their defaults differ. Kafka’s default reliability model relies on replication alone: when a broker acknowledges a message, it sits in the page cache of several brokers but has not necessarily been flushed to disk. Per AceMQ’s comparison, RabbitMQ quorum queues replicate and fsync to disk before confirming to the publisher, which is a materially stronger guarantee for payments, orders and regulated workloads. Kafka can be configured towards stricter flushing, but that is a deliberate tuning decision with throughput consequences.

Kafka and RabbitMQ durability guarantees compared

RabbitMQ 4.0 removed classic queue mirroring in favour of quorum queues, which are based on the Raft consensus protocol. For Kafka, cluster coordination now uses KRaft, which replaced ZooKeeper, as AWS describes.

Delivery semantics also deserve attention in system design. An IBM community post by Kim Clark argues that messaging suits transactional integrity, whereas Kafka’s at-least-once delivery works best when consumers poll for batches of records at high rates. Either way, a consumer that must not apply the same business action twice needs idempotency keys or deduplication at the application layer.

A 2025 GlobeCom paper on message-oriented middleware concluded that Kafka outperformed RabbitMQ on performance and security for mission-critical applications and needed far less bandwidth. It also reported that around a quarter of Kafka messages could be lost in the presence of network partitions, which means resilience mechanisms in the application remain a requirement. Fault injection testing is therefore a sensible part of any broker evaluation.

Throughput and latency: what benchmarks show

Published benchmarks do not agree, and the disagreement is instructive. Results move with message size, acknowledgement settings, batching, hardware and the version tested.

  • In a cross-broker benchmark run by Confluent and summarised by AceCloud, Kafka reached roughly 605 MB/s peak throughput with a p99 latency of about 5 ms at 200 MB/s. RabbitMQ achieved about 1 ms latency but only at a reduced 30 MB/s load, and its latencies deteriorated above that. Confluent is a Kafka vendor, so the result should be read with that in mind.
  • A 2026 academic study in JUTI found Kafka stronger with 10 KB messages (105.24 against 48.37 msgs/s), yet RabbitMQ far ahead with 100 KB messages (117.67 against 12.08 msgs/s) and in stress tests (158.97 against 40.19 msgs/s).
  • A 2024 thesis benchmarking brokers on Azure Kubernetes Service with the OpenMessaging Benchmark Framework found Kafka the overall better-performing broker.

The Kafka project itself states that it delivers high throughput at network-limited rates with latencies as low as 2 ms, scaling to a thousand brokers and hundreds of thousands of partitions. For streaming specifically, the RabbitMQ documentation says RabbitMQ Streams and Kafka land in the same ballpark on throughput and latency.

The defensible reading is as follows. Kafka is engineered for sustained high-volume streams and scales horizontally through partitions. RabbitMQ delivers very low latency at moderate load and, thanks to its push model, usually posts lower end-to-end latency for background jobs and request/reply, because no poll interval separates arrival from delivery. Neither number is portable to another environment; a benchmark using the production message size, durability settings and consumer behaviour is the only reliable arbiter.

Operations, protocols and cost

Operational surface is where teams often feel the difference most. According to the Vercel comparison, Kafka requires management of partitions, rebalancing and retention tuning, with a minimum of three broker nodes for production. RabbitMQ has a lower operational surface: a single node is viable and a management UI is built in.

Protocol support shapes integration. RabbitMQ natively supports AMQP 1.0, AMQP 0-9-1, MQTT, STOMP, JMS and its stream protocol, so devices publishing over MQTT, microservices over AMQP 1.0 and legacy applications over AMQP 0-9-1 can all land messages in the same stream. Kafka clients must speak the Kafka protocol, which is supported by mature libraries in most languages but requires an adapter or connector for other sources.

On cost, both can be self-hosted or consumed as managed services. Managed offerings exist from the major clouds and from specialist vendors, and their price lists change often, so current figures should be taken from the provider’s own pricing page. The structural point holds regardless of price: Kafka’s three-node minimum and tuning effort raise the baseline for small workloads, whereas its economics improve as sustained volume grows.

When each broker fits

The RabbitMQ documentation summarises the split concisely: Kafka suits event streaming, replay, multiple independent readers, analytics and long histories, while RabbitMQ suits task queues, background workers, request/reply, flexible routing, priorities and IoT connectivity over MQTT.

Workload Better starting point Reason
Clickstream, telemetry and audit event history Kafka Retention, replay and many independent consumer groups
Order events feeding analytics, search and billing Kafka Per-key ordering and the same history read by several teams
Email, PDF generation, webhook delivery RabbitMQ Work queues with retries, TTL and dead-letter handling
Request/reply between services RabbitMQ Push delivery and low end-to-end latency
Device connectivity over MQTT RabbitMQ Native protocol support
Prioritised or delayed jobs RabbitMQ Native priorities and delayed delivery

Field experience is consistent with this. Checkr moved from RabbitMQ to Kafka for greater messaging reliability, valuing disk persistence and a default seven-day retention that allows replay. Such migrations are driven by the particular failure modes of a given deployment, and the lesson is to match the broker to the access pattern rather than to follow a trend.

Using Kafka and RabbitMQ together

The two are not mutually exclusive. Confluent’s comparison notes that RabbitMQ can handle low-latency operations while Kafka provides durable, high-throughput streaming, and that RabbitMQ can collect data from varied sources and forward it to Kafka for processing and analytics. A typical division is as follows.

  1. Every state change, such as an order being placed or paid, is published as an immutable event to Kafka, where several services consume it independently.
  2. An integration consumer translates the events that require action into commands and places them on RabbitMQ work queues.
  3. RabbitMQ workers execute the tasks, with retries and dead-letter handling for failures.

The principal hazard is the dual-write problem: updating a database and publishing a message are two operations that can fail independently. The transactional outbox pattern, in which the event is written to the database in the same transaction and relayed afterwards, is the usual remedy. Classifying messages before choosing a technology, commands to RabbitMQ and event streams to Kafka, avoids most of the confusion.

What ‘Kafka vs MQ’ means

Searches for Kafka against MQ generally refer to traditional message queuing, whether IBM MQ, ActiveMQ or RabbitMQ, as a category. IBM’s own explanation is that enterprise messaging excels at request-for-processing scenarios, while event streaming specialises in publish/subscribe with persistent stream history. The two are complementary rather than interchangeable.

A queue treats a message as a command addressed to a recipient and removes it after delivery, a point-to-point model with destructive consumption. A log treats an event as a fact that stays available, so new consumers can be added later without the producer knowing. That is a stronger form of decoupling across teams, but it also demands more discipline in schema management and retention.

Perspective

In our delivery work, the broker is rarely the hard decision. The hard decision is what the system must be able to do two years from now: replay history for a new analytics consumer, absorb ten times the volume, or integrate a legacy application that only speaks one protocol. A broker chosen for the first month’s workload tends to be the one that is migrated away from at the worst moment.

For that reason we begin with the message classes. Commands that must be executed once, with priorities and retries, point towards RabbitMQ. Events that several teams will want to read, and possibly re-read, point towards Kafka. Where both exist, which is common in enterprise and SaaS platforms, we design the boundary between them explicitly, with an outbox for consistency, idempotent consumers and observability across both.

Because our engineers work directly with clients, with no account-manager layer, that analysis ends in a written recommendation that the people who will operate the system have helped to shape. Message-driven architecture is part of our everyday stack alongside Java, Quarkus, Spring Boot and Kubernetes-based platforms, and we have no vendor bias: the recommendation follows the workload.

Pepe F.

Choosing a broker with an engineering partner

A broker decision touches data models, failure handling, hosting and team skills, so it is best taken as part of an architecture review rather than in isolation. Vicedomini Softworks is an Italian software engineering company serving organisations across EMEA and North America, and its engineers handle architecture and technology stack selection, system integration, legacy modernisation and long-term support for message-driven platforms.

Vicedomini Softworks

For organisations that want senior technical guidance on an ongoing basis, our CTO Advisory, Fractional CTO and CTO Partner plans start from €1,800, €4,000 and €7,500 per month respectively, and an initial assessment is available from €3,500 one-off. Teams that need the build itself can review our architecture, integration and development services, and every consulting engagement concludes with written recommendations.

FAQ

When would you use Kafka over RabbitMQ?

Kafka is the better fit when events must be retained and replayed, when several independent consumers need the same data, or when sustained volume is very high. Typical cases are analytics pipelines, audit trails and event-sourced systems. If the need is mainly to hand tasks to workers, RabbitMQ is usually simpler.

Can RabbitMQ replace Kafka now that it has Streams?

For some replay use cases, yes. RabbitMQ Streams provide an append-only, re-readable log, and the RabbitMQ documentation places their throughput and latency in the same ballpark as Kafka. Kafka remains stronger where very large retention, tiered storage to object storage and a wide ecosystem of stream-processing tools are required.

Is Kafka faster than RabbitMQ?

Not universally. Kafka generally sustains higher throughput under heavy load, while RabbitMQ delivers very low latency at moderate load, and published studies show each ahead in different scenarios depending on message size and settings. A test with realistic messages and durability configuration gives the only answer that applies to a given system.

Can both be used in the same architecture?

Yes, and it is common. Kafka usually acts as the durable event backbone, while RabbitMQ distributes operational tasks such as emails and webhooks. The transactional outbox pattern keeps the two consistent with the database.

Sources

This article was produced with AI assistance and reviewed for accuracy. It is provided for general information only and is not professional advice.

Tell us about your project