REAL STORIES
BY REAL PEOPLE Search
Monday, November 20, 2017

Apache Kafka Versus RabbitMQ: Which One Should You Opt For?

by Kaushal Shah (writer), , April 18, 2017

This article looks at the comparative merits of two popular message brokers Kafka and RabbitMQ - highlighting the pros and cons for those considering a Message Oriented Middleware.

Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. The middleware is also known as message broker or message queue. There are many popular message brokers that provide an asynchronous communication protocol. Asynchronous meaning the sender and receiver are completely detached and don’t need to interact with the queueing system simultaneously.

The software is therefore very useful for:

  • Decoupling of services
  • Painless scaling
  • Backpressure management
  • Operational visibility

Apache Kafka and RabbitMQ are two of the most popular message brokers and often the question asked by those considering a MOM is ‘Kafka or RabbitMQ?”. Since there is no easy way to answer this, let’s analyze the strengths of each and also the differences.

RabbitMQ

RabbitMQ is a popular message broker with many powerful features. Their website features excellent documentation and there are also many books on the subject. It is written in Erlang (not a well-known programming language but perfect for the task). RabbitMQ has been developed and maintained by Pivotal. It is easy to use and supports a number of development platforms. In short, it is a robust, mature, general purpose message broker that supports many standard protocols such as AMQP.

Features:

1. Reliability

RabbitMQ has features that let you trade off performance with reliability, including persistence, delivery acknowledgements, publisher confirms, and high availability.

2. Flexible Routing

Messages are routed through exchanges before landing at queues. RabbitMQ has many built-in exchange types for typical routing logic. For more complex routing, combine exchanges together or write your own exchange type as a plugin.

3. Clustering

Multiple RabbitMQ servers on the same local network can be grouped together to form a single logical broker.

4. Federation

RabbitMQ provides a federation model for servers that need to be more loosely coupled and unreliably connected than clustering allows.

5. Highly Available Queues

It has capability to mirror queues across several machines in a cluster to ensure that messages are secure in the event of a hardware failure.

6. Multi-protocol

It supports messaging over a variety of messaging protocols.

Additionally, there are many other features like supporting multiple clients, simpler management UI, tracing support for messages, and a variety of plugins that are extensible. RabbitMQ can work well for “fast” consumers, if overall throughput is enough for your requirements.

Kafka

Kafka is a distributed streaming platform. It is used for real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, fast, and runs in production in thousands of companies. Think of it as a message bus optimized for high-ingress data streams and replay.

Features

1. Kafka lets you publish and subscribe to streams of records. Here it behaves similar to a message queue or enterprise messaging system.

2. It allows you to store streams of records in a fault-tolerant way.

3. It lets you process streams of records as they occur.

4. Kafka is excellent for building real-time streaming data pipelines that reliably get data between systems or applications. It is also good for building real-time streaming applications that transform or react to the streams of data

5. Kafka is a highly scalable, robust, high performance and a durable messaging system.

Differences between the two:

RabbitMQ:

  • RabbitMQ is more solid, mature and a general purpose message broker that supports several standardized protocols such as AMQP
  • High-throughput and reliable background jobs, communication and integration within, and between applications.
  • Open Source: Mozilla Public License
  • Erlang: Programming Language
  • Many mature libraries including: Ruby, Python, Node.js, Clojure, Go, Java and C

Apache Kafka:

  • Apache Kafka is a message bus optimized for high-ingress data streams and replay
  • Distributed streaming platform used for real-time data pipelines. Build applications that process and re-process streamed data on disk
  • Open Source: Apache License 2.0
  • Scala (JVM): Programming Language
  • Many including: Ruby, Python, Node.js and Java

RabbitMQ or Kafka?

The answer to this question is ‘it depends’. It depends on the use-case and specific message broking requirements. Message brokers/MOM are a very broad and complex topic, and we have just scratched the surface. However, if you have to take a call on which message broking system to use, here is a handy tip:

  • Kafka would work better for “fast” and reliable consumers
  • RabbitMQ might be the answer for “slow” and unreliable consumers

Use your own discretion and research before you take the call as no two requirements are alike. If you would like to customize your message distribution system then you can use the services of a specialized Java enterprise development company like Rishabh Software. We would like to know your views on message brokers and whether you find one favorable over the other. Please let us know in the comments section below.



About the Writer

Kaushal Shah is a writer for BrooWaha. For more information, visit the writer's website.
Want to write articles too? Sign up & become a writer!

0 comments on Apache Kafka Versus RabbitMQ: Which One Should You Opt For?



Add A Comment!

Click here to signup or login.


Rate This Article


Your vote matters to us



x


x