Understanding Apache Kafka Architecture
RabbitMQ - Table Of ContentsWhat is Apache Kafka Understanding Apache Kafka Architecture Internal Working Of Apache Kafka Getting Started with Apache Kafka - Hello World Example Spring Boot + Apache Kafka Example
VideoThis tutorial is explained in the below Youtube Video.
Apache Kafka TopicApache Kafka is a messaging system where messages are sent by producers and these messages are consumed by one or more consumers. Producers send the messages to Apache Kafka Topics. From the topics these messages are then consumed by the consumers. Topics have unique names which are used by producers and consumers for sending/consuming messages.
Topics are the base abstraction of where data lives within Kafka. They can be considered similar to the concept of table in a database. Each topic is backed by logs which are partitioned and distributed.
Apache Kafka BrokerThe physical/virtual machines or servers where topics reside are called brokers. Kafka Broker is a software process that runs on machine. Broker has access to resources such as file systems where logs are maintained. A single topic can have multiple partitions running on different brokers.
We can add more brokers to scale the Kafka Application.
The advantages of using Apache Kafka Cluster are as follows -
- Clustering - Apache Kafka has a clustered set of brokers that work in unison.
- Distributed - The data in Apache Kafka is distributed in all the brokers. This is done by making use of partitions which are distributed across multiple brokers.
- Fault Tolerant - Apache Kafka maintains replicated copies of data. So if any broker in the cluster goes down, it does not affect the working of Apache Kafka Cluster. This is done by setting the replication value to greater than 1.
- Application scaling - Apache Kafka can be scaled horizontally. This increases the throughput.