Internal Working Of Apache Kafka

In previous tutorial we saw architecture of Apache Kafka. In this tutorial we will be looking at internal working of Apache Kafka.

RabbitMQ - Table Of Contents
What is Apache Kafka
Understanding Apache Kafka Architecture
Internal Working Of Apache Kafka
Getting Started with Apache Kafka - Hello World Example
Spring Boot + Apache Kafka Example

Topics are the base abstraction of where data lives within Kafka. They can be considered similar to the concept of table in a database. Each topic is backed by logs which are partitioned and distributed.

Each Message has a timestamp Identifier Binary Payload

Message offset is last read message position. Consumer is responsible of keeping track of message consumed. Consumer keeps track of the successfully processed message. apache kafka retains all published messages. The default is 168

Each Topic has one or more partitions Partition allows - Scale Fault tolerant Achieve higher level of throughput

Each topic should have minimum 1 partition