Search Tutorials


Apache Camel + Kafka Hello World Example | JavaInUse

Apache Camel + Kafka Hello World Example

In this post we will integrate Apache Camel and Apache Kafka instance.
In a previous post we had seen how to get Apache Kafka up and running.

Apache Camel - Table of Contents

File Transfer Using Java DSL Apache Camel Apache Camel Java DSL + Spring Integration Hello World Example Apache Camel Exception Handling Using Simple Example Apache Camel Redelivery policy using example Integrate Apache Camel and ActiveMQ EIP patterns using Apache Camel Apache Camel Tutorial- Integrate Spring Boot+ Apache Camel Apache Camel Tutorial- Integrate with MySQL DB using SQL query Apache Camel + Spring + ActiveMQ + JBoss Fuse Apache Camel EIP - Splitter and Aggregator pattern Apache Camel Unit Testing Apache Camel + Spring + Quartz Hello World Example Camel application deployment on JBoss Fuse Apache Camel + Apache CXF SOAP Webservices Apache Camel + JAX-RS REST Webservice Apache Camel + CXFRS REST Webservice Apache Camel Routing Slip EIP Pattern Apache Camel Dynamic Router Pattern Apache Camel Load Balancer EIP Pattern Apache Camel Interceptors Apache Camel + Kafka Hello World Example Apache Camel - Marshalling/Unmarshalling XML/JSON Data Example Calling and Consuming Webservices using Apache Camel Apache Camel Tutorial - Send SMTP Email Using Gmail Apache Camel Tutorial - SEDA component Hello World Example Spring Boot + Apache Camel + RabbitMQ - Hello World Example Apache Camel Tutorial - Idempotent Consumer using MemoryIdempotentRepository and FileIdempotentRepository Spring Boot + Apache Camel JDBC component + MySQL - Hello World Example Spring Boot + Apache Camel SQL component + MySQL - Hello World Example Spring Boot + Apache Camel SQL component + Transaction Management Example

Video

This tutorial is explained in the below Youtube Video.

Lets Begin

We will create Eclipse maven project as follows-

Apache Camel Kafka Integration Tutorial
Our pom file will be as follows-
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.javainuse</groupId>
	<artifactId>camel-kafka-dsl</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<dependencies>
		<dependency>
			<groupId>org.apache.camel</groupId>
			<artifactId>camel-core</artifactId>
			<version>2.13.0</version>
		</dependency>
		
		<dependency>
			<groupId>org.apache.camel</groupId>
			<artifactId>camel-kafka</artifactId>
			<version>2.16.3</version>
		</dependency>
		
		<dependency>
			<groupId>org.scala-lang</groupId>
			<artifactId>scala-library</artifactId>
			<version>2.11.0</version>
		</dependency>
	</dependencies>
</project>
Define the route.
package com.javainuse;

import org.apache.camel.builder.RouteBuilder;

public class SimpleRouteBuilder extends RouteBuilder {

	@Override
	public void configure() throws Exception {

		String topicName = "topic=javainuse-topic";
		String kafkaServer = "kafka:localhost:9092";
		String zooKeeperHost = "zookeeperHost=localhost&zookeeperPort=2181";
		String serializerClass = "serializerClass=kafka.serializer.StringEncoder";

		String toKafka = new StringBuilder().append(kafkaServer).append("?").append(topicName).append("&")
				.append(zooKeeperHost).append("&").append(serializerClass).toString();

		from("file:C:/inbox?noop=true").split().tokenize("\n").to(toKafka);
	}
}
When Camel is started, it creates a CamelContext object that contains the definition of the Route to be started. Below we create default camel context and load the routes created in SimpleRouteBuilder.
package com.javainuse;

import org.apache.camel.CamelContext;
import org.apache.camel.impl.DefaultCamelContext;

public class MainApp {

    public static void main(String[] args) {
        SimpleRouteBuilder routeBuilder = new SimpleRouteBuilder();
        CamelContext ctx = new DefaultCamelContext();
        try {
            ctx.addRoutes(routeBuilder);
            ctx.start();
            Thread.sleep(5 * 60 * 1000);
            ctx.stop();
        }
        catch (Exception e) {
            e.printStackTrace();
        }

    }
}



We are done with the required Java code. Now lets start Apache Kafka. As we had explained in detail in the Getting started with Apache Kafka perform the following.
  • Start Apache Zookeper-
    C:\kafka_2.12-0.10.2.1>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
    


    boot21_3
  • Start Apache Kafka-
    C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-server-start.bat .\config\server.properties
    

    boot21_4
  • Open a new command prompt and create a topic with name javainuse-topic, that has only one partition & one replica.
    C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic javainuse-topic
    

    boot21_5
  • Also Start the consumer listening to the javainuse-topic-
    C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic javainuse-topic --from-beginning
    
  • In C:/inbox folder we have a file with following content-
    learn 
    camel 
    using 
    javainuse
    
    Next start the Apache Camel Application by running it as a Java Application. The above file content will be sent to split line by line and sent to kafka.
    Apache Camel Kafka Integration Example

Download Source Code

Download it - Apache Camel Kafka Example

See Also

Spring Boot Hello World Application- Create simple controller and jsp view using Maven Spring Boot Tutorial-Spring Data JPA Spring Boot + Simple Security Configuration Pagination using Spring Boot Simple Example Spring Boot + ActiveMQ Hello world Example Spring Boot + Swagger Example Hello World Example Spring Boot + Swagger- Understanding the various Swagger Annotations Spring Boot Main Menu Spring Boot Interview Questions