Spring Boot Batch Chunk Processing Hello World Example
So previously we implemented a Spring Boot Batch Tasklet example to delete a file from a particular location. But now suppose we have more complex tasks such as processing multiple files and moving these files daily from one location to another. This task can obviously be done using Spring Batch Tasklet but it will not be optimal code as we will need to write a lot of complex code in the tasklet. Also other features like parallel processing, chunk processing these we would need to implement ourselves inside the tasklet.
In case of such complex tasks which can be broken into various subtasks like reading, processing and writing, it is optimal to make use of Spring Batch Chunk Processing. Spring Batch Chunk Processing provides various specialized classes to implement the subtasks. To process this data we can either process one unit of data at a time or we can process a group of data i.e. chunk at a time. In this example we will be processing a single record/unit at a time. So in this example we will keep the chunk size as 1. In the next tutorial we will have a look at the chunk size in detail.
Implementation
The spring boot project we will be implementing is as follows -
Spring Batch chunk processing provides three key interfaces to help perform bulk reading, processing and writing- ItemReader, ItemProcessor, and ItemWriter.
Go to Spring Initilizr to create the spring boot project.
The maven project will be creating is as follows -
The pom.xml will be as follows-
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>3.0.2</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.javainuse</groupId> <artifactId>spring-batch-chunk-example</artifactId> <version>0.0.1-SNAPSHOT</version> <name>spring-batch-chunk-example</name> <description>Demo project for Spring Boot</description> <properties> <java.version>17</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-test</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>Modify the SpringBatchChunkExampleApplication which is the bootstrap class by adding the EnableBatchProcessing annotation. This enables Spring Batch features and provide a base configuration for setting up batch jobs in @Configuration class.
package com.javainuse.springbatchchunkexample;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@EnableBatchProcessing
@SpringBootApplication
public class SpringBatchChunkExampleApplication {
public static void main(String[] args) {
SpringApplication.run(SpringBatchChunkExampleApplication.class, args);
}
}
We will first be creating a custom Reader class by implementing ItemReader<T> interface.
An Item Reader reads data into the spring batch application from the specified input source.
Calling it returns one item or null if no more items are left. For this example we will be reading the input from the provided string array.
package com.example.bootbatchtaskhello;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
public class Reader implements ItemReader<String> {
private String[] messages = { "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine", "Ten" };
private int count = 0;
@Override
public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
if (count < messages.length) {
return messages[count++];
} else {
count = 0;
}
return null;
}
}
Next we will be creating custom processor class by implementing the ItemProcessor interface. The item processor transforms read items before sending them to the writer.
package com.example.bootbatchtaskhello;
import org.springframework.batch.item.ItemProcessor;
public class Processor implements ItemProcessor<String, String> {
@Override
public String process(String data) throws Exception {
return data.toUpperCase();
}
}
Next we will be creating custom writer class by implementing the ItemWriter interface.
ItemWriter is a interface for generic output operations. Using it we can write data to a file or stream.
package com.example.bootbatchtaskhello;
import java.util.List;
import org.springframework.batch.item.ItemWriter;
public class Writer implements ItemWriter<String> {
@Override
public void write(List<? extends String> messages) throws Exception {
for (String msg : messages) {
System.out.println("Writing the data " + msg);
}
}
}
Next we will be creating the spring batch configuration class. In this class we autowire the JobBuilderFactory and StepBuilderFactory using which
we create the spring batch job and spring batch step.
package com.example.bootbatchtaskhello;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class BatchConfig {
@Autowired
private JobBuilderFactory jobFactory;
@Autowired
private StepBuilderFactory stepFactory;
@Bean
public Step hellowWorldStep() {
return stepFactory.get("readJsonStep").<String, String>chunk(1).reader(new Reader()).processor(new Processor())
.writer(new Writer()).build();
}
@Bean
public Job helloWorldJob() {
return jobFactory.get("helloworld").flow(hellowWorldStep()).end().build();
}
}
Next we will be creating the controller class. In this class we expose a GET Rest API. When this GET call is made using the spring batch job launcher we trigger
the spring batch job named helloWorldJob which we have defined in the configuration before.
package com.example.bootbatchtaskhello;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class JobInvokerController {
@Autowired
JobLauncher jobLauncher;
@Autowired
Job helloWorldJob;
@RequestMapping("/invokejob")
public String handle() throws Exception {
JobParameters jobParameters = new JobParametersBuilder().toJobParameters();
jobLauncher.run(helloWorldJob, jobParameters);
return "Task Batch job has been invoked";
}
}
Finally in the application.properties file define the spring batch and h2 db configuration as follows -
spring.datasource.url=jdbc:h2:file:./DB spring.jpa.properties.hibernate.hbm2ddl.auto=update spring.batch.job.enabled=false spring.batch.initialize-schema=always spring.datasource.url=jdbc:h2:mem:testdbStart the spring boot application. If we now go to localhost:8080/invokejob, the spring batch job will be started.
The spring batch process gets executed. If we now go to the url - localhost:8080/h2-console/login.do. In the JDBC url use jdbc:h2:mem:testdb. Keep the password blank.Click on Connect. We can see the tables created by spring batch
