Search Tutorials


Spring Boot + Spring Batch Job Parameters Example | JavaInUse

Spring Boot + Spring Batch Job Parameters Example

Overview

In enterprise application batch job usually consists of multiple steps. During each step we may perform the following operations -
  • read input data
  • process input data
  • write the processed data
Also if a batch job has multiple steps then the output of one step is many times used as input of the next step and so on.
Spring Batch Job Multiple Steps
One of the features of Spring Batch is that is provides resusable components. These components can be used to process large valumes of data and perform operations like JobRestart, Skip, Job processing metadata, Logging and tracing and Transaction Management. What are Job Parameters Job parameters are key-value-pairs of data you provide when you start a job. A job has different job instances and these job instances differ on the basis of job parameters. JobParameters can be used for identification during the job run. They have reserved names, so to access them we can use Spring Expression Language
For example we can provide the following as job parameters-
  • File path of the source location containing the files to be processed by String Batch
  • The path of the target location where the processed files should be written by Spring Batch

Spring Batch JobParameters

Lets Begin

We will be creating a Spring batch job to transfer files from the source location to the destination location. The source and the destination location we will be providing using job parameters.
Spring Batch job parameters Hello world example
The maven project structure we will be developing is as follows -
Spring Batch Maven
The pom.xml will be as follows-
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.javainuse</groupId>
	<artifactId>spring-batch-jobparameter</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>
	<name>jobParameters</name>
	<description>Demo project for Spring Boot</description>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.1.1.RELEASE</version>
		<relativePath /> <!-- lookup parent from repository -->
	</parent>
	
	<properties>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-batch</artifactId>
		</dependency>
		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
			<scope>runtime</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
</project>

	

Job Configuration

This is the job configuration class where we are creating the necessary beans to perform the batch job.
  • JobBuilderFactory - Convenient factory for a JobBuilder which sets the JobRepository automatically. We will be autowiring this bean provided by Spring Batch.
  • StepBuilderFactory - Convenient factory for a StepBuilder which sets the JobRepository and PlatformTransactionManager automatically. We will be autowiring this bean provided by Spring Batch.
  • Job - Batch domain object representing a job. A job execution may consist of multiple steps which need to be excuted. We are executing a single step named moveFilesStep. Also we start the job using the autowired JobBuilderFactory.
  • Step - Batch domain interface representing the configuration of a step. In this step we call the moveFilesTasklet to execute the task of moving the files from source to destination directory.
  • Tasklet - The Tasklet is a simple interface with just one method execute. Using this we can perform single tasks like executing queries, deleting files. In our case we will be moving the files from source to destination directory.
package com.javainuse.configuration;

import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;

@Configuration
public class JobConfiguration {

	@Autowired
	private JobBuilderFactory jobBuilderFactory;

	@Autowired
	private StepBuilderFactory stepBuilderFactory;

	@Bean
	@StepScope
	public Tasklet moveFilesTasklet(@Value("#{jobParameters['sourceDir']}") String sourceDirLocation,
			@Value("#{jobParameters['destinationDir']}") String destinationDirLocation) {
		return (stepContribution, chukContext) -> {

			Path sourceDir = Paths.get(sourceDirLocation);
			Path destinationDir = Paths.get(destinationDirLocation);

			try (DirectoryStream<Path> directoryStream = Files.newDirectoryStream(sourceDir)) {
				for (Path path : directoryStream) {
					System.out.println("copying " + path.toString());
					Path d2 = destinationDir.resolve(path.getFileName());
					System.out.println("destination File=" + d2);
					Files.move(path, d2, REPLACE_EXISTING);
				}
			} catch (IOException ex) {
				ex.printStackTrace();
			}
			return RepeatStatus.FINISHED;
		};
	}

	@Bean
	public Step moveFilesStep() {
		return stepBuilderFactory.get("moveFilesStep").tasklet(moveFilesTasklet(null, null)).build();
	}

	@Bean
	public Job jobParametersJob() {
		return jobBuilderFactory.get("jobParametersJob").start(moveFilesStep()).build();
	}
}

MainApp

This is Spring Boot main project. Just run as a spring boot project
  • JobLauncher - Simple interface for controlling jobs, including possible ad-hoc executions, based on different runtime identifiers. It is extremely important to note that this interface makes absolutely no guarantees about whether or not calls to it are executed synchronously or asynchronously. The javadocs for specific implementations should be checked to ensure callers fully understand how the job will be run.
  • Job - In a Spring Batch application, a job is the batch process that is to be executed. It runs from start to finish without interruption.
In this class we will be creating the job parameters and then passing these job parameters along with the job to the job launcher.
package com.javainuse;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@EnableBatchProcessing
@SpringBootApplication
public class JobParametersApplication implements CommandLineRunner {

	@Autowired
	private JobLauncher jobLauncher;

	@Autowired
	private Job job;

	public static void main(String[] args) {
		SpringApplication.run(JobParametersApplication.class, args);
	}

	@Override
	public void run(String... args) throws Exception {

		// Pass the required Job Parameters from here to read it anywhere within
		// Spring Batch infrastructure
		JobParameters jobParameters = new JobParametersBuilder().addString("sourceDir", "C://inputLocation")
				.addString("destinationDir", "C://outputLocation").toJobParameters();

		JobExecution execution = jobLauncher.run(job, jobParameters);
		System.out.println("STATUS :: " + execution.getStatus());
	}
}
Also by default, Spring Boot executes any job in out application context at startup. So our tasklet will be executed twice: once at application startup and once when we invoke the API using the JobLauncher. On application startup when the tasklet is involved the job parameters are null and so our application will throw a null pointer exception.
We will be disabling running jobs at startup using following property -
spring.batch.job.enabled=false

Console output

After running the program, you'll see that "message" value has been read in the Tasklet here.

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.1.RELEASE)

2021-01-05 00:12:18.377  INFO 7764 --- [           main] com.javainuse.JobParametersApplication   : Starting JobParametersApplication on VAIO with PID 7764 (C:\codeusingjava-show\spring-batch-db\target\classes started by SONY in C:\codeusingjava-show\spring-batch-db)
2021-01-05 00:12:18.382  INFO 7764 --- [           main] com.javainuse.JobParametersApplication   : No active profile set, falling back to default profiles: default
2021-01-05 00:12:19.960  INFO 7764 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-01-05 00:12:20.397  INFO 7764 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-01-05 00:12:20.416  INFO 7764 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: H2
2021-01-05 00:12:20.649  INFO 7764 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
2021-01-05 00:12:21.005  INFO 7764 --- [           main] com.javainuse.JobParametersApplication   : Started JobParametersApplication in 3.134 seconds (JVM running for 3.653)
2021-01-05 00:12:21.116  INFO 7764 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=jobParametersJob]] launched with the following parameters: [{sourceDir=C://inputLocation, destinationDir=C://outputLocation}]
2021-01-05 00:12:21.152  INFO 7764 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
copying C:\inputLocation\hello.txt
destination File=C:\outputLocation\hello.txt
copying C:\inputLocation\hello2.txt
destination File=C:\outputLocation\hello2.txt
copying C:\inputLocation\hello3.txt
destination File=C:\outputLocation\hello3.txt
copying C:\inputLocation\hello4.txt
destination File=C:\outputLocation\hello4.txt
copying C:\inputLocation\hello5.txt
destination File=C:\outputLocation\hello5.txt
copying C:\inputLocation\hello6.txt
destination File=C:\outputLocation\hello6.txt
2021-01-05 00:12:21.260  INFO 7764 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=jobParametersJob]] completed with the following parameters: [{sourceDir=C://inputLocation, destinationDir=C://outputLocation}] and the following status: [COMPLETED]
STATUS :: COMPLETED
2021-01-05 00:12:21.285  INFO 7764 --- [       Thread-2] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-01-05 00:12:21.288  INFO 7764 --- [       Thread-2] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

Download Source Code

Download it - Spring Boot Batch Job Parameters example