Contents 12 sections
- How to use this guide
- Architecture diagrams
- What is Spring Batch?
- Spring Boot project setup
- Configuring Jobs & Steps
- Chunk processing (Reader-Processor-Writer)
- Types of ItemReader
- Types of ItemWriter
- Parallel processing & partitioning
- Scheduling & restart strategy
- Batch testing
- Frequently asked questions
How to use this guide
How to read itUnderstanding Spring Batch as a real-world workflow
Build large-scale data processing pipelines with Spring Batch 5 on Spring Boot. Design a practical batch system: the Job/Step/Chunk architecture, ItemReader/Writer, scheduling, parallel processing, and restart strategies. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspectiveBackend / systems development
Before syntax, pin down the boundaries a request crosses: validation, processing, storage, and response.
Large-scale ETL processingSettlement/aggregation batchesData migrationSpring Boot scheduling
Architecture diagrams
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning Spring Batch.
Learning flow
Architecture view
What is Spring Batch?
When you first open Spring Batch, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
Spring Batch is a lightweight batch framework, built on Spring Boot, for processing large volumes of records. It provides the Job β Step β Chunk hierarchy and restart, skip, and retry mechanisms out of the box.
Spring Boot project setup
Here you look at Spring Boot project setup alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The single spring-boot-starter-batch auto-configures the core components needed to run Jobs/Steps, such as JobRepository and JobLauncher. spring-batch-test lets you test at the Job and Step level without running the real batch, so always include it in the test dependencies.
plugins {
id("java")
id("org.springframework.boot") version "3.3.5"
id("io.spring.dependency-management") version "1.1.6"
}
java { toolchain { languageVersion.set(JavaLanguageVersion.of(21)) } }
dependencies {
implementation("org.springframework.boot:spring-boot-starter-batch")
implementation("org.springframework.boot:spring-boot-starter-data-jpa")
implementation("org.springframework.boot:spring-boot-starter-validation")
compileOnly("org.projectlombok:lombok")
annotationProcessor("org.projectlombok:lombok")
runtimeOnly("com.h2database:h2") // for development
runtimeOnly("org.postgresql:postgresql") // for production
testImplementation("org.springframework.boot:spring-boot-starter-test")
testImplementation("org.springframework.batch:spring-batch-test")
}
Configuring Jobs & Steps
Here you look at Configuring Jobs & Steps alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Declare @EnableBatchProcessing, have JobRepository and PlatformTransactionManager injected, and register the Job and Steps as beans.
@Configuration
@EnableBatchProcessing
@RequiredArgsConstructor
public class BatchConfig {
private final JobRepository jobRepository;
private final PlatformTransactionManager transactionManager;
private final UserItemReader userItemReader;
private final UserItemProcessor userItemProcessor;
private final UserItemWriter userItemWriter;
@Bean
public Job userMigrationJob() {
return new JobBuilder("userMigrationJob", jobRepository)
.start(userMigrationStep())
.build();
}
@Bean
public Step userMigrationStep() {
return new StepBuilder("userMigrationStep", jobRepository)
.<User, MigratedUser>chunk(100, transactionManager)
.reader(userItemReader)
.processor(userItemProcessor)
.writer(userItemWriter)
.faultTolerant()
.skipLimit(10)
.skip(Exception.class)
.build();
}
}
Chunk processing (Reader-Processor-Writer)
Here you look at Chunk processing (Reader-Processor-Writer) alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The Chunk style wraps the cycle of reading N items β processing them β writing them in a transaction. On failure only that chunk is rolled back. Because commits are split per chunk rather than wrapping all the data in one transaction, even if a job fails midway through millions of records, the earlier chunks already committed stay as they are and you can retry from the failed chunk.
@Component
public class UserItemProcessor implements ItemProcessor<User, MigratedUser> {
@Override
public MigratedUser process(User user) {
// When null is returned, the item is not passed to the Writer (filtering)
if (!user.isActive()) return null;
return MigratedUser.builder()
.originalId(user.getId())
.email(user.getEmail().toLowerCase())
.createdAt(LocalDateTime.now())
.build();
}
}
Types of ItemReader
Here you look at Types of ItemReader alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Spring Batch provides a variety of ItemReaders out of the box, for JPA, JDBC, CSV, XML, and more. JpaCursorItemReader reads with a cursor, streaming large volumes of data rather than loading it all into memory at once, which suits large migrations, and with setSaveState(true) on, the batch can restart from the last processed point even if it dies midway.
// ββ JpaCursorItemReader (recommended for large volumes) ββ
@Component
@StepScope
public class UserItemReader extends JpaCursorItemReader<User> {
public UserItemReader(EntityManagerFactory emf) {
setEntityManagerFactory(emf);
setQueryString("SELECT u FROM User u WHERE u.active = true ORDER BY u.id");
setSaveState(true); // resume from the point of interruption on restart
}
}
// ββ FlatFileItemReader (reading CSV) ββββββββββββββ
@Bean
@StepScope
public FlatFileItemReader<UserCsvRow> csvReader(
@Value("#{jobParameters['file']}") String file) {
return new FlatFileItemReaderBuilder<UserCsvRow>()
.name("csvReader")
.resource(new FileSystemResource(file))
.delimited().delimiter(",")
.names("id", "email", "name")
.targetType(UserCsvRow.class)
.linesToSkip(1) // skip the header
.build();
}
Types of ItemWriter
Here you look at Types of ItemWriter alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
JpaItemWriter merges through the EntityManager, and JdbcBatchItemWriter uses JDBC bulk INSERT/UPDATE.
// ββ JpaItemWriter βββββββββββββββββββββββββββββββββ
@Component
@RequiredArgsConstructor
public class UserItemWriter implements ItemWriter<MigratedUser> {
private final MigratedUserRepository repository;
@Override
public void write(Chunk<? extends MigratedUser> chunk) {
repository.saveAll(chunk.getItems());
}
}
// ββ JdbcBatchItemWriter (optimized for bulk performance) ββ
@Bean
public JdbcBatchItemWriter<MigratedUser> jdbcWriter(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<MigratedUser>()
.dataSource(dataSource)
.sql("INSERT INTO migrated_users (original_id, email, created_at) " +
"VALUES (:originalId, :email, :createdAt)")
.beanMapped()
.build();
}
Parallel processing & partitioning
Here you look at Parallel processing & partitioning alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Splitting the data with a Partitioner and processing it in parallel with a TaskExecutor greatly speeds up large batch jobs. The master Step only decides how to divide the data, and the worker Steps each process their assigned range independently, so simply raising gridSize scales throughput horizontally to match the number of CPU cores.
@Bean
public Step partitionedStep() {
return new StepBuilder("partitionedStep", jobRepository)
.partitioner("workerStep", new RangePartitioner(userRepository))
.step(workerStep())
.taskExecutor(new SimpleAsyncTaskExecutor())
.gridSize(4) // process 4 partitions in parallel
.build();
}
// Partitioner β splits by ID range
public class RangePartitioner implements Partitioner {
private final UserRepository repository;
@Override
public Map<String, ExecutionContext> partition(int gridSize) {
long min = repository.findMinId();
long max = repository.findMaxId();
long range = (max - min) / gridSize + 1;
Map<String, ExecutionContext> result = new HashMap<>();
for (int i = 0; i < gridSize; i++) {
ExecutionContext ctx = new ExecutionContext();
ctx.putLong("minId", min + i * range);
ctx.putLong("maxId", min + (i + 1) * range - 1);
result.put("partition" + i, ctx);
}
return result;
}
}
Scheduling & restart strategy
Here you look at Scheduling & restart strategy alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Set up periodic execution with @Scheduled and control the restart policy through allowStartIfComplete. Unless the JobParameters are filled with a different value each time (the run time and so on), Spring Batch judges it to be "the same Job, already completed" and refuses to run it again, so scheduled jobs must always include a parameter that differs on every run.
@Component
@RequiredArgsConstructor
public class BatchScheduler {
private final JobLauncher jobLauncher;
private final Job userMigrationJob;
@Scheduled(cron = "0 0 2 * * *") // every day at 2 AM
public void runBatch() throws Exception {
JobParameters params = new JobParametersBuilder()
.addLocalDateTime("runAt", LocalDateTime.now()) // run with a fresh parameter every time
.toJobParameters();
jobLauncher.run(userMigrationJob, params);
}
}
// application.yml
// spring.batch.job.enabled: false β prevents automatic execution at server startup
Batch testing
Here you look at Batch testing alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Have JobLauncherTestUtils injected with @SpringBatchTest and verify the whole Job or individual Steps. Running the whole Job is slow and makes the failure point hard to pin down, so running just one chosen Step with launchStep, to verify the behavior of the Reader/Processor/Writer narrowly, is more efficient for debugging.
@SpringBatchTest
@SpringBootTest
class UserMigrationBatchTest {
@Autowired JobLauncherTestUtils jobLauncherTestUtils;
@Autowired JobRepositoryTestUtils jobRepositoryTestUtils;
@Autowired MigratedUserRepository migratedUserRepository;
@BeforeEach
void setUp() {
jobRepositoryTestUtils.removeJobExecutions();
}
@Test
void fullBatchRun_succeeds() throws Exception {
JobExecution execution = jobLauncherTestUtils.launchJob();
assertThat(execution.getStatus()).isEqualTo(BatchStatus.COMPLETED);
}
@Test
void runsSingleStepOnly() throws Exception {
JobExecution execution = jobLauncherTestUtils.launchStep("userMigrationStep");
StepExecution step = execution.getStepExecutions().iterator().next();
assertThat(step.getStatus()).isEqualTo(BatchStatus.COMPLETED);
assertThat(step.getWriteCount()).isGreaterThan(0);
}
}
Spring Batch: frequently asked questions
When do I use Spring Batch?
Use it for jobs that read, process, and store large amounts of data at scheduled times. Settlement, statistics aggregation, and data migration are typical. It commits in units of a set number of items, restarts from the point of failure, and records execution history.
How do chunk-oriented steps and tasklets differ?
A chunk-oriented step repeats a cycle of reading and processing data, then writing and committing once a set number of items has accumulated. A tasklet simply runs one task once and is used for things like deleting files or calling a procedure. Processing a large number of records calls for chunks.
Does Spring Batch also do scheduling?
No. Spring Batch is a framework for defining and running jobs, and a separate scheduler decides when they run. Manage run times with a scheduler tool, a Kubernetes CronJob, or a workflow tool.