Hi All,
Greetings of the day!
Today we will explore how to skip records on certain exceptions so that the entire batch job does not fail and the batch jobs process other valid records. Also, we will explore No Skip in the spring batch
Below is the log when the batch job fails and you have not configured the skip logic
Job: [SimpleJob: [name=processEmployeeData]] completed with the following parameters: [{run.id=13}] and the following status: [FAILED] in 70ms
So there are two approaches you can use to implement skip logic in Spring Batch.
- Use skip limit
- Use skip policy
Before we start let's see input what will be valid data to avoid any exception
Employee-Id, Employee-Name, Employee-Salary, Employee-Department
1,Rajesh,20000,Sales
2,Praktik,19899,Marketing
Now suppose you pass in a string for the employee id and unfortunately your code doesn't handle this exception and throws a FlatFileParseException. So let's see how the FlatFileParseException is skipped and prevents the batch job from failing.
How to configure skip logic
- Using Skip Limit
You need to add fault tolerance and specify exceptions to skip and a skip limit i.e. Number of times to skip a specific exception
Example
@Bean public Step processEmployee() { return stepBuilderFactory.get("processEmployee") .<Employee,Employee>chunk(1) .reader(readCSVFile()) .processor(employeeProcessor()) .writer(employeeWrite()).faultTolerant(). skip(FlatFileParseException.class).skipLimit(1) .build(); }
The configuration above is fault-tolerant for a FlatFileParseException to skip only one record. So if a FlatFileParseException occurs multiple times while reading, processing, or writing a file, the batch job will fail with the following log:
SkipLimitExceededException: Skip limit of '1' exceeded
Job: [SimpleJob: [name=processEmployeeData]] completed with the following parameters: [{run.id=14}] and the following status: [FAILED] in 92ms
- Using skip policy
You can configure a custom skip policy to skip records. You must create a class that implements SkipPolicy and override the following methods of the SkipPolicy interface.
public boolean shouldSkip(Throwable throwable, int i) throws SkipLimitExceededException
Let's see how to implement a SkipPolicy for FlatFileParseException with at most 1 record.
package com.employee.mgt.util; import org.springframework.batch.core.step.skip .SkipLimitExceededException; import org.springframework.batch.core.step.skip.SkipPolicy; import org.springframework.batch.item.file .FlatFileParseException; import org.springframework.stereotype.Component; @Component public class RecordSkipPolicy implements SkipPolicy { @Override public boolean shouldSkip(Throwable throwable, int i) throws SkipLimitExceededException { if(throwable instanceof FlatFileParseException && i==0){ return true; } return false; } }
In the code above you can write multiple if conditions for different exceptions. int i gives the count of the specific exception.
Next, we need to specify a skip policy on the processEmployee step that reads, processes, and writes the file.
@Bean public Step processEmployee() { return stepBuilderFactory.get("processEmployee") .<Employee,Employee>chunk(1) .reader(readCSVFile()) .processor(employeeProcessor()) .writer(employeeWrite()) .faultTolerant(). skipPolicy(recordSkipPolicy) .build(); }
You can also specify noSkip in the spring batch. Here we tell the spring batch to fail whenever a specific exception occurs.
@Bean public Step processEmployee() { return stepBuilderFactory.get("processEmployee") .<Employee,Employee>chunk(1) .reader(readCSVFile()) .processor(employeeProcessor()) .writer(employeeWrite()). faultTolerant(). skipPolicy(recordSkipPolicy). noSkip(NumberFormatException.class) .build(); }
Since we are specifying no skip for NumberFormatException, here, the batch job will fail as soon as this exception occurs.
You've learned how to implement skip logic using skip limit and skip policy along with no skip concept
If you have any questions, feel free to leave a comment.
Thanks!
Happy Learning.
Below is a post that you are welcome to refer to!
- Conditional Flow + Decider In Spring Batch + JobExecutionDecider + Spring Boot
- Spring Batch Tasklet With Spring Boot + How to create Tasklet in Spring Batch
- Spring Batch Job Listener + beforeJob +& afterJob In Spring Batch & Spring Boot+ JobExecutionListener
- Spring Batch Example Database To CSV Export
- Reading And Writing Multiple Files In Spring Batch Using MultiResourceItemReader & ItemReader
- Spring Cloud Bus-Refresh Config Changes With RabbitMQ
- Spring Microservices Overview
- Spring Microservices - Netflix Eureka Service Registration and Discovery
- Java 8
0 Comments
If you have any doubts let me know.