Hi All,

Greetings of the day!

Today we will explore how to skip records on certain exceptions so that the entire batch job does not fail and the batch jobs process other valid records. Also, we will explore No Skip in the spring batch

Below is the log when the batch job fails and you have not configured the skip logic

Job: [SimpleJob: [name=processEmployeeData]] completed with the following parameters: [{run.id=13}] and the following status: [FAILED] in 70ms

So there are two approaches you can use to implement skip logic in Spring Batch.

  • Use skip limit
  • Use skip policy

Before we start let's see input what will be valid data to avoid any exception

Employee-Id, Employee-Name, Employee-Salary, Employee-Department
1,Rajesh,20000,Sales
2,Praktik,19899,Marketing

Now suppose you pass in a string for the employee id and unfortunately your code doesn't handle this exception and throws a FlatFileParseException. So let's see how the FlatFileParseException is skipped and prevents the batch job from failing.

How to configure skip logic
  • Using Skip Limit
You need to add fault tolerance and specify exceptions to skip and a skip limit i.e. Number of times to skip a specific exception

Example
  
@Bean
public Step processEmployee() {
	return stepBuilderFactory.get("processEmployee")
		.<Employee,Employee>chunk(1)
		.reader(readCSVFile())
		.processor(employeeProcessor())
		.writer(employeeWrite()).faultTolerant().
		skip(FlatFileParseException.class).skipLimit(1)
		.build();
}

The configuration above is fault-tolerant for a FlatFileParseException to skip only one record. So if a FlatFileParseException occurs multiple times while reading, processing, or writing a file, the batch job will fail with the following log:

SkipLimitExceededException: Skip limit of '1' exceeded
Job: [SimpleJob: [name=processEmployeeData]] completed with the following parameters: [{run.id=14}] and the following status: [FAILED] in 92ms

  • Using skip policy
You can configure a custom skip policy to skip records. You must create a class that implements SkipPolicy and override the following methods of the SkipPolicy interface.

public boolean shouldSkip(Throwable throwable, int i) throws SkipLimitExceededException 

Let's see how to implement a SkipPolicy for FlatFileParseException with at most 1 record.
package com.employee.mgt.util;

import org.springframework.batch.core.step.skip
		.SkipLimitExceededException;
import org.springframework.batch.core.step.skip.SkipPolicy;
import org.springframework.batch.item.file
		.FlatFileParseException;
import org.springframework.stereotype.Component;

@Component
public class RecordSkipPolicy implements SkipPolicy {

	@Override
	public boolean shouldSkip(Throwable throwable, int i)
			throws SkipLimitExceededException {
		if(throwable instanceof FlatFileParseException
				&& i==0){
			return true;
		}
		return false;
	}
}

In the code above you can write multiple if conditions for different exceptions. int i gives the count of the specific exception.
Next, we need to specify a skip policy on the processEmployee step that reads, processes, and writes the file.
@Bean
public Step processEmployee() {
	return stepBuilderFactory.get("processEmployee")
			.<Employee,Employee>chunk(1)
			.reader(readCSVFile())
			.processor(employeeProcessor())
			.writer(employeeWrite())
            .faultTolerant().
			skipPolicy(recordSkipPolicy)
			.build();
}

You can also specify noSkip in the spring batch. Here we tell the spring batch to fail whenever a specific exception occurs.

@Bean
public Step processEmployee() {
	return stepBuilderFactory.get("processEmployee")
			.<Employee,Employee>chunk(1)
			.reader(readCSVFile())
			.processor(employeeProcessor())
			.writer(employeeWrite()).
			faultTolerant().
			skipPolicy(recordSkipPolicy).
			noSkip(NumberFormatException.class)
			.build();
}
 
Since we are specifying no skip for NumberFormatException, here, the batch job will fail as soon as this exception occurs.

You've learned how to implement skip logic using skip limit and skip policy along with no skip concept

If you have any questions, feel free to leave a comment.

Thanks!
Happy Learning.

Below is a post that you are welcome to refer to!