目录
ItemReader
读平面文件
案例1
案例2
转视频版
接着上篇:Spring Batch 作业重启,了解Spring Batch 作业重启方式后,接下来一起学习一下Spring Batch 读入组件,了解一下Spring Batch如何读取平面文件数据的。
ItemReader
居于块操作的步骤由一个ItemReader,一个ItemProcessor和一个ItemWriter组成,一个负责读取数据,一个负责处理数据,一个负责输出数据,上一章节讲完步骤,接下来就重点讲解Spring Batch 输入组件:ItemReader
ItemReader 是Spring Batch 提供的输入组件,规范接口是ItemReader<T>, 里面有个read() 方法,我们可以实现该接口去定制输入逻辑。
public interface ItemReader<T> {
@Nullable
T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException;
}
Spring Batch 根据常用的输入类型,提供许多默认的实现,包括:平面文件、数据库、JMS资源和其他输入源等,接下来一起操作一下比较场景的输入场景。
读平面文件
平面文件一般指的都是简单行/多行结构的纯文本文件,比如记事本记录文件。与xml这种区别在于没有结构,没有标签的限制。Spring Batch默认使用 FlatFileItemReader 实现平面文件的输入。接下来以案例的形式来讲解 FlatFileItemReader 类的使用。
案例1
需求:读取user.txt文件,解析出所有用户信息
user.txt , 放置在资源文件中(classpath路径)
1#dafei#18
2#xiaofei#16
3#laofei#20
4#zhongfei#19
5#feifei#15
实体类
@Getter
@Setter
@ToString
public class User {
private Long id;
private String name;
private int age;
}
实现作业
package com.langfeiyes.batch._21_itemreader_flat;
import com.langfeiyes.batch._20_job_restart_allow.JobAllowRestartJob;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import java.util.List;
@SpringBootApplication
@EnableBatchProcessing
public class FlatReaderJob {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public FlatFileItemReader<User> userItemReader(){
return new FlatFileItemReaderBuilder<User>()
.name("userItemReader")
.resource(new ClassPathResource("users.txt"))
.delimited().delimiter("#")
.names("id", "name", "age")
.targetType(User.class)
.build();
}
@Bean
public ItemWriter<User> itemWriter(){
return new ItemWriter<User>() {
@Override
public void write(List<? extends User> items) throws Exception {
items.forEach(System.err::println);
}
};
}
@Bean
public Step step(){
return stepBuilderFactory.get("step1")
.<User, User>chunk(1)
.reader(userItemReader())
.writer(itemWriter())
.build();
}
@Bean
public Job job(){
return jobBuilderFactory.get("flat-reader-job")
.start(step())
.build();
}
public static void main(String[] args) {
SpringApplication.run(FlatReaderJob.class, args);
}
}
核心在userItemReader() 实例方法
//FlatFileItemReader spring batch 平面文件读入类
//这个类操作特点:一行一行的读数据
@Bean
public FlatFileItemReader<User> userItemReader(){
return new FlatFileItemReaderBuilder<User>()
.name("userItemReader")
.resource(new ClassPathResource("users.txt")) //指定读取的文件
.delimited().delimiter("#") //读取出一行数据,该如何分割数据,默认以,分割,当前使用#号分割
.targetType(User.class) //读取出一行数据封装成什么对象
//给分割后数据打name标记,后续跟User对象属性进行映射
.names("id", "name", "age")
.build();
}
案例2
需求:读取user2.txt文件,解析出所有用户信息
user2.txt
1#dafei#18#广东#广州#天河区
2#xiaofei#16#四川#成都#武侯区
3#laofei#20#广西#桂林#雁山区
4#zhongfei#19#广东#广州#白云区
5#feifei#15#广东#广州#越秀区
很明显,数据结果更加复杂了。
用户对象
@Getter
@Setter
@ToString
public class User {
private Long id;
private String name;
private int age;
private String address;
}
观察,user2.txt文件中有 id name age province city area 按理用户对象属性应该一一对应,但是此时User只有address,也就是说,后续要将 province , city , area 合并成 address 地址值。此时怎么办?这是就需要自定义FieldSetMapper 啦。
public class UserFieldMapper implements FieldSetMapper<User> {
@Override
public User mapFieldSet(FieldSet fieldSet) throws BindException {
//自己定义映射逻辑
User User = new User();
User.setId(fieldSet.readLong("id"));
User.setAge(fieldSet.readInt("age"));
User.setName(fieldSet.readString("name"));
String addr = fieldSet.readString("province") + " "
+ fieldSet.readString("city") + " " + fieldSet.readString("area");
User.setAddress(addr);
return User;
}
}
上面代码实现FieldSet与User对象映射,将province city area 合并成一个属性address。另外readXxx 是FieldSet 独有的方法,Xxx是java基本类型。
package com.langfeiyes.batch._22_itemreader_flat_mapper;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.io.ClassPathResource;
import java.util.List;
@SpringBootApplication
@EnableBatchProcessing
public class MapperFlatReaderJob {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public UserFieldMapper userFieldMapper(){
return new UserFieldMapper();
}
@Bean
public FlatFileItemReader<User> userItemReader(){
return new FlatFileItemReaderBuilder<User>()
.name("userMapperItemReader")
.resource(new ClassPathResource("users2.txt"))
.delimited().delimiter("#")
.names("id", "name", "age", "province", "city", "area")
.fieldSetMapper(userFieldMapper())
.build();
}
@Bean
public ItemWriter<User> itemWriter(){
return new ItemWriter<User>() {
@Override
public void write(List<? extends User> items) throws Exception {
items.forEach(System.err::println);
}
};
}
@Bean
public Step step(){
return stepBuilderFactory.get("step1")
.<User, User>chunk(1)
.reader(userItemReader())
.writer(itemWriter())
.build();
}
@Bean
public Job job(){
return jobBuilderFactory.get("mapper-flat-reader-job")
.start(step())
.build();
}
public static void main(String[] args) {
SpringApplication.run(MapperFlatReaderJob.class, args);
}
}
上面代码核心在userItemReader实例方法
.fieldSetMapper(userFieldMapper()) : 用上自定义的字段映射器
.names("id", "name", "age", "province", "city", "area") : users2.txt 每一行使用#分割出现6列,给每一列取名字,然后将其封装到FieldSet对象中
.targetType(User.class) : 注意,使用了fieldSetMapper 之后,不需要在加上这行
到这,本篇就结束了,欲知后事如何,请听下回分解~