在上一篇文章 Spring Batch 之 背景框架简介 中,已经概述了Batch的基本架构组织,并且运行了简易demo。 在接下来的篇幅中,将逐步介绍每个组件的使用方式,并结合业务进行批处理。
一个job是如何诞生的? 由什么组成的? Spring Batch 又是如何去调用执行?
首先,我们先来了解Job是如何定义与实现:
git传送门:spring-batch/README.md at master · vincent9309/spring-batch · GitHub
一个job可以由一个或多个step组成,通过JobBuilderFactory实例创建Bean,使用next指向下一个step;
package com.batch.demo.flow.jobFlowDemoOne;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class JobFlowDemoOne {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job JobFlowDemo1(){
return jobBuilderFactory.get("jobFlowDemo1")
.start(step1())
.next(step2())
.next(step3())
.build();
}
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.tasklet(new Tasklet() {
@Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
System.out.println("step 1");
return RepeatStatus.FINISHED;
}
}).build();
}
@Bean
public Step step2() {
return stepBuilderFactory.get("step2")
.tasklet((contribution,context)->{
System.out.println("step 2");
return RepeatStatus.FINISHED;
}).build();
}
@Bean
public Step step3() {
return stepBuilderFactory.get("step3")
.tasklet(new Tasklet() {
@Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
System.out.println("step 3");
return RepeatStatus.FINISHED;
}
}).build();
}
}
执行该job,我们可以在控制台和数据库中看到相对应的job、step信息:
2018-12-30 16:47:32.517 INFO 4972 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=jobFlowDemo1]] launched with the following parameters: [{}]
2018-12-30 16:47:32.631 INFO 4972 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
step 1
2018-12-30 16:47:32.742 INFO 4972 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
step 2
2018-12-30 16:47:32.786 INFO 4972 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
step 3
2018-12-30 16:47:32.809 INFO 4972 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=jobFlowDemo1]] completed with the following parameters: [{}] and the following status: [COMPLETED]
STATUS 状态是 COMPLETED时候,就表明该step已经运行完成。
但是,往往在现实业务处理中,我们希望能够根据 每个step操作返回的不同状态,进行判定是否进入下一个step,或者进行其他处理流程。
所以,此处我们可以修改下job的配置,使它变成有状态判定的:
@Bean
public Job JobFlowDemo1(){
return jobBuilderFactory.get("jobFlowDemo1")
// .start(step1())
// .next(step2())
// .next(step3())
// .build();
.start(step1())
.on("COMPLETED").to(step2())
.from(step2()).on("COMPLETED").to(step3())
.from(step3()).end()
.build();
}
当step1 成功执行完成后,返回COMPLETED, 才调用step2进行下一步处理。但是过多的step,不易于程序维护和复用,因此后续篇幅会引入 Spring Batch 之 flow 介绍和使用 。
可能有些同学在一步测试,多次启动项目时候,就会发现只会在第一次启动时候成功,剩下都报错,如下:
2018-12-30 16:59:48.504 INFO 7744 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=jobFlowDemo1]] completed with the following parameters: [{}] and the following status: [COMPLETED]
这是因为,Spring Batch中相同的Job,当所带参数一致的时候,有且只会启动一次。 因此我们可以通过修改job名,或者导入不同参数进行测试。
那么Job 的参数又是如何引入的? 别急,这块结合接下来马上进入的 jobLauncher、jobOperator 的demo中,给大家介绍。
在成功创建一个job后,Spring Batch 默认在项目启动时候执行配置的job。往往在正常业务处理中,需要我们手动或者定时去触发job,所以这边便引入了jobLauncher、jobOperator两个执行器。
在上一篇文章 Spring Batch 之 背景框架简介 的demo中提到 :
当application.properties 配置 spring.batch.job.enabled = false 时,即可关闭 Batch自动执行job的操作。
如何配置 jobLauncher、jobOperator? 废话不多说,我们直接看代码。
jobLauncher
此处我们通过web的API接口去调用 jobLauncher,通过接口传入job的参数。调用的Job 是根据 在创建job时候,Bean name去指定。
@Autowired
private JobLauncher jobLauncher;
@Autowired
private Job jobLaunchDemoJob;
@GetMapping("/{job1param}")
public String runJob1(@PathVariable("job1param") String job1param) throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {
System.out.println("Request to run job1 with param: " + job1param);
JobParameters jobParameters = new JobParametersBuilder()
.addString("job1param",job1param)
.toJobParameters();
jobLauncher.run(jobLaunchDemoJob,jobParameters);
return "Job1 success.";
}
接下来我们看 jobOperator的使用:
@Autowired
private JobRepository jobRepository;
@Autowired
private JobExplorer jobExplorer;
@Autowired
private JobRegistry jobRegistry;
@Autowired
private JobLauncher jobLauncher;
@Bean
public JobOperator jobOperator(){
SimpleJobOperator operator = new SimpleJobOperator();
operator.setJobLauncher(jobLauncher);
operator.setJobParametersConverter(new DefaultJobParametersConverter());
operator.setJobRepository(jobRepository);
operator.setJobExplorer(jobExplorer);
operator.setJobRegistry(jobRegistry);
return operator;
}
@Autowired
private JobOperator jobOperator;
@GetMapping("/{job2param}")
public String runJob1(@PathVariable("job2param") String job2param) throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobInstanceAlreadyExistsException, NoSuchJobException {
System.out.println("Request to run job2 with param: " + job2param);
jobOperator.start("jobOperatorDemoJob","job2param="+job2param);
return "Job2 success.";
}
最后,定时任务调用,熟悉的同学应该已经猜到了:
通过corn表达式,满足条件时候,即执行
@Scheduled(fixedDelay = 5000)
public void scheduler() throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException, JobParametersNotFoundException, NoSuchJobException {
jobOperator().startNextInstance("jobScheduledDemoJob");
}
补充:
1、通过配置文件制定数据源DataSource,Spring Batch 根据sourceTye去指定数据库类型,执行脚本。
在多数据源情况下,默认使用primary。手动指定增加如下配置:
@Bean
JobRepository obRepository(PlatformTransactionManager platformTransactionManager){
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
jobRepositoryFactoryBean.setDatabaseType(DatabaseType.MYSQL.getProductName());
//配置指定的dataSource
jobRepositoryFactoryBean.setDataSource(dataSource);
jobRepositoryFactoryBean.setTransactionManager(platformTransactionManager);
return jobRepositoryFactoryBean.getObject();
}
2、job的参数是在整个job的step的生命周期中都可以使用到,我们可以根据不同业务处理逻辑,传入所需参数。
调用过程,demo如下:
package com.batch.demo.flow.jobParametersDemo;
import org.springframework.batch.core.*;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.Map;
@Configuration
public class JobParametersDemoConfiguration implements StepExecutionListener{
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
private Map<String, JobParameter> params;
@Bean
public Job myJobParametersDemoJob(){
return jobBuilderFactory.get("myJobParametersDemoJob")
.start(myJobParametersDemoStep())
.build();
}
@Bean
public Step myJobParametersDemoStep() {
return stepBuilderFactory.get("myJobParametersDemoStep")
.listener(this)
.tasklet(((contribution, chunkContext) -> {
System.out.println("Parameter is : " + params.get("info"));
return RepeatStatus.FINISHED;
})).build();
}
@Override
public void beforeStep(StepExecution stepExecution) {
params = stepExecution.getJobParameters().getParameters();
}
@Override
public ExitStatus afterStep(StepExecution stepExecution) {
return null;
}
}
demo中使用到了 step的监听器,在后续章节会逐步讲解,此处不予细讲。