1. 背景
最近数仓系统的元数据分析和任务依赖分析用到Neo4J,原有的后台模块使用了Mysql,需要在同一个模块中同时操作Mysql和Neo4J,整合的方案网上也有不少,感觉别人好像不会有整合的问题一样,而我在整合的过程中,碰到一系列问题。
2. 整合步骤
1. 添加Jar包依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-neo4j</artifactId>
</dependency>
2. 添加Neo4J数据源配置(我贴自己的配置文件)
spring:
datasource:
type: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://10.200.1.112:3306/datacenter?useUnicode=true&useSSL=false&characterEncoding=utf8&serverTimezone=UTC
username: root
password: 123123
druid:
initial-size: 8
min-idle: 3
max-active: 20
max-wait: 60000
time-between-eviction-runsMillis: 60000
min-evictable-idle-timeMillis: 300000
validation-query: SELECT 1 FROM DUAL
test-while-idle: true
test-on-borrow: false
test-on-return: false
pool-prepared-statements: true
max-open-prepared-statements: 20
max-pool-prepared-statement-per-connection-size: 20
filters: stat,wall,slf4j
connection-properties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
use-global-data-source-stat: true
web-stat-filter:
enabled: true
url-pattern: /*
exclusions: '*.js,*.gif,*.jpg,*.bmp,*.png,*.css,*.ico,/druid/*'
stat-view-servlet:
enabled: true
allow:
reset-enable: 'false'
url-pattern: '/druid/*'
login-username: ''
login-password: ''
quartz:
job-store-type: jdbc
properties:
org:
quartz:
scheduler:
instanceName: ClusteredScheduler
instanceId: AUTO
jobStore:
class: org.quartz.impl.jdbcjobstore.JobStoreTX
driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
tablePrefix: QRTZ_
isClustered: true
useProperties: false
dontSetAutoCommitFalse: true
misfireThreshold: 600000
threadPool:
class: org.quartz.simpl.SimpleThreadPool
threadCount: 10
threadPriority: 5
threadsInheritContextClassLoaderOfInitializingThread: true
redis:
host: 10.200.1.112
port: 6379
password:
timeout: 30000
jedis:
pool:
max-active: 200
max-wait: -1ms
max-idle: 8
min-idle: 0
data:
neo4j:
uri: bolt://10.200.1.111:7687
username: neo4j
password: test
3. 创建实体Entity
创建实体类和关系实体类时,建议单独放到一个包或者说文件夹下,便于后面配置EntityScan,千万不要和Mysql相关的类放到一个文件夹,避免引起冲突。例子的包名为com.github.datacenter.neo4j.entity
package com.github.datacenter.neo4j.entity
// 实体类
@NodeEntity(label = "dept")
@Data
@Builder
public class Dept {
@Id
@GeneratedValue
private Long id;
@Property(name = "deptName")
private String deptName;
}
package com.github.datacenter.neo4j.entity
// 关系实体类
@RelationshipEntity(type = "relationShip")
@Data
@Builder
public class RelationShip {
@Id
@GeneratedValue
private Long id;
@StartNode
private Dept parent;
@EndNode
private Dept child;
}
3. 创建操作类
创建操作类时,建议单独放到一个包或者说文件夹下,便于后面配@EnableNeo4jRepositories
,千万不要和Mysql相关Mapper类放到一个文件夹,避免引起冲突。 例子的包名为com.github.datacenter.neo4j.repositories
package com.github.datacenter.neo4j.repositories
@Repository
public interface DeptRepository extends Neo4jRepository<Dept, Long> {
Dept findByName(String name);
}
4. 配置Neo4J,避免冲突
package com.github.datacenter.neo4j.config
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.neo4j.ogm.session.SessionFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.neo4j.repository.config.EnableNeo4jRepositories;
import org.springframework.data.neo4j.transaction.Neo4jTransactionManager;
import org.springframework.data.transaction.ChainedTransactionManager;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.sql.DataSource;
@Configuration
@EnableTransactionManagement
@EnableNeo4jRepositories(basePackages = {"com.github.datacenter.neo4j.repositories"})
@EntityScan(basePackages = {"com.github.datacenter.neo4j.entity"} )
public class Neo4JConfig {
// 1. 此处为了修改默认事务,必须改。加载了Neo4J依赖库之后,transactionManager变成Neo4jTransactionManager,不增加此处,启动会报错,Mysql无法使用。
@Bean("transactionManager")
@Primary
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
// 2. Neo4J的事务管理
@Bean("neo4jTransactionManager")
public Neo4jTransactionManager neo4jTransactionManager(SessionFactory sessionFactory) {
return new Neo4jTransactionManager(sessionFactory);
}
// 3. 此处可选,非必须。需要使用多种事务时,需要加。
@Autowired
@Bean(name = "multiTransactionManager")
public PlatformTransactionManager multiTransactionManager(
Neo4jTransactionManager neo4jTransactionManager,
DataSourceTransactionManager mysqlTransactionManager) {
return new ChainedTransactionManager(
neo4jTransactionManager, mysqlTransactionManager);
}
}
1. @EnableNeo4jRepositories是为了配置Neo4J的扫描路径,不配置,会把所有的文件夹中的类都当作Neo4jRepository来扫描
2. @EntityScan为了配置实体和实体关系节点扫描路径,不配置,查询的时候找不到类,会报错。
3. 事务管理器说明
@Bean("transactionManager")
@Primary(有两个事务管理器时,最好加上这个注解@Primary)
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
如果想单独引用neo4j的事务也可以这样引用,代码如下
@Transactional(value="neo4jTransactionManager")
3. 碰到的问题
我特地试了一下缺少上面的某个代码段,会报什么样子的错误。下面某些问题,注释掉部分代码实验得到的,有些是真的碰到的。
1. 加载Neo4J依赖库,导致了默认transactionManager被Neo4J的事务管理局替换。现象如下:
Description:
Field dataSourceTransactionManager in xxxxxx required a bean of type 'org.springframework.jdbc.datasource.DataSourceTransactionManager' that could not be found.
The injection point has the following annotations:
- @org.springframework.beans.factory.annotation.Autowired(required=true)
The following candidates were found but could not be injected:
- Bean method 'transactionManager' in 'DataSourceTransactionManagerAutoConfiguration.DataSourceTransactionManagerConfiguration' not loaded because @ConditionalOnMissingBean (types: org.springframework.transaction.PlatformTransactionManager; SearchStrategy: all) found beans of type 'org.springframework.transaction.PlatformTransactionManager' transactionManager
Action:
Consider revisiting the entries above or defining a bean of type 'org.springframework.jdbc.datasource.DataSourceTransactionManager' in your configuration.
解决方案:
@Bean("transactionManager")
@Primary (@Primary注解可选)
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
2. neo4jTransactionManager事务管理器问题
===2022-01-18 17:00:41.204 [http-nio-8080-exec-1] ERROR xxxx Line:108 - No bean named 'neo4jTransactionManager' available: No matching TransactionManager bean found for qualifier 'neo4jTransactionManager' - neither qualifier match nor bean name match!
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'neo4jTransactionManager' available: No matching TransactionManager bean found for qualifier 'neo4jTransactionManager' - neither qualifier match nor bean name match!
at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.qualifiedBeanOfType(BeanFactoryAnnotationUtils.java:136)
at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.qualifiedBeanOfType(BeanFactoryAnnotationUtils.java:95)
at org.springframework.transaction.interceptor.TransactionAspectSupport.determineQualifiedTransactionManager(TransactionAspectSupport.java:492)
at org.springframework.transaction.interceptor.TransactionAspectSupport.determineTransactionManager(TransactionAspectSupport.java:470)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:335)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org
解决方案:
@Bean("neo4jTransactionManager")
public Neo4jTransactionManager neo4jTransactionManager(SessionFactory sessionFactory) {
return new Neo4jTransactionManager(sessionFactory);
}
3. 无法找到实体节点类和实体关系类
org.springframework.data.mapping.MappingException: Couldn't find PersistentEntity for type com.github.datacenter.neo4j.entity.Dept!
at org.springframework.data.mapping.context.MappingContext.getRequiredPersistentEntity(MappingContext.java:119)
at org.springframework.data.mapping.context.PersistentPropertyPathFactory.createPersistentPropertyPath(PersistentPropertyPathFactory.java:203)
at org.springframework.data.mapping.context.PersistentPropertyPathFactory.lambda$getPersistentPropertyPath$2(PersistentPropertyPathFactory.java:183)
at java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:324)
at org.springframework.data.mapping.context.PersistentPropertyPathFactory.getPersistentPropertyPath(PersistentPropertyPathFactory.java:182)
at org.springframework.data.mapping.context.PersistentPropertyPathFactory.from(PersistentPropertyPathFactory.java:97)
解决方案:
@EntityScan(basePackages = {"com.github.datacenter.neo4j.entity"} )
4. 无法找到Neo4J操作类
此种错误一般就是因为没有扫描到此组件导致的。
Description:
Field Dept in xxxx required a bean of type 'com.github.datacenter.neo4j.repositories.DeptRepository' that could not be found.
The injection point has the following annotations:
- @org.springframework.beans.factory.annotation.Autowired(required=true)
Action:
Consider defining a bean of type 'com.github.datacenter.neo4j.repositories.DeptRepository' in your configuration.
解决方案:
@EnableNeo4jRepositories(basePackages = {"com.github.datacenter.neo4j.repositories"})