一,应用场景,在开放平台中,网关校验参数时,需要读取数据,即从缓存中读取
核心概念
1,本地缓存
2,redis缓存
3,缓存字符串与缓存对象,对象保存到redis需要序列化
4,监听器刷新缓存
二,读取缓存流程顺序
读取本地缓存-redis-数据库
详细流程
1,首先从本地缓存读取,本地缓存实现方式,通过ConcurrentHashMap实现
/**
* 接口详情
* key:接口名称method
* value:接口详情
*/
public final static ConcurrentHashMap<String, ApiDocument> OPEN_DOCUMENT_CACHE = new ConcurrentHashMap<>();
2,本地无,读取redis缓存,然后保存到本地缓存
注意:如果缓存值为对象,从redis读取后,需要对数据进行反序列化之前,保存到本地缓存
3,redis无数据,从数据库中读取数据,然后保存到redis及本地缓存
注意:如果缓存值为对象,保存到redis需要进行序列化,保存到本地为从数据库中取到对象值
三,刷新缓存
应用场景:在anji-open-service项目中进行更新或者删除操作后,需要更新缓存,通过stream-rabbit消息队列异步更新缓存
即,删除本地缓存,redis缓存,从新从数据库中读取数据,再写入redis缓存,本地缓存
实现原理,通过监听器实现
四,代码实现
1,缓存帮助类
/**
* 缓存帮助类
* @author lr
* @date 2019-07-26 16:42
*/
public class CacheHelper {
private Logger logger = LoggerFactory.getLogger(CacheHelper.class);
private RedisTemplate redisTemplate;
public CacheHelper(RedisTemplate redisTemplate) {
this.redisTemplate = redisTemplate;
}
@Autowired
private ApiProjectMapper apiProjectMapper;
@Autowired
private StringRedisTemplate stringRedisTemplate;
@Autowired
private ApiDocumentMapper apiDocumentMapper;
@Autowired
private AppMapper appMapper;
/**
* 获取应用秘钥
* @param appId
* @return
*/
public String getOpenAppSecret(String appId) {
//1、本地缓存
String secret = LocalCache.OPEN_APP_SECRET_CACHE.get(appId);
if (secret != null) {
return secret;
}
//2、redis缓存
String redisKey = CacheKeyEnum.APP.getKey() + appId;
BoundValueOperations<String, String> operations = stringRedisTemplate.boundValueOps(redisKey);
if (operations.get() != null) {
String secretValue = operations.get();
if (secretValue != null) {
LocalCache.OPEN_APP_SECRET_CACHE.put(appId, secretValue);
return secretValue;
}
}
//3、从数据库获取
OpenApp openApp = appMapper.selectOne(new QueryWrapper<OpenApp>()
.eq(GatewayConstant.STATUS, GatewayConstant.ENABLE)
.eq(GatewayConstant.APP_ID, appId));
if (openApp == null || StringUtils.isBlank(openApp.getAppSecret())) {
logger.error("平台不存在该应用对应的信息,{}",appId);
return null;
}
secret = openApp.getAppSecret();
//保存redis和本地缓存
ValueOperations<String, String> operationsToSave = stringRedisTemplate.opsForValue();
operationsToSave.set(redisKey, secret);
LocalCache.OPEN_APP_SECRET_CACHE.put(appId, secret);
return secret;
}
/**
* 获取项目服务地址
* @param projectCode
* @return
*/
public String getApiProjectUrl(String projectCode) {
//1、本地缓存
String projectUrl = LocalCache.OPEN_PROJECT_URL_CACHE.get(projectCode);
if (projectUrl != null) {
return projectUrl;
}
//2、redis缓存
String redisKey = CacheKeyEnum.PROJECT.getKey() + projectCode;
BoundValueOperations<String, String> operations = stringRedisTemplate.boundValueOps(redisKey);
if (operations.get() != null) {
projectUrl = operations.get();
if (projectUrl != null) {
LocalCache.OPEN_PROJECT_URL_CACHE.put(projectCode, projectUrl);
return projectUrl;
}
}
//3、从数据库获取
projectUrl = apiProjectMapper.selectOne(new QueryWrapper<ApiProject>().eq(GatewayConstant.PROJECT_CODE, projectCode)).getProductUrl();
if (projectUrl == null) {
logger.error("平台不存在该项目对应的信息,{}",projectCode);
return null;
}
//保存redis和本地缓存
ValueOperations<String, String> operationsToSave = stringRedisTemplate.opsForValue();
operationsToSave.set(redisKey, projectUrl);
LocalCache.OPEN_PROJECT_URL_CACHE.put(projectCode, projectUrl);
return projectUrl;
}
/**
* 获取接口相关信息
* @param method
* @return
*/
public ApiDocument getApiDocument(String method) {
//1、本地缓存
ApiDocument apiDocument = LocalCache.OPEN_DOCUMENT_CACHE.get(method);
if (apiDocument != null) {
return apiDocument;
}
//2、redis缓存
BoundValueOperations<String, byte[]> operations = redisTemplate.boundValueOps(CacheKeyEnum.DOCUMENT.getKey() + method);
if (operations.get() != null) {
byte[] bytes = operations.get();
apiDocument = ProtoStuffUtils.deSerialize(bytes, ApiDocument.class);
if (apiDocument != null) {
LocalCache.OPEN_DOCUMENT_CACHE.put(method, apiDocument);
return apiDocument;
}
}
//3、从数据库获取
apiDocument = apiDocumentMapper.selectOne(new QueryWrapper<ApiDocument>()
.eq(GatewayConstant.STATUS, GatewayConstant.METHOD_PUBLISHED)
.eq(GatewayConstant.METHOD, method));
if (apiDocument == null) {
logger.error("平台不存在该接口对应的信息,{}",method);
return null;
}
//保存redis和本地缓存
ValueOperations<String, byte[]> operationsToSave = redisTemplate.opsForValue();
operationsToSave.set(CacheKeyEnum.DOCUMENT.getKey() + apiDocument.getMethod(), ProtoStuffUtils.serialize(apiDocument));
LocalCache.OPEN_DOCUMENT_CACHE.put(method, apiDocument);
return apiDocument;
}
/**
* 刷新应用秘钥
* @param appId
*/
public void refreshAppCache(String appId) {
//1、删除本地缓存
LocalCache.OPEN_APP_SECRET_CACHE.remove(appId);
//2、删除redis
String redisKey = CacheKeyEnum.APP.getKey()+appId;
stringRedisTemplate.delete(redisKey);
//3、从数据库获取
OpenApp openApp = appMapper.selectOne(new QueryWrapper<OpenApp>()
.eq(GatewayConstant.STATUS, GatewayConstant.ENABLE)
.eq(GatewayConstant.APP_ID, appId));
if (openApp == null) {
logger.error("刷新操作:平台不存在该应用对应的信息或执行删除操作,{}",appId);
return;
}
String appSecret = openApp.getAppSecret();
//保存redis和本地缓存
ValueOperations<String, String> operationsToSave = stringRedisTemplate.opsForValue();
operationsToSave.set(redisKey, appSecret);
LocalCache.OPEN_APP_SECRET_CACHE.put(appId, appSecret);
}
/**
* 刷新项目服务地址
* @param projectCode
*/
public void refreshProjectCache(String projectCode) {
//1、删除本地缓存
LocalCache.OPEN_PROJECT_URL_CACHE.remove(projectCode);
//2、删除redis
String redisKey = CacheKeyEnum.PROJECT.getKey()+projectCode;
stringRedisTemplate.delete(redisKey);
//3、从数据库获取
ApiProject apiProject = apiProjectMapper.selectOne(new QueryWrapper<ApiProject>().eq(GatewayConstant.PROJECT_CODE, projectCode));
if (apiProject == null) {
logger.error("刷新操作:平台不存在该项目对应的信息或执行删除操作,{}",projectCode);
return;
}
//保存redis和本地缓存
String projectUrl = apiProject.getProductUrl();
ValueOperations<String, String> operationsToSave = stringRedisTemplate.opsForValue();
operationsToSave.set(redisKey, projectUrl);
LocalCache.OPEN_PROJECT_URL_CACHE.put(projectCode, projectUrl);
}
/**
* 刷新接口缓存
* @param method
*/
public void refreshDocumentCache(String method) {
//1、删除本地缓存
LocalCache.OPEN_DOCUMENT_CACHE.remove(method);
LocalCache.PROTO_BUFF_CACHE.remove(method);
LocalCache.PROTO_STUFF_CACHE.remove(method);
//2、删除redis
redisTemplate.delete(CacheKeyEnum.DOCUMENT.getKey()+method);
//3、从数据库获取
ApiDocument apiDocument = apiDocumentMapper.selectOne(new QueryWrapper<ApiDocument>()
.eq(GatewayConstant.STATUS, GatewayConstant.METHOD_PUBLISHED)
.eq(GatewayConstant.METHOD, method));
if (apiDocument == null) {
logger.error("刷新操作:平台不存在该接口对应的信息或执行删除操作,{}",method);
return;
}
//保存redis和本地缓存
ValueOperations<String, byte[]> operationsToSave = redisTemplate.opsForValue();
operationsToSave.set(CacheKeyEnum.DOCUMENT.getKey() + apiDocument.getMethod(), ProtoStuffUtils.serialize(apiDocument));
LocalCache.OPEN_DOCUMENT_CACHE.put(method, apiDocument);
}
}
2,本地缓存实现
/**
* 本地缓存
* @author lr
* @date 2019-09-06 15:57
*/
public class LocalCache {
/**
* 缓存请求监控数据
* key:uuid,请求唯一标识
* value:包含一些请求信息
*/
public final static ConcurrentHashMap<String, OpenMonitorEntity> MONITOR_CACHE = new ConcurrentHashMap<>();
/**
* 应用秘钥缓存
* key:appId
* value:应用详情
*/
public final static ConcurrentHashMap<String, String> OPEN_APP_SECRET_CACHE = new ConcurrentHashMap<>();
/**
* 项目服务地址缓存
* key:项目编码
* value:项目详情
*/
public final static ConcurrentHashMap<String, String> OPEN_PROJECT_URL_CACHE = new ConcurrentHashMap<>();
/**
* 接口详情
* key:接口名称method
* value:接口详情
*/
public final static ConcurrentHashMap<String, ApiDocument> OPEN_DOCUMENT_CACHE = new ConcurrentHashMap<>();
/**
* protoStuff对应method的缓存
*/
public final static ConcurrentHashMap<String, Class<Object>> PROTO_STUFF_CACHE = new ConcurrentHashMap<>();
/**
* proto_buff对应的description
*/
public final static ConcurrentHashMap<String, Descriptors.Descriptor> PROTO_BUFF_CACHE = new ConcurrentHashMap<>();
}
3,序列化及反序列化实现
/**
* 序列化,保存对象到redis
* @author lr
* @date 2019-07-30 11:40
*/
public class ProtoStuffUtils {
/**
* 序列化
* @param message
* @return
*/
public static <T> byte[] serialize(T message) {
Class<T> cls = (Class<T>) message.getClass();
LinkedBuffer linkedBuffer = LinkedBuffer.allocate();
Schema<T> schema = RuntimeSchema.getSchema(cls);
byte[] bytes = ProtostuffIOUtil.toByteArray(message,schema, linkedBuffer);
return bytes;
}
/**
* 反序列化
* @param bytes
* @return
*/
public static <T> T deSerialize(byte[] bytes,Class<T> cls) {
T message = null;
try {
message = cls.newInstance();
ProtostuffIOUtil.mergeFrom(bytes,message, RuntimeSchema.getSchema(cls));
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return message;
}
}
4,监听器实现刷新缓存
/**
* 本地缓存更新
* @author lr
* @date 2019-09-09 10:54
*/
@EnableBinding(Sink.class)
public class LocalCacheListener {
@Autowired
private CacheHelper cacheHelper;
/**
* 监听本地缓存变化
* @param message
*/
@StreamListener(value = Sink.INPUT)
public void onListener(Message<JSONObject> message){
JSONObject payload = message.getPayload();
String key = payload.getString(ApiConstants.LOCAL_CACHE_LISTENER_KEY);
String type = payload.getString(ApiConstants.LOCAL_CACHE_LISTENER_TYPE);
//获取本地缓存类型
LocalCacheType match = LocalCacheType.match(type);
//刷新本地缓存和redis
switch (match){
case OPEN_APP:
cacheHelper.refreshAppCache(key);
break;
case OPEN_PROJECT:
cacheHelper.refreshProjectCache(key);
break;
case OPEN_DOCUMENT:
cacheHelper.refreshDocumentCache(key);
break;
default:
}
}
}
疑问
1,何时刷新缓存,实现原理
在anji-open-service项目中进行更新或者删除操作后,需要更新缓存,通过stream-rabbit消息队列异步更新缓存
疑问:在service项目中发布消息,在getway中获取消息更新缓存,实现原理,怎么接收到的?为什么放到消息队列为什么不直接更新?
service与getway通过通道配置互为消费者生产者,针对刷新缓存来说,service即admin为生产者,getway为消费者
open-admin-dev.yaml
output:
destination: refresh-cache open-gateway-dev.yaml
input:
destination: refresh-cache
----------------------------------------------------
1,表中唯一字段加唯一索引,如code,name