一、Eureka Server
Eureka Server为了避免同时读写内存数据结构造成的并发冲突问题，采用了多级缓存机制来进一步提升服务请求的响应速度。
Eureka Server存在三个变量：（registry、readWriteCacheMap、readOnlyCacheMap）保存服务注册信息，默认情况下定时任务每30s将readWriteCacheMap同步至readOnlyCacheMap，每60s清理超过90s未续约的节点，Eureka Client每30s从readOnlyCacheMap更新服务注册信息，而UI则从registry更新服务注册信息。
三级缓存
| 缓存 | 类型 | 说明 |
|---|---|---|
| registry | ConcurrentHashMap | 实时更新，类AbstractInstanceRegistry成员变量，UI端请求的是这里的服务注册信息 |
| readWriteCacheMap | Guava Cache/LoadingCache | 实时更新，类ResponseCacheImpl成员变量，缓存时间180秒 |
| readOnlyCacheMap | ConcurrentHashMap | 周期更新，类ResponseCacheImpl成员变量，默认每30s从readWriteCacheMap更新，Eureka client默认从这里更新服务注册信息，可配置直接从readWriteCacheMap更新 |
注意： readWriteCacheMap：是Guava缓存，数据主要同步于存储层即注册表registry 。当获取缓存时判断缓存中是否没有数据，如果不存在此数据，则通过 CacheLoader 的 load 方法去加载，加载成功之后将数据放入缓存，同时返回数据。默认180s过期，当服务下线、过期、注册、状态变更，都会来清除此缓存中的数据。
缓存工作方式：
###缓存相关配置
| 配置 | 默认 | 说明 |
|---|---|---|
| eureka.server.useReadOnlyResponseCache | true | Client从readOnlyCacheMap更新数据，false则跳过readOnlyCacheMap直接从readWriteCacheMap更新 |
| eureka.server.responsecCacheUpdateIntervalMs | 30000 | readWriteCacheMap更新至readOnlyCacheMap周期，默认30s |
| eureka.server.evictionIntervalTimerInMs | 60000 | 清理未续约节点周期，默认60s |
| eureka.instance.leaseExpirationDurationInSeconds | 90 | 清理未续约节点超时时间，默认90s |
eureka server端的多级缓存机制
- 重点看看eureka server端的多级缓存机制的过期失效机制。
在server端，关于过期，其实有3中机制，分别是主动过期，被动过期和定时过期。
1、主动过期
主动过期主要是针对RW缓存，有新的服务注册、下线、故障都会刷新RW缓存的Map。
比如有一个新实例来注册，在注册逻辑最后会调用invalidateCache方法，这个方法就是去过期掉RW缓存的Map。
Eureka Server在接受Eureka Client服务注册的流程，即AbstractInstanceRegistry类的register方法最后会调用invalidateCache方法清理缓存为入口
public abstract class AbstractInstanceRegistry implements InstanceRegistry {
private final ConcurrentHashMap<String, Map<String, Lease<InstanceInfo>>> registry
= new ConcurrentHashMap<String, Map<String, Lease<InstanceInfo>>>();
public void register(InstanceInfo registrant, int leaseDuration, boolean isReplication) {
try {
// 上只读锁
read.lock();
// 从本地MAP里面获取当前实例的信息。
Map<String, Lease<InstanceInfo>> gMap = registry.get(registrant.getAppName());
//省略中间代码。。。。。。
// 放入本地Map中
gMap.put(registrant.getId(), lease);
//省略中间代码。。。。。。
// 设置注册类型为添加
registrant.setActionType(ActionType.ADDED);
// 租约变更记录队列，记录了实例的每次变化， 用于注册信息的增量获取
recentlyChangedQueue.add(new RecentlyChangedItem(lease));
registrant.setLastUpdatedTimestamp();
// 清理缓存 ，传入的参数为key
invalidateCache(registrant.getAppName(), registrant.getVIPAddress(), registrant.getSecureVipAddress());
logger.info("Registered instance {}/{} with status {} (replication={})",
registrant.getAppName(), registrant.getId(), registrant.getStatus(), isReplication);
} finally {
read.unlock();
}
}
}
public abstract class AbstractInstanceRegistry implements InstanceRegistry {
protected volatile ResponseCache responseCache;
private void invalidateCache(String appName, @Nullable String vipAddress, @Nullable String secureVipAddress) {
// 清除缓存
responseCache.invalidate(appName, vipAddress, secureVipAddress);
}
}
public class ResponseCacheImpl implements ResponseCache {
@Override
public void invalidate(String appName, @Nullable String vipAddress, @Nullable String secureVipAddress) {
for (Key.KeyType type : Key.KeyType.values()) {
for (Version v : Version.values()) {
invalidate(
new Key(Key.EntityType.Application, appName, type, v, EurekaAccept.full),
new Key(Key.EntityType.Application, appName, type, v, EurekaAccept.compact),
new Key(Key.EntityType.Application, ALL_APPS, type, v, EurekaAccept.full),
new Key(Key.EntityType.Application, ALL_APPS, type, v, EurekaAccept.compact),
new Key(Key.EntityType.Application, ALL_APPS_DELTA, type, v, EurekaAccept.full),
new Key(Key.EntityType.Application, ALL_APPS_DELTA, type, v, EurekaAccept.compact)
);
if (null != vipAddress) {
invalidate(new Key(Key.EntityType.VIP, vipAddress, type, v, EurekaAccept.full));
}
if (null != secureVipAddress) {
invalidate(new Key(Key.EntityType.SVIP, secureVipAddress, type, v, EurekaAccept.full));
}
}
}
}
}
在这里会调用readWriteCacheMap.invalidate(key)来过期RW缓存Map的数据，服务下线、故障都会走类似的逻辑。
public class ResponseCacheImpl implements ResponseCache {
private final LoadingCache<Key, Value> readWriteCacheMap;
public void invalidate(Key... keys) {
for (Key key : keys) {
logger.debug("Invalidating the response cache key : {} {} {} {}, {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());
readWriteCacheMap.invalidate(key);
Collection<Key> keysWithRegions = regionSpecificKeys.get(key);
if (null != keysWithRegions && !keysWithRegions.isEmpty()) {
for (Key keysWithRegion : keysWithRegions) {
logger.debug("Invalidating the response cache key : {} {} {} {} {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());
readWriteCacheMap.invalidate(keysWithRegion);
}
}
}
}
}
2、被动过期
被动过期，主要是针对RO缓存，readOnlyCacheMap默认是每隔30秒，执行一个定时调度的线程任务，TimerTask，对readOnlyCacheMap和readWriteCacheMap中的数据进行一个比对，如果两块数据不一致的，那么就将readWriteCacheMap中的数据放到readOnlyCacheMap中来。
比如说readWriteCacheMap中，ALL_APPS这个key对应的缓存没了，那么最多30秒过后，就会同步到readOnelyCacheMap中去。
这段代码依然在ResponseCacheImpl的构造方法里，这个timer叫做一个eureka缓存填充的timer。
public class ResponseCacheImpl implements ResponseCache {
private final java.util.Timer timer = new java.util.Timer("Eureka-CacheFillTimer", true);
ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
if (shouldUseReadOnlyResponseCache) {
timer.schedule(getCacheUpdateTask(),
new Date(((System.currentTimeMillis() / responseCacheUpdateIntervalMs) * responseCacheUpdateIntervalMs)
+ responseCacheUpdateIntervalMs),
responseCacheUpdateIntervalMs);
}
try {
Monitors.registerObject(this);
} catch (Throwable e) {
logger.warn("Cannot register the JMX monitor for the InstanceRegistry", e);
}
}
}
可以看到它的getCacheUpdateTask()方法直接返回一个TimerTask，就是完成RW缓存和RO缓存数据交互的逻辑。
public class ResponseCacheImpl implements ResponseCache {
private final ConcurrentMap<Key, Value> readOnlyCacheMap = new ConcurrentHashMap<Key, Value>();
private final LoadingCache<Key, Value> readWriteCacheMap;
private TimerTask getCacheUpdateTask() {
return new TimerTask() {
@Override
public void run() {
logger.debug("Updating the client cache from response cache");
for (Key key : readOnlyCacheMap.keySet()) {
if (logger.isDebugEnabled()) {
logger.debug("Updating the client cache from response cache for key : {} {} {} {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType());
}
try {
CurrentRequestVersion.set(key.getVersion());
Value cacheValue = readWriteCacheMap.get(key);
Value currentCacheValue = readOnlyCacheMap.get(key);
//如果RO缓存中的数据和RW不一致，则put
if (cacheValue != currentCacheValue) {
readOnlyCacheMap.put(key, cacheValue);
}
} catch (Throwable th) {
logger.error("Error while updating the client cache from response cache for key {}", key.toStringCompact(), th);
} finally {
CurrentRequestVersion.remove();
}
}
}
};
}
}
而这个responseCacheUpdateIntervalMs，默认30s。
@Singleton
public class DefaultEurekaServerConfig implements EurekaServerConfig {
@Override
public long getResponseCacheUpdateIntervalMs() {
return configInstance.getIntProperty(
namespace + "responseCacheUpdateIntervalMs", (30 * 1000)).get();
}
}
3、定时过期
这个定时过期，实际上也是针对RW缓存的那个readWriteCacheMap的，在构建的时候会指定一个自动过期的时间，默认是180s，因此放入RW缓存中的数据默认会在3分钟之内过期掉。
public class ResponseCacheImpl implements ResponseCache {
private final ConcurrentMap<Key, Value> readOnlyCacheMap = new ConcurrentHashMap<Key, Value>();
private final LoadingCache<Key, Value> readWriteCacheMap;
ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
this.readWriteCacheMap =
CacheBuilder.newBuilder().initialCapacity(serverConfig.getInitialCapacityOfResponseCache())
.expireAfterWrite(serverConfig.getResponseCacheAutoExpirationInSeconds(), TimeUnit.SECONDS)
.removalListener(new RemovalListener<Key, Value>() {
@Override
public void onRemoval(RemovalNotification<Key, Value> notification) {
Key removedKey = notification.getKey();
if (removedKey.hasRegions()) {
Key cloneWithNoRegions = removedKey.cloneWithoutRegions();
regionSpecificKeys.remove(cloneWithNoRegions, removedKey);
}
}
})
.build(new CacheLoader<Key, Value>() {
@Override
public Value load(Key key) throws Exception {
if (key.hasRegions()) {
Key cloneWithNoRegions = key.cloneWithoutRegions();
regionSpecificKeys.put(cloneWithNoRegions, key);
}
Value value = generatePayload(key);
return value;
}
});
//省略部分代码......
}
}
通过源码可以明确，这个getResponseCacheAutoExpirationInSeconds()的默认值就是180s。
@Singleton
public class DefaultEurekaServerConfig implements EurekaServerConfig {
@Override
public long getResponseCacheAutoExpirationInSeconds() {
return configInstance.getIntProperty(
namespace + "responseCacheAutoExpirationInSeconds", 180).get();
}
}
Eureka Client获取注册信息
Eureka Client获取注册信息通过ApplicationsResource类的getContainers方法为入口
@Path("/{version}/apps")
@Produces({"application/xml", "application/json"})
public class ApplicationsResource {
@GET
public Response getContainers(@PathParam("version") String version,
@HeaderParam(HEADER_ACCEPT) String acceptHeader,
@HeaderParam(HEADER_ACCEPT_ENCODING) String acceptEncoding,
@HeaderParam(EurekaAccept.HTTP_X_EUREKA_ACCEPT) String eurekaAccept,
@Context UriInfo uriInfo,
@Nullable @QueryParam("regions") String regionsStr) {
boolean isRemoteRegionRequested = null != regionsStr && !regionsStr.isEmpty();
String[] regions = null;
if (!isRemoteRegionRequested) {
EurekaMonitors.GET_ALL.increment();
} else {
regions = regionsStr.toLowerCase().split(",");
Arrays.sort(regions); // So we don't have different caches for same regions queried in different order.
EurekaMonitors.GET_ALL_WITH_REMOTE_REGIONS.increment();
}
// Check if the server allows the access to the registry. The server can
// restrict access if it is not
// ready to serve traffic depending on various reasons.
// EurekaServer无法提供服务，返回403
if (!registry.shouldAllowAccess(isRemoteRegionRequested)) {
return Response.status(Status.FORBIDDEN).build();
}
CurrentRequestVersion.set(Version.toEnum(version));
// 设置返回数据格式，默认JSON
KeyType keyType = Key.KeyType.JSON;
String returnMediaType = MediaType.APPLICATION_JSON;
// 如果接收到的请求头部没有具体格式信息，则返回格式为XML
if (acceptHeader == null || !acceptHeader.contains(HEADER_JSON_VALUE)) {
keyType = Key.KeyType.XML;
returnMediaType = MediaType.APPLICATION_XML;
}
//创建一个缓存对象 构建缓存键
Key cacheKey = new Key(Key.EntityType.Application,
ResponseCacheImpl.ALL_APPS, //全量
keyType, CurrentRequestVersion.get(), EurekaAccept.fromString(eurekaAccept), regions
);
Response response;
// 返回不同的编码类型的数据，去缓存中取数据的方法基本一致
if (acceptEncoding != null && acceptEncoding.contains(HEADER_GZIP_VALUE)) {
response = Response.ok(responseCache.getGZIP(cacheKey)) //获取压缩的数据
.header(HEADER_CONTENT_ENCODING, HEADER_GZIP_VALUE)
.header(HEADER_CONTENT_TYPE, returnMediaType)
.build();
} else {
response = Response.ok(responseCache.get(cacheKey))
.build();
}
CurrentRequestVersion.remove();
return response;
}
}
responseCache.getGZIP(cacheKey)
- 从缓存中读取GZIP压缩数据。
public class ResponseCacheImpl implements ResponseCache {
private final ConcurrentMap<Key, Value> readOnlyCacheMap = new ConcurrentHashMap<Key, Value>();
private final LoadingCache<Key, Value> readWriteCacheMap;
public byte[] getGZIP(Key key) {
Value payload = getValue(key, shouldUseReadOnlyResponseCache);
if (payload == null) {
return null;
}
return payload.getGzipped();
}
@VisibleForTesting
Value getValue(final Key key, boolean useReadOnlyCache) {
Value payload = null;
try {
if (useReadOnlyCache) {
//首先从只读缓存中获取, 即readOnlyCacheMap
final Value currentPayload = readOnlyCacheMap.get(key);
if (currentPayload != null) {
payload = currentPayload;
} else {
//只读缓存readOnlyCacheMap中没有,从readWriteCacheMap缓存中获取
payload = readWriteCacheMap.get(key);
//回写只读缓存readOnlyCacheMap
readOnlyCacheMap.put(key, payload);
}
} else {
payload = readWriteCacheMap.get(key);
}
} catch (Throwable t) {
logger.error("Cannot get value for key : {}", key, t);
}
return payload;
}
}
responseCache.get(cacheKey)
- 从缓存中读取数据。
public class ResponseCacheImpl implements ResponseCache {
private final ConcurrentMap<Key, Value> readOnlyCacheMap = new ConcurrentHashMap<Key, Value>();
private final LoadingCache<Key, Value> readWriteCacheMap;
public String get(final Key key) {
return get(key, shouldUseReadOnlyResponseCache);
}
@VisibleForTesting
String get(final Key key, boolean useReadOnlyCache) {
Value payload = getValue(key, useReadOnlyCache);
if (payload == null || payload.getPayload().equals(EMPTY_PAYLOAD)) {
return null;
} else {
return payload.getPayload();
}
}
@VisibleForTesting
Value getValue(final Key key, boolean useReadOnlyCache) {
Value payload = null;
try {
if (useReadOnlyCache) {
//首先从只读缓存中获取, 即readOnlyCacheMap
final Value currentPayload = readOnlyCacheMap.get(key);
if (currentPayload != null) {
payload = currentPayload;
} else {
//只读缓存readOnlyCacheMap中没有,从readWriteCacheMap缓存中获取
payload = readWriteCacheMap.get(key);
//回写只读缓存readOnlyCacheMap
readOnlyCacheMap.put(key, payload);
}
} else {
payload = readWriteCacheMap.get(key);
}
} catch (Throwable t) {
logger.error("Cannot get value for key : {}", key, t);
}
return payload;
}
}
二、Eureka Client
Eureka Client存在两种角色：服务提供者和服务消费者，作为服务消费者一般配合Ribbon或Feign（Feign内部使用Ribbon）使用。Eureka Client启动后，作为服务提供者立即向Eureka Server注册，默认情况下每30s续约；作为服务消费者立即向Server全量更新服务注册信息，默认情况下每30s增量更新服务注册信息；Ribbon延时1s向Client获取使用的服务注册信息，默认每30s更新使用的服务注册信息，只保存状态为UP的服务。
二级缓存
| 缓存 | 类型 | 说明 |
|---|---|---|
| localRegionApps | AtomicReference | 周期更新，类DiscoveryClient成员变量，Eureka Client保存服务注册信息，启动后立即向Server全量更新，默认每30s增量更新 |
| upServerListZoneMap | ConcurrentHashMap | 周期更新，类LoadBalancerStats成员变量，Ribbon保存使用且状态为UP的服务注册信息，启动后延时1s向Client更新，默认每30s更新 |
缓存相关配置
| 配置 | 默认 | 说明 |
|---|---|---|
| eureka.instance.leaseRenewalIntervalInSeconds | 30 | Eureka Client续约周期，默认30s |
| eureka.client.registryFetchIntervalSeconds | 30 | Eureka Client增量更新周期，默认30s（正常情况下增量更新，超时或与Server端不一致等情况则全量更新） |
| ribbon.ServerListRefreshInterval | 30000 | Ribbon更新周期，默认30s |
EurekaClient 缓存
EurekaClient也存在缓存，应用服务实例列表信息在每个EurekaClient服务消费端都有缓存。一般的，Ribbon的LoadBalancer会读取这个缓存，来知道当前有哪些实例可以调用，从而进行负载均衡。这个loadbalancer同样也有缓存。
首先看这个LoadBalancer的缓存更新机制，相关类是PollingServerListUpdater：
public class PollingServerListUpdater implements ServerListUpdater {
@Override
public synchronized void start(final UpdateAction updateAction) {
if (isActive.compareAndSet(false, true)) {
final Runnable wrapperRunnable = new Runnable() {
@Override
public void run() {
if (!isActive.get()) {
if (scheduledFuture != null) {
scheduledFuture.cancel(true);
}
return;
}
try {
//从EurekaClient缓存中获取服务实例列表，保存在本地缓存
updateAction.doUpdate();
lastUpdated = System.currentTimeMillis();
} catch (Exception e) {
logger.warn("Failed one update cycle", e);
}
}
};
// 使用线程池周期性的执行wrapperRunnable任务
scheduledFuture = getRefreshExecutor().scheduleWithFixedDelay(
wrapperRunnable,
initialDelayMs,
refreshIntervalMs,
TimeUnit.MILLISECONDS
);
} else {
logger.info("Already active, no-op");
}
}
}
DynamicServerListLoadBalancer.updateListOfServers()代码逻辑
public class DynamicServerListLoadBalancer<T extends Server> extends BaseLoadBalancer {
public DynamicServerListLoadBalancer(IClientConfig clientConfig) {
class NamelessClass_1 implements UpdateAction {
public void doUpdate() {
DynamicServerListLoadBalancer.this.updateListOfServers();
}
}
}
@VisibleForTesting
public void updateListOfServers() {
List<T> servers = new ArrayList();
if (this.serverListImpl != null) {
servers = this.serverListImpl.getUpdatedListOfServers();
LOGGER.debug("List of Servers for {} obtained from Discovery client: {}", this.getIdentifier(), servers);
if (this.filter != null) {
servers = this.filter.getFilteredListOfServers((List)servers);
LOGGER.debug("Filtered List of Servers for {} obtained from Discovery client: {}", this.getIdentifier(), servers);
}
}
this.updateAllServerList((List)servers);
}
}
serverListImpl.getUpdatedListOfServers()会调用DiscoveryEnabledNIWSServerList.obtainServersViaDiscovery()方法获取servers集合
DiscoveryEnabledNIWSServerList.obtainServersViaDiscovery()方法
public class DiscoveryEnabledNIWSServerList extends AbstractServerList<DiscoveryEnabledServer>{
@Override
public List<DiscoveryEnabledServer> getUpdatedListOfServers(){
return obtainServersViaDiscovery();
}
private List<DiscoveryEnabledServer> obtainServersViaDiscovery() {
List<DiscoveryEnabledServer> serverList = new ArrayList<DiscoveryEnabledServer>();
if (eurekaClientProvider == null || eurekaClientProvider.get() == null) {
logger.warn("EurekaClient has not been initialized yet, returning an empty list");
return new ArrayList<DiscoveryEnabledServer>();
}
EurekaClient eurekaClient = eurekaClientProvider.get();
if (vipAddresses!=null){
for (String vipAddress : vipAddresses.split(",")) {
// if targetRegion is null, it will be interpreted as the same region of client
List<InstanceInfo> listOfInstanceInfo = eurekaClient.getInstancesByVipAddress(vipAddress, isSecure, targetRegion);
for (InstanceInfo ii : listOfInstanceInfo) {
if (ii.getStatus().equals(InstanceStatus.UP)) {
if(shouldUseOverridePort){
if(logger.isDebugEnabled()){
logger.debug("Overriding port on client name: " + clientName + " to " + overridePort);
}
// copy is necessary since the InstanceInfo builder just uses the original reference,
// and we don't want to corrupt the global eureka copy of the object which may be
// used by other clients in our system
InstanceInfo copy = new InstanceInfo(ii);
if(isSecure){
ii = new InstanceInfo.Builder(copy).setSecurePort(overridePort).build();
}else{
ii = new InstanceInfo.Builder(copy).setPort(overridePort).build();
}
}
DiscoveryEnabledServer des = createServer(ii, isSecure, shouldUseIpAddr);
serverList.add(des);
}
}
if (serverList.size()>0 && prioritizeVipAddressBasedServers){
break; // if the current vipAddress has servers, we dont use subsequent vipAddress based servers
}
}
}
return serverList;
}
}
从代码中可以看到，listOfInstanceInfo持有从DiscoveryClient.LocalRegionApps/remoteRegionVsApps获取到的信息后，与region和zone结合形成DiscoveryEnabledServer实例，流入到List集合返回
public class DynamicServerListLoadBalancer<T extends Server> extends BaseLoadBalancer {
protected void updateAllServerList(List<T> ls) {
if (this.serverListUpdateInProgress.compareAndSet(false, true)) {
try {
Iterator var2 = ls.iterator();
while(var2.hasNext()) {
T s = (Server)var2.next();
s.setAlive(true);
}
//调用setServersList方法
this.setServersList(ls);
super.forceQuickPing();
} finally {
this.serverListUpdateInProgress.set(false);
}
}
}
public void setServersList(List lsrv) {
// 赋值给BaseLoadBalacer.upServerList
super.setServersList(lsrv);
Map<String, List<Server>> serversInZones = new HashMap();
Iterator var4 = lsrv.iterator();
while(var4.hasNext()) {
Server server = (Server)var4.next();
// 赋值给LoadBalancerStats.zoneStatsMap
this.getLoadBalancerStats().getSingleServerStat(server);
String zone = server.getZone();
if (zone != null) {
zone = zone.toLowerCase();
List<Server> servers = (List)serversInZones.get(zone);
if (servers == null) {
servers = new ArrayList();
serversInZones.put(zone, servers);
}
((List)servers).add(server);
}
}
this.setServerListForZones(serversInZones);
}
protected void setServerListForZones(Map<String, List<Server>> zoneServersMap) {
LOGGER.debug("Setting server list for zones: {}", zoneServersMap);
//更新upServerListZoneMap缓存
this.getLoadBalancerStats().updateZoneServerMapping(zoneServersMap);
}
}
public class LoadBalancerStats implements IClientConfigAware {
volatile Map<String, ZoneStats> zoneStatsMap = new ConcurrentHashMap<String, ZoneStats>();
volatile Map<String, List<? extends Server>> upServerListZoneMap = new ConcurrentHashMap<String, List<? extends Server>>();
public void updateZoneServerMapping(Map<String, List<Server>> map) {
upServerListZoneMap = new ConcurrentHashMap<String, List<? extends Server>>(map);
// make sure ZoneStats object exist for available zones for monitoring purpose
for (String zone: map.keySet()) {
//更新zoneStatsMap
getZoneStats(zone);
}
}
private ZoneStats getZoneStats(String zone) {
zone = zone.toLowerCase();
ZoneStats zs = zoneStatsMap.get(zone);
if (zs == null){
zoneStatsMap.put(zone, new ZoneStats(this.getName(), zone, this));
zs = zoneStatsMap.get(zone);
}
return zs;
}
}
这个updateAction.doUpdate();就是从EurekaClient缓存中获取服务实例列表，保存在BaseLoadBalancer的本地缓存，入口在DynamicServerListLoadBalancer的setServersList方法的super.setServersList(lsrv)方法处：
public class BaseLoadBalancer extends AbstractLoadBalancer implements
PrimeConnections.PrimeConnectionListener, IClientConfigAware {
@Monitor(name = PREFIX + "AllServerList", type = DataSourceType.INFORMATIONAL)
protected volatile List<Server> allServerList = Collections
.synchronizedList(new ArrayList<Server>());
public void setServersList(List lsrv) {
//写入allServerList的代码，这里略
}
@Override
public List<Server> getAllServers() {
return Collections.unmodifiableList(allServerList);
}
}
这里的getAllServers会在每个负载均衡规则中被调用，例如RoundRobinRule：
public class RoundRobinRule extends AbstractLoadBalancerRule {
public Server choose(ILoadBalancer lb, Object key) {
if (lb == null) {
log.warn("no load balancer");
return null;
}
Server server = null;
int count = 0;
while (server == null && count++ < 10) {
List<Server> reachableServers = lb.getReachableServers();
//获取服务实例列表，调用的就是刚刚提到的getAllServers
List<Server> allServers = lb.getAllServers();
int upCount = reachableServers.size();
int serverCount = allServers.size();
if ((upCount == 0) || (serverCount == 0)) {
log.warn("No up servers available from load balancer: " + lb);
return null;
}
int nextServerIndex = incrementAndGetModulo(serverCount);
server = allServers.get(nextServerIndex);
if (server == null) {
/* Transient. */
Thread.yield();
continue;
}
if (server.isAlive() && (server.isReadyToServe())) {
return (server);
}
// Next.
server = null;
}
if (count >= 10) {
log.warn("No available alive servers after 10 tries from load balancer: "
+ lb);
}
return server;
}
}
这个缓存需要注意下，有时候我们只修改了EurekaClient缓存的更新时间，但是没有修改这个LoadBalancer的刷新本地缓存时间，就是ribbon.ServerListRefreshInterval，这个参数可以设置的很小，因为没有从网络读取，就是从一个本地缓存刷到另一个本地缓存。
然后我们来看一下EurekaClient本身的缓存，直接看关键类DiscoveryClient的相关源码，我们这里只关心本地Region的，多Region配置我们先忽略：
@Singleton
public class DiscoveryClient implements EurekaClient {
//本地缓存，可以理解为是一个软链接
private final AtomicReference<Applications> localRegionApps = new AtomicReference<Applications>();
/**
* 初始化所有计划的任务
*/
private void initScheduledTasks() {
//如果配置为需要拉取服务列表，则设置定时拉取任务，这个配置默认是需要拉取服务列表
if (clientConfig.shouldFetchRegistry()) {
// registry cache refresh timer
int registryFetchIntervalSeconds = clientConfig.getRegistryFetchIntervalSeconds();
int expBackOffBound = clientConfig.getCacheRefreshExecutorExponentialBackOffBound();
cacheRefreshTask = new TimedSupervisorTask(
"cacheRefresh",
scheduler,
cacheRefreshExecutor,
registryFetchIntervalSeconds,
TimeUnit.SECONDS,
expBackOffBound,
new CacheRefreshThread()
);
scheduler.schedule(
cacheRefreshTask,
registryFetchIntervalSeconds, TimeUnit.SECONDS);
}
//其他定时任务初始化的代码，忽略
}
//定时从EurekaServer拉取服务列表的任务
class CacheRefreshThread implements Runnable {
public void run() {
refreshRegistry();
}
}
@VisibleForTesting
void refreshRegistry() {
try {
//多Region配置处理代码，忽略
boolean success = fetchRegistry(remoteRegionsModified);
if (success) {
registrySize = localRegionApps.get().size();
lastSuccessfulRegistryFetchTimestamp = System.currentTimeMillis();
}
//日志代码，忽略
} catch (Throwable e) {
logger.error("Cannot fetch registry from server", e);
}
}
//定时从EurekaServer拉取服务列表的核心方法
private boolean fetchRegistry(boolean forceFullRegistryFetch) {
Stopwatch tracer = FETCH_REGISTRY_TIMER.start();
try {
// If the delta is disabled or if it is the first time, get all
// applications
Applications applications = getApplications();
//判断，如果是第一次拉取，或者app列表为空，就进行全量拉取，否则就会进行增量拉取
if (clientConfig.shouldDisableDelta()
|| (!Strings.isNullOrEmpty(clientConfig.getRegistryRefreshSingleVipAddress()))
|| forceFullRegistryFetch
|| (applications == null)
|| (applications.getRegisteredApplications().size() == 0)
|| (applications.getVersion() == -1)) //Client application does not have latest library supporting delta
{
logger.info("Disable delta property : {}", clientConfig.shouldDisableDelta());
logger.info("Single vip registry refresh property : {}", clientConfig.getRegistryRefreshSingleVipAddress());
logger.info("Force full registry fetch : {}", forceFullRegistryFetch);
logger.info("Application is null : {}", (applications == null));
logger.info("Registered Applications size is zero : {}",
(applications.getRegisteredApplications().size() == 0));
logger.info("Application version is -1: {}", (applications.getVersion() == -1));
//全量拉取更新
getAndStoreFullRegistry();
} else {
//增量拉取更新
getAndUpdateDelta(applications);
}
applications.setAppsHashCode(applications.getReconcileHashCode());
logTotalInstances();
} catch (Throwable e) {
logger.error(PREFIX + "{} - was unable to refresh its cache! status = {}", appPathIdentifier, e.getMessage(), e);
return false;
} finally {
if (tracer != null) {
tracer.stop();
}
}
//缓存更新完成，发送个event给观察者
onCacheRefreshed();
// 检查下远端的服务实例列表里面包括自己，并且状态是否对
updateInstanceRemoteStatus();
// registry was fetched successfully, so return true
return true;
}
}
全量更新本地缓存的服务列表
getAndStoreFullRegistry方法负责全量更新，代码如下所示，非常简单的逻辑：
@Singleton
public class DiscoveryClient implements EurekaClient {
//本地缓存，可以理解为是一个软链接
private final AtomicReference<Applications> localRegionApps = new AtomicReference<Applications>();
private void getAndStoreFullRegistry() throws Throwable {
long currentUpdateGeneration = fetchRegistryGeneration.get();
logger.info("Getting all instance registry info from the eureka server");
Applications apps = null;
//由于并没有配置特别关注的region信息，
//因此会调用eurekaTransport.queryClient.getApplications方法从服务端获取服务列表
EurekaHttpResponse<Applications> httpResponse = clientConfig.getRegistryRefreshSingleVipAddress() == null
? eurekaTransport.queryClient.getApplications(remoteRegionsRef.get())
: eurekaTransport.queryClient.getVip(clientConfig.getRegistryRefreshSingleVipAddress(), remoteRegionsRef.get());
if (httpResponse.getStatusCode() == Status.OK.getStatusCode()) {
//返回对象就是服务列表
apps = httpResponse.getEntity();
}
logger.info("The response status is {}", httpResponse.getStatusCode());
if (apps == null) {
logger.error("The application is null for some reason. Not storing this information");
//考虑到多线程同步，只有CAS成功的线程，才会把自己从Eureka server获取的数据来替换本地缓存
} else if (fetchRegistryGeneration.compareAndSet(currentUpdateGeneration, currentUpdateGeneration + 1)) {
//localRegionApps就是本地缓存，是个AtomicReference实例
localRegionApps.set(this.filterAndShuffle(apps));
logger.debug("Got full registry with apps hashcode {}", apps.getAppsHashCode());
} else {
logger.warn("Not updating applications as another thread is updating it already");
}
}
}
获取服务列表信息的增量更新
获取服务列表信息的增量更新是通过getAndUpdateDelta方法完成的，具体分析请看下面的中文注释：
@Singleton
public class DiscoveryClient implements EurekaClient {
//本地缓存，可以理解为是一个软链接
private final AtomicReference<Applications> localRegionApps = new AtomicReference<Applications>();
private void getAndUpdateDelta(Applications applications) throws Throwable {
long currentUpdateGeneration = fetchRegistryGeneration.get();
Applications delta = null;
//增量信息是通过eurekaTransport.queryClient.getDelta方法完成的
EurekaHttpResponse<Applications> httpResponse = eurekaTransport.queryClient.getDelta(remoteRegionsRef.get());
if (httpResponse.getStatusCode() == Status.OK.getStatusCode()) {
//delta中保存了Eureka server返回的增量更新
delta = httpResponse.getEntity();
}
if (delta == null) {
logger.warn("The server does not allow the delta revision to be applied because it is not safe. "
+ "Hence got the full registry.");
//如果增量信息为空，就直接发起一次全量更新
getAndStoreFullRegistry();
}
//考虑到多线程同步问题，这里通过CAS来确保请求发起到现在是线程安全的，
//如果这期间fetchRegistryGeneration变了，就表示其他线程也做了类似操作，因此放弃本次响应的数据
else if (fetchRegistryGeneration.compareAndSet(currentUpdateGeneration, currentUpdateGeneration + 1)) {
logger.debug("Got delta update with apps hashcode {}", delta.getAppsHashCode());
String reconcileHashCode = "";
if (fetchRegistryUpdateLock.tryLock()) {
try {
//用Eureka返回的增量数据和本地数据做合并操作，这个方法稍后会细说
updateDelta(delta);
//用合并了增量数据之后的本地数据来生成一致性哈希码
reconcileHashCode = getReconcileHashCode(applications);
} finally {
fetchRegistryUpdateLock.unlock();
}
} else {
logger.warn("Cannot acquire update lock, aborting getAndUpdateDelta");
}
//Eureka server在返回增量更新数据时，也会返回服务端的一致性哈希码，
//理论上每次本地缓存数据经历了多次增量更新后，计算出的一致性哈希码应该是和服务端一致的，
//如果发现不一致，就证明本地缓存的服务列表信息和Eureka server不一致了，需要做一次全量更新
if (!reconcileHashCode.equals(delta.getAppsHashCode()) || clientConfig.shouldLogDeltaDiff()) {
//一致性哈希码不同，就在reconcileAndLogDifference方法中做全量更新
reconcileAndLogDifference(delta, reconcileHashCode); // this makes a remoteCall
}
} else {
logger.warn("Not updating application delta as another thread is updating it already");
logger.debug("Ignoring delta update with apps hashcode {}, as another thread is updating it already", delta.getAppsHashCode());
}
}
}
上面就是对于EurekaClient拉取服务实例信息的源代码分析：
- EurekaClient第一次全量拉取，定时增量拉取应用服务实例信息，保存在缓存中。
- EurekaClient增量拉取失败，或者增量拉取之后对比hashcode发现不一致，就会执行全量拉取，这样避免了网络某时段分片带来的问题。
- 同时对于服务调用，如果涉及到ribbon负载均衡，那么ribbon对于这个实例列表也有自己的缓存，这个缓存定时从EurekaClient的缓存更新
参考： https://blog.csdn.net/qq_40378034/article/details/103850144
https://www.processon.com/view/5d4e871ce4b04399f59f9e22
https://blog.csdn.net/Josh_scott/article/details/119150421
https://my.oschina.net/u/3747772/blog/1588958
