Soul网关 ---- HTTP 长轮询同步数据
HTTP长轮询机制:
soul-bootstrap
请求 soul-admin
的配置服务(/configs/listener
接口),soul-bootstrap
设置了读取超时时间为 90s,意味着soul-bootstrap
请求soul-admin
的配置服务(/configs/listener
接口)最多会等待 90s,这样便于 soul-admin
及时响应变更数据,从而实现准实时推送。
soul-bootstrap
通过HTTP
长轮询soul-admin
的 /configs/listener
接口
soul-bootstrap
中引入了 soul-spring-boot-starter-sync-data-http
的依赖,通过 http
长轮询 soul-admin
进行数据的同步。
soul-spring-boot-starter-sync-data-http
的自动配置类HttpSyncDataConfiguration
注入了一个 HttpSyncDataService
bean。
/**
* Http 长轮询数据同步服务
* 初始化bean时,从 spring bean 工厂里找到下面这些bean,当这个@ConfigurationProperties(prefix = "soul.sync.http")配置为开启时,就会自动装载 httpConfig 这个bean,所以可以在 soul-bootstrap的配置中通过soul.sync.http来设置连接超时时间之类的
* @param httpConfig Http配置(包括请求地址,延迟时间,连接超时时间),
* @param pluginSubscriber the plugin subscriber
* @param metaSubscribers the meta subscribers
* @param authSubscribers the auth subscribers
* @return the sync data service
*/
@Bean
public SyncDataService httpSyncDataService(final ObjectProvider<HttpConfig> httpConfig, final ObjectProvider<PluginDataSubscriber> pluginSubscriber,
final ObjectProvider<List<MetaDataSubscriber>> metaSubscribers, final ObjectProvider<List<AuthDataSubscriber>> authSubscribers) {
log.info("you use http long pull sync soul data");
// 创建了 HttpSyncDataService
return new HttpSyncDataService(Objects.requireNonNull(httpConfig.getIfAvailable()), Objects.requireNonNull(pluginSubscriber.getIfAvailable()),
metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
}
HttpSyncDataService
中创建了一个线程,这个线程会通过 HttpLongPollingTask
任务来长轮询 soul-admin
(/configs/listener
接口)
// 专门启动一个线程来做长轮询
private void start() {
// It could be initialized multiple times, so you need to control that.
if (RUNNING.compareAndSet(false, true)) {
// fetch all group configs.
this.fetchGroupConfig(ConfigGroupEnum.values());
int threadSize = serverList.size();
this.executor = new ThreadPoolExecutor(threadSize, threadSize, 60L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(),
SoulThreadFactory.create("http-long-polling", true));
// 通过 HttpLongPollingTask 任务开启长轮询 each server creates a thread to listen for changes.
this.serverList.forEach(server -> this.executor.execute(new HttpLongPollingTask(server)));
} else {
log.info("soul http long polling was started, executor=[{}]", executor);
}
}
通过 HttpClient
向soul-admin
发起请求:
// HTTP POST 请求 soul-amin
String json = this.httpClient.postForEntity(listenerUrl, httpEntity, String.class).getBody();
在 soul-bootstrap
的配置中通过soul.sync.http
来设置连接超时时间之类的,当自动装载bean
: httpConfig
时就会初始化配置好的HTTP
配置
/**
* Http 配置的自动装载
* @return the http config
*/
@Bean
@ConfigurationProperties(prefix = "soul.sync.http")
public HttpConfig httpConfig() {
return new HttpConfig();
}
soul-admin
处理soul-bootstrap
过来的HTTP
长轮询
soul-bootstrap
发起的 Http
请求到达soul-admin
之后, soul-admin
并非立马响应数据,而是利用Servlet3.0
的异步机制,异步响应数据。首先,将长轮询请求任务 LongPollingClient
扔到 BlocingQueue
中,并且开启调度任务ScheduledThreadPoolExecutor
,60s 后执行,这样做的目的是 60s 后将该长轮询请求移出BlocingQueue
队列,因为即便这段时间内没有发生配置数据变更,也得给soul-bootstrap
以响应(soul-bootstrap
请求soul-admin
的/configs/listener
接口时,也可以设置超时时间)
soul-admin
的配置类中条件初始化bean:
/**
* HTTP 长轮询,当soul.sync.http.enabled配置开启时,会自动装载bean:HttpLongPollingDataChangedListener
*/
@Configuration
@ConditionalOnProperty(name = "soul.sync.http.enabled", havingValue = "true")
@EnableConfigurationProperties(HttpSyncProperties.class)
static class HttpLongPollingListener {
@Bean
@ConditionalOnMissingBean(HttpLongPollingDataChangedListener.class)
public HttpLongPollingDataChangedListener httpLongPollingDataChangedListener(final HttpSyncProperties httpSyncProperties) {
return new HttpLongPollingDataChangedListener(httpSyncProperties);
}
}
Http长轮询数据变化监听器初始化创建了一个大小为 1024 的BlockingQueue
来存放「长轮询请求任务 LongPollingClient
」,
/**
* 初始化 Http长轮询数据变化监听器
* @param httpSyncProperties the HttpSyncProperties
*/
public HttpLongPollingDataChangedListener(final HttpSyncProperties httpSyncProperties) {
this.clients = new ArrayBlockingQueue<>(1024);
this.scheduler = new ScheduledThreadPoolExecutor(1,
SoulThreadFactory.create("long-polling", true));
this.httpSyncProperties = httpSyncProperties;
}
创建了一个带延迟队列的线程池来将 长轮询请求 移出队列BlocingQueue
:
// 核心线程数给了 1 个
this.scheduler = new ScheduledThreadPoolExecutor(1,
SoulThreadFactory.create("long-polling", true));
public ScheduledThreadPoolExecutor(int corePoolSize,
ThreadFactory threadFactory) {
super(corePoolSize, Integer.MAX_VALUE, 0, NANOSECONDS,
new DelayedWorkQueue(), threadFactory);
}
doLongPolling
利用 Servlet3.0
的异步机制,异步响应数据:
// 处理soul-bootstrap过来的请求
public void doLongPolling(final HttpServletRequest request, final HttpServletResponse response) {
// compare group md5
List<ConfigGroupEnum> changedGroup = compareChangedGroup(request);
String clientIp = getRemoteIp(request);
// 如果配置数据有改动,则立即响应
if (CollectionUtils.isNotEmpty(changedGroup)) {
this.generateResponse(response, changedGroup);
log.info("send response with the changed group, ip={}, group={}", clientIp, changedGroup);
return;
}
// 监听配置数据的变化
final AsyncContext asyncContext = request.startAsync();
// AsyncContext.settimeout() does not timeout properly, so you have to control it yourself
asyncContext.setTimeout(0L);
// 调度任务 60 秒后会将请求移出BlocingQueue队列
scheduler.execute(new LongPollingClient(asyncContext, clientIp, HttpConstants.SERVER_MAX_HOLD_TIMEOUT));
}