海豚2.0.5 告警组件-HTTP试用

  • 背景
  • 测试HTTP
  • 调用手机号码归属地查询API
  • HTTP 配置
  • 问题
  • 代码优化
  • 修改后的源码
  • 最终结果
  • 总结


背景

最早用的海豚1.2.1版本,告警是通过properties文件配置的,支持Email、SMS、企业微信,2.0的时候发现SMS不见了,想着重新增加一个SMS(Short Messaging Service)短信息服务插件。网上搜索SMS,弹出一堆推广平台。基本上都是HTTP调用这些SMS平台的短信息服务API,实现消息推送。此刻恍然大悟,原来2.0告警是支持短信的,只要提供了相关API,通过HTTP调用即可实现。于是去网上找了各种SMS平台,但是短信通知全部都要企业认证,个人想测试还测不了啦。那就测试HTTP吧,研究下相关参数的用法。

都用不了!!!!!!!!!!!!

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_HTTP


Dolphinscheduler yarn 配置 dolphinscheduler 2.0_海豚调度_02

测试HTTP

调用手机号码归属地查询API

万维易源找的接口,实名认证后,免费调用,每天可以调用100次。虽然短信用不了,好在没白忙活,还有点用。

手机归属地查询接口文档

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_apache_03


例:

http://route.showapi.com/6-1?showapi_appid=952523&showapi_sign=759aa24fxxxxx66d03b6d2a6c&num=187XXXXX0

HTTP 配置

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_API_04


Dolphinscheduler yarn 配置 dolphinscheduler 2.0_API_05


Dolphinscheduler yarn 配置 dolphinscheduler 2.0_apache_06

POST
{"Content-Type":"application/json"}
{"num":"1879xxxxxx0"}
content

只要调通API,alertsend就是成功状态

问题

bodyParams配置了没效果,请求执行的时候不会使用该参数

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_告警_07

代码优化

对HttpSender进行优化,通过RequestBuilder将bodyParams添加到request请求中。
增加builder变量

private HttpUriRequest httpRequest;//lw 20220331 modify 
    private RequestBuilder builder;//lw 20220331 add

增加日志

//lw 20220331 test http alert
      log.info("http alert response msg: {},url: {},result: {}", resp,httpRequest.getURI(),builder.getParameters());

修改代码

private void createHttpRequest(String msg) {
        if (REQUEST_TYPE_POST.equals(requestType)) {
        	builder =RequestBuilder.post();//lw 20220331
            //httpRequest = new HttpPost(url);
        	addRequestParams(builder, msg);
            httpRequest = builder.setUri(url).build();
            //setHeader();
            //POST request add param in request body
            //setMsgInRequestBody(msg);
        } else if (REQUEST_TYPE_GET.equals(requestType)) {
            //GET request add param in url
            setMsgInUrl(msg);
            httpRequest = new HttpGet(url);
            setHeader();
        }
    }

增加添加参数方法

/**
     * renxiaozhao 20220331 add param for request
     * @param builder
     * @param msg
     * @throws Exception
     */
    private void addRequestParams(RequestBuilder builder, String msg) {
    	//head
    	HashMap<String, Object> map = JSONUtils.parseObject(headerParams, HashMap.class);
        for (Map.Entry<String, Object> entry : map.entrySet()) {
        	builder.setHeader(entry.getKey(), String.valueOf(entry.getValue()));
        }
        //param、entity
    	Map<String,String> bodyMap = JSONUtils.toMap(bodyParams);
    	if(!bodyMap.isEmpty()) {
    		for(Map.Entry<String,String> entry:bodyMap.entrySet()) {
    			builder.addParameter(entry.getKey(),entry.getValue());
    		}
    		builder.setEntity(new StringEntity(bodyParams, ContentType.APPLICATION_JSON));
    	}
    	//add task result detail 
    	builder.addParameter("taskrst",msg);
	}

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_HTTP_08

修改后的源码

HttpSender

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.dolphinscheduler.plugin.alert.http;

import java.util.HashMap;
import java.util.Map;

import org.apache.dolphinscheduler.alert.api.AlertResult;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;

import com.fasterxml.jackson.databind.node.ObjectNode;

public final class HttpSender {
    private static final Logger log = org.slf4j.LoggerFactory.getLogger(HttpSender.class);
    private static final String URL_SPLICE_CHAR = "?";
    /**
     * request type post
     */
    private static final String REQUEST_TYPE_POST = "POST";
    /**
     * request type get
     */
    private static final String REQUEST_TYPE_GET = "GET";
    private static final String DEFAULT_CHARSET = "utf-8";
    private final String headerParams;
    private final String bodyParams;
    private final String contentField;
    private final String requestType;
    private String url;
    private HttpUriRequest httpRequest;//lw 20220331 modify 
    private RequestBuilder builder;//lw 20220331 add 
    

    public HttpSender(Map<String, String> paramsMap) {

        url = paramsMap.get(HttpAlertConstants.URL);
        headerParams = paramsMap.get(HttpAlertConstants.HEADER_PARAMS);
        bodyParams = paramsMap.get(HttpAlertConstants.BODY_PARAMS);
        contentField = paramsMap.get(HttpAlertConstants.CONTENT_FIELD);
        requestType = paramsMap.get(HttpAlertConstants.REQUEST_TYPE);
    }

    public AlertResult send(String msg) {

        AlertResult alertResult = new AlertResult();

        createHttpRequest(msg);

        if (httpRequest == null) {
            alertResult.setStatus("false");
            alertResult.setMessage("Request types are not supported");
            return alertResult;
        }

        try {
            CloseableHttpClient httpClient = HttpClientBuilder.create().build();
            CloseableHttpResponse response = httpClient.execute(httpRequest);
            HttpEntity entity = response.getEntity();
            String resp = EntityUtils.toString(entity, DEFAULT_CHARSET);
            alertResult.setStatus("true");
            alertResult.setMessage(resp);
            //lw 20220331 test http alert
            log.info("http alert response msg: {},url: {},result: {}", resp,httpRequest.getURI(),builder.getParameters());
        } catch (Exception e) {
            log.error("send http alert msg  exception : {}", e.getMessage());
            alertResult.setStatus("false");
            alertResult.setMessage("send http request  alert fail.");
        }

        return alertResult;
    }
    
    /**
     * renxiaozhao 20220331 add param for request
     * @param builder
     * @param msg
     * @throws Exception
     */
    private void addRequestParams(RequestBuilder builder, String msg) {
    	//head
    	HashMap<String, Object> map = JSONUtils.parseObject(headerParams, HashMap.class);
        for (Map.Entry<String, Object> entry : map.entrySet()) {
        	builder.setHeader(entry.getKey(), String.valueOf(entry.getValue()));
        }
        //param、entity
    	Map<String,String> bodyMap = JSONUtils.toMap(bodyParams);
    	if(!bodyMap.isEmpty()) {
    		for(Map.Entry<String,String> entry:bodyMap.entrySet()) {
    			builder.addParameter(entry.getKey(),entry.getValue());
    		}
    		builder.setEntity(new StringEntity(bodyParams, ContentType.APPLICATION_JSON));
    	}
    	//add task result detail 
    	builder.addParameter("taskrst",msg);
	}
    
    private void createHttpRequest(String msg) {
        if (REQUEST_TYPE_POST.equals(requestType)) {
        	builder =RequestBuilder.post();//lw 20220331
            //httpRequest = new HttpPost(url);
        	addRequestParams(builder, msg);
            httpRequest = builder.setUri(url).build();
            //setHeader();
            //POST request add param in request body
            //setMsgInRequestBody(msg);
        } else if (REQUEST_TYPE_GET.equals(requestType)) {
            //GET request add param in url
            setMsgInUrl(msg);
            httpRequest = new HttpGet(url);
            setHeader();
        }
    }

    /**
     * add msg param in url
     */
    private void setMsgInUrl(String msg) {

        if (StringUtils.isNotBlank(contentField)) {
            String type = "&";
            //check splice char is & or ?
            if (!url.contains(URL_SPLICE_CHAR)) {
                type = URL_SPLICE_CHAR;
            }
            url = String.format("%s%s%s=%s", url, type, contentField, msg);
        }
    }

    /**
     * set header params
     */
    private void setHeader() {

        if (httpRequest == null) {
            return;
        }

        HashMap<String, Object> map = JSONUtils.parseObject(headerParams, HashMap.class);
        for (Map.Entry<String, Object> entry : map.entrySet()) {
            httpRequest.setHeader(entry.getKey(), String.valueOf(entry.getValue()));
        }
    }

    /**
     * set body params
     */
    private void setMsgInRequestBody(String msg) {
        ObjectNode objectNode = JSONUtils.parseObject(bodyParams);
        //set msg content field
        objectNode.put(contentField, msg);
        try {
            StringEntity entity = new StringEntity(bodyParams, ContentType.APPLICATION_JSON);
            ((HttpPost) httpRequest).setEntity(entity);
        } catch (Exception e) {
            log.error("send http alert msg  exception : {}", e.getMessage());
        }
    }
}

最终结果

Dolphinscheduler yarn 配置 dolphinscheduler 2.0_海豚调度_09

总结

1.源码只修改了POST提交方式涉及的代码,增加了bodyParams的内容,至于taskrst则是后台写死的,对应任务的详情
2.源码GET请求方式,会把contentField拼接到URL,就是taskrst的内容,同样不会拼接bodyParams,如果需要的化,体验需要添加进来
3.HTTP可以实现SMS告警,前提是有短信平台
4.通过Alertmanager实现告警