本文章主要结合prometheus的blackbox黑盒插件,监控拨测(ICMP_ping)主机的公网地址,并使用该数据源,通过自定义grafana的dashboards看板展示出来。当前grafana的版本号:Version 10.3.3,此处使用grafana的time-serial为例。

检查环境

1、安装并启动服务:prometheus、blackbox、grafana等;

2、确认数据源,grafana的数据库确定使用prometheus;

3、确认拨测服务已经开始采集数据,就是通过ping 公网地址方式采集数据;

新建dashboard

准备条件

1、需要监控指标和监控项:比如probe_duration_seconds,probe_icmp_duration_seconds,probe_success等,以下全部选用probe_dutation_seconds探针为例(含以下截图等),

自定义grafana-time-serial(数据源Prometheus)_icmp

自定义grafana-time-serial(数据源Prometheus)_icmp_02

自定义grafana-time-serial(数据源Prometheus)_time_serial_03

2、配置变量:再配置前需要先考虑几个变量,比如根据模块$mode--icmp和http等、组或分类$group--能源系统、应急系统等、最后一个就是实例$instance--公网地址、域名等对象。这三个变量根据实际情况增加或减少。【可以稍后根据需要添加】

新建dashboard--panel

先以time-serial为例。

自定义grafana-time-serial(数据源Prometheus)_time_serial_04

进行查询,获取想要的数据

自定义grafana-time-serial(数据源Prometheus)_black_box_05

根据步骤开始选择【先测试可以查询的数据】:

1、选择 metric----监控项,比如peobe_duration_seconds
2、选择需要监控对象的属性,比如group
3、选择多个值,通过多个值组合缩小范围
4、对选择结果生成相应的查询语句,开始运行

自定义grafana-time-serial(数据源Prometheus)_time_serial_06

自定义grafana-time-serial(数据源Prometheus)_Prometheus+grafana_07

自定义grafana-time-serial(数据源Prometheus)_time_serial_08

自定义grafana-time-serial(数据源Prometheus)_black_box_09

对生成的结果进行微调

自定义grafana-time-serial(数据源Prometheus)_Prometheus+grafana_10

自定义grafana-time-serial(数据源Prometheus)_Prometheus+grafana_11

自定义grafana-time-serial(数据源Prometheus)_icmp_12

    此例中instance是自带的内部变量,无需自定义变量。

设置相应的变量,进行筛选

  新建一个可以获取ip地址的变量 比如自定义的ip_addr,此变量仅仅是为了筛选分组等使用。

自定义grafana-time-serial(数据源Prometheus)_black_box_13

自定义grafana-time-serial(数据源Prometheus)_icmp_14

自定义grafana-time-serial(数据源Prometheus)_Prometheus+grafana_15

自定义grafana-time-serial(数据源Prometheus)_time_serial_16

调整画面显示

比如坐标轴显示、颜色实例显示、历史最大最小值等显示。期望结果如下:

自定义grafana-time-serial(数据源Prometheus)_icmp_17

 以下所有操作均在显示编辑区调整

自定义grafana-time-serial(数据源Prometheus)_icmp_18

调整显示坐标,Y轴 ICMP延时,和值的大小范围。

Axis----label:ICMP时延  Y轴名称显示

            Soft min和Soft max :Y轴值得显示范围

自定义grafana-time-serial(数据源Prometheus)_Prometheus+grafana_19

Legend----values:设置值的显示 比如最小、最大、平均、当前等值

                mode:设置以列表或表格形式显示(可以尝试后看效果)

Standard options----unit: seconds  时间单位统一

自定义grafana-time-serial(数据源Prometheus)_icmp_20

tooltip----鼠标放上去显示信息,比如 全部监控对象,以及是否排序等

自定义grafana-time-serial(数据源Prometheus)_icmp_21


以上是自定义grafana的time-serial类型的图形操作,相关的json如下:

##
{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 8,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "d1bfba55-f3f4-4b11-8530-b1e0e0ca6db0"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "ICMP时延",
            "axisPlacement": "left",
            "axisSoftMax": 10,
            "axisSoftMin": 0,
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineStyle": {
              "fill": "solid"
            },
            "lineWidth": 1,
            "pointSize": 4,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "fieldMinMax": false,
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          },
          "unit": "s",
          "unitScale": true
        },
        "overrides": []
      },
      "gridPos": {
        "h": 10,
        "w": 23,
        "x": 0,
        "y": 0
      },
      "id": 1,
      "options": {
        "legend": {
          "calcs": [
            "min",
            "max",
            "mean",
            "last"
          ],
          "displayMode": "table",
          "placement": "bottom",
          "showLegend": true,
          "sortBy": "Name",
          "sortDesc": false
        },
        "tooltip": {
          "mode": "multi",
          "sort": "desc"
        }
      },
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "d1bfba55-f3f4-4b11-8530-b1e0e0ca6db0"
          },
          "editorMode": "code",
          "expr": "probe_duration_seconds{group=\"vsr_icmp\",job=\"icmp_status\"}",
          "instant": false,
          "legendFormat": "{{ instance }}",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "拨测延时 $ip_addr",
      "type": "timeseries"
    }
  ],
  "refresh": "30s",
  "schemaVersion": 39,
  "tags": [],
  "templating": {
    "list": [
      {
        "allValue": "",
        "current": {
          "selected": true,
          "text": [
            "All"
          ],
          "value": [
            "$__all"
          ]
        },
        "datasource": {
          "type": "prometheus",
          "uid": "d1bfba55-f3f4-4b11-8530-b1e0e0ca6db0"
        },
        "definition": "label_values(probe_icmp_duration_seconds,instance)",
        "hide": 0,
        "includeAll": true,
        "label": "",
        "multi": true,
        "name": "ip_addr",
        "options": [],
        "query": {
          "qryType": 1,
          "query": "label_values(probe_icmp_duration_seconds,instance)",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-30m",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "browser",
  "title": "my_dashboard 拨测",
  "uid": "df7083ca-79e2-4a81-8efc-d020a07ba6fc",
  "version": 11,
  "weekStart": ""
}


后续会继续grafana中其他类型的展示……