Prometheus配置监控ip、端口连通，get、post接口连通和状态码

##方法：blackbox_exporter 部署在一台主机上集中配置

第一部分
1.1下载blackbox_exporter安装包
wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz

tar xf blackbox_exporter-0.19.0.linux-amd64.tar.gz
cd blackbox_exporter-0.19.0.linux-amd64/

1.2，启动配置
cat > /usr/lib/systemd/system/blackbox_exporter.service << EOF

[Unit]
Description=blackbox_exporter
After=network.target

[Service]
User=root
Type=simple
ExecStart=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

1.3 加载，启动
systemctl daemon-reload
systemctl start blackbox_exporter.service
systemctl enable blackbox_exporter.service
systemctl status blackbox_exporter.service

1.4 blackbox_exporter配置文件
blackbox.yml
位置：/…/blackbox_exporter/blackbox.yml（取决于blackbox_exporter的安装位置）

modules:
 http_2xx:
 prober: http
 http_post_2xx:
 prober: http
 http:
 method: POST
 tcp_connect:
 prober: tcp
 pop3s_banner:
 prober: tcp
 tcp:
 query_response:
 - expect: “^+OK”
 tls: true
 tls_config:
 insecure_skip_verify: false
 grpc:
 prober: grpc
 grpc:
 tls: true
 preferred_ip_protocol: “ip4”
 grpc_plain:
 prober: grpc
 grpc:
 tls: false
 service: “service1”
 ssh_banner:
 prober: tcp
 tcp:
 query_response:
 - expect: “^SSH-2.0-”
 - send: “SSH-2.0-blackbox-ssh-check”
 irc_banner:
 prober: tcp
 tcp:
 query_response:
 - send: “NICK prober”
 - send: “USER prober prober prober :prober”
 - expect: “PING 😦[^ ]+)”
 send: “PONG ${1}”
 - expect: “:[ ]+ 001”
 icmp:
 prober: icmp

上面是自带的，下面是自定义的

http_post_2xx_with_prometheus_post_check_token:
 prober: http
 http:
 method: POST
 headers:
 Content-Type: application/json #添加头部
 body: ‘{“token”:“prometheus_post_check_token”}’ #发送的相关数据第二部分
 prometheus配置
 prometheus.yml
 位置：/etc/prometheus/prometheus.yml

全局配置

global:

默认拉取频率

scrape_interval: 15s

拉取超时时间

scrape_timeout: 10s

评估规则频率

evaluation_interval: 15s

规则文件配置

rule_files: [‘/etc/prometheus/rules/*.yml’]

告警配置

alerting:
 alertmanagers:• follow_redirects: true
 scheme: http
 timeout: 10s
 api_version: v2
 static_configs: 
  • targets: []

拉取配置，添加监控项

scrape_configs:

监控prometheus

  • job_name: prometheus
    metrics_path: /metrics
    static_configs:
  • targets:
  • localhost:9090

监控ip是否能ping通，docker启动的blackbox-exporter不建议用此监控，可能会有报错

  • job_name: icmp_ping
    metrics_path: /probe
    params:
    module: [icmp]
    file_sd_configs:
  • files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/icmp_ping.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    regex: (.*)(:80)?
    target_label: __param_target
    replacement: ${1}
  • source_labels: [__param_target]
    target_label: instance
  • source_labels: [__param_target]
    regex: (.*)
    target_label: ping
    replacement: ${1}
  • source_labels: []
    regex: .*
    target_label: address
    replacement: localhost:9115

#icmp_ping.yml
位置：/etc/prometheus/conf.d/icmp_ping/icmp_ping.yml

  • targets: [‘192.168.7.254’, ‘192.168.10.200’]
    labels:
    group: ‘ping监控’

监控端口是否能连通

  • job_name: tcp_port
    metrics_path: /probe
    params:
    module: [tcp_connect]
    file_sd_configs:
  • files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/tcp_port.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    target_label: __param_target
  • source_labels: [__param_target]
    target_label: instance
  • target_label: address
    replacement: localhost:9115

#tcp_port.yml
位置：/etc/prometheus/conf.d/tcp_port/tcp_port.yml

  • targets: [‘10.120.150.12:80’, ‘10.120.150.13:80’]
    labels:
    group: ‘Nginx-端口监控’

监控get请求

    • job_name: http_get
 metrics_path: /probe
 params:
 module: [http_2xx]
 file_sd_configs: 
  • files: [‘/etc/prometheus/conf.d/http_get/*.yml’]
 refresh_interval: 10s
 relabel_configs:• source_labels: [address]
 target_label: __param_target• source_labels: [__param_target]
 target_label: instance• target_label: address
 replacement: 192.168.7.254:9115

    #http_get.yml
    位置：/etc/prometheus/conf.d/http_get/http_get.yml

    • targets:
    • targets:

    监控post请求

      • job_name: http_post
 metrics_path: /probe
 params:
 module: [http_post_2xx]
 file_sd_configs: 
  • files: [‘/etc/prometheus/conf.d/http_post/*.yml’]
 refresh_interval: 10s
 relabel_configs:• source_labels: [address]
 target_label: __param_target• source_labels: [__param_target]
 target_label: instance• target_label: address
 replacement: 192.168.7.254:9115

      #监控post请求

        • job_name: http_post_with_token
 metrics_path: /probe
 params:
 module: [http_post_2xx_with_prometheus_post_check_token]
 file_sd_configs: 
  • files: [‘/etc/prometheus/conf.d/http_post_with_token/*.yml’]
 refresh_interval: 10s
 relabel_configs:• source_labels: [address]
 target_label: __param_target• source_labels: [__param_target]
 target_label: instance• target_label: address
 replacement: 192.168.7.254:9115

        #http_post.yml
        位置：/etc/prometheus/conf.d/http_post/http_post.yml

        • targets:

        #http_post_with_token.yml
        位置：/etc/prometheus/conf.d/http_post_with_token/http_post_with_token.yml

        • targets: 
        #规则
 rules.yml
 位置：/etc/prometheus/rules/rules.ymlgroups:
• name: probe_http_status_code
 rules: 
  • alert: probe_http_status_code
 expr: probe_http_status_code != 200
 for: 1m
 labels:
 severity: critical
 annotations:
 summary: “{{ $labels.instance }} 状态码异常”
 description: “请尽快检测”
 groups:• name: probe_success
 rules: 
  • alert: probe_success
 expr: probe_success == 0
 for: 1m
 labels:
 severity: critical
 annotations:
 summary: “接口/主机/端口 {{ $labels.instance }} 无法联通”
 description: “请尽快检测”

        #模板ID 9965号模板，数据源选择Prometheus 模板下载地址
        https://grafana.com/grafana/dashboards/9965