Logstash 之 Nginx日志收集分析

一、作用

通过收集Nginx Access Log 日志，分析用户的访问量，请求状态，响应时间等数据

二、设置Nginx日志为json格式（便于logstash的快速收集）

log_format access_json '{"@timestamp":"$time_iso8601",'
                           '"host":"$server_addr",'
                           '"clientip":"$remote_addr",'
                           '"size":$body_bytes_sent,'
                           '"responsetime":$request_time,'
                           '"upstreamhost":"$upstream_addr",'
                           '"http_host":"$host",'
                           '"url":"$request",'
                           '"domain":"$host",'
                           '"referer":"$http_referer",'
                           '"status":"$status"}';
access_log  /var/log/nginx/access.log  access_json;

三、配置logstash的日志收集配置文件

input {
    file {
        path => ["/usr/local/nginx/logs/access.log"]
        type => "zhaobiao-nginx"
        codec => "json"
    }
}
output {
  elasticsearch {
        user => "elastic"
        password => "Password"
        hosts => ["http://10.224.14.14:9200","http://10.224.14.15:9200","http://10.224.14.16:9200"]
        index => "logstash-%{type}-%{+YYYY.MM.dd}"
    }
}

四、运行logstash监听NGINX日志信息

/usr/share/logstash/bin/logstash -f /usr/share/logstash/ansible-nginx.conf &

五、在Kibana上创建相关索引