目的:通过filebeat获取nginx日志,发送给ElasticSearch,filebeat可以解析json格式日志,所以设置nginx日志为json格式。
1、配置nginx配置文件
log_format jsonTest '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"service":"nginxTest",' '"trace":"$upstream_http_ctx_transaction_id",' '"log":"log",' '"clientip":"$remote_addr",' '"remote_user":"$remote_user",' '"request":"$request",' '"http_user_agent":"$http_user_agent",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"url":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"status":"$status"}'; access_log /var/log/nginx/access.log jsonTest;
定义jsonTest的json格式,其中trace是页面response headers的值 ctx-transaction-id,通过upstream_http_ctx_transaction_id可以获取头文件属性。
将日志输出到/var/log/nginx/access.log
2、配置filebeat配置文件
filebeat.prospectors: - type: log paths: - '/root/front/logs/*.log' json.message_key: log json.keys_under_root: true output.elasticsearch: hosts: ["*.*.*.*:9200"]
将日志传输给ElasticSearch