1 概述

ELK套件(ELK stack)是指ElasticSearch、Logstash和Kibana三件套。这三个软件可以组成一套日志分析和监控工具。

由于三个软件各自的版本号太多,建议采用ElasticSearch官网推荐的搭配组合:http://www.elasticsearch.org/overview/elkdownloads/

2 环境准备

2.1 软件要求

具体的版本要求如下:

  • 操作系统版本:CentOS 6.4;
  • JDK版本:1.7.0;
  • Logstash版本:1.4.2;
  • ElasticSearch版本:1.4.2;
  • Kibana版本:3.1.2;

2.2 防火墙配置

为了正常使用HTTP服务等,需要关闭防火墙:

[plain] view plain copy 
1. # service iptables stop

或者可以不关闭防火墙,但是要在iptables中打开相关的端口:

[plain] view plain copy 
1. # vim /etc/sysconfig/iptables  
2. -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT  
3. -A INPUT -m state --state NEW -m tcp -p tcp --dport 9200 -j ACCEPT  
4. -A INPUT -m state --state NEW -m tcp -p tcp --dport 9292 -j ACCEPT  
5. # service iptables restart

3 安装JDK

ElasticSearch和Logstash依赖于JDK,所以需要安装JDK:

[plain] view plain copy 
1. # yum -y install java-1.7.0-openjdk*  
2. # java -version

4 安装ElasticSearch

ElasticSearch默认的对外服务的HTTP端口是9200,节点间交互的TCP端口是9300。

下载ElasticSearch:

[plain] view plain copy
1. # mkdir -p /opt/software && cd /opt/software  
2. # sudo wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.tar.gz  
3. # sudo tar -zxvf elasticsearch-1.4.2.tar.gz -C /usr/local/  
4. # ln -s /usr/local/elasticsearch-1.4.2 /usr/local/elasticsearch

安装elasticsearch-servicewrapper,并启动ElasticSearch服务:

[plain] view plain copy
1. # sudo wget https://github.com/elasticsearch/elasticsearch-servicewrapper/archive/master.tar.gz  
2. # sudo tar -zxvf master  
3. # mv /opt/software/elasticsearch-servicewrapper-master/service /usr/local/elasticsearch/bin/  
4. # /usr/local/elasticsearch/bin/service/elasticsearch start

测试ElasticSearch服务是否正常,预期返回200的状态码:

[plain] view plain copy
1. # curl -X GET http://localhost:9200

5 安装Logstash

Logstash默认的对外服务的端口是9292。

下载Logstash:

[plain] view plain copy
1. # sudo wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz  
2. # sudo tar -zxvf logstash-1.4.2.tar.gz -C /usr/local/  
3. # ln -s /usr/local/logstash-1.4.2 /usr/local/logstash

简单测试Logstash服务是否正常,预期可以将输入内容以简单的日志形式打印在界面上:

[plain] view plain copy
1. # /usr/local/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'

创建Logstash配置文件,并再次测试Logstash服务是否正常,预期可以将输入内容以结构化的日志形式打印在界面上:

[plain] view plain copy
1. # mkdir -p /usr/local/logstash/etc  
2. # vim /usr/local/logstash/etc/hello_search.conf  
3. input {  
4.   stdin {  
5.     type => "human"  
6.   }  
7. }  
8.   
9. output {  
10.   stdout {  
11.     codec => rubydebug  
12.   }  
13.   
14.   elasticsearch {  
15.     host => "10.111.121.22"  
16.     port => 9300  
17.   }  
18. }  
19. # /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/hello_search.conf

6 安装Kibana

CentOS默认预装了Apache,所以将Kibana的代码直接拷贝到Apache可以访问的目录下即可。

[plain] view plain copy
1. # sudo wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz  
2. # sudo tar -zxvf kibana-3.1.2.tar.gz  
3. # mv kibana-3.1.2 /var/www/html/kibana

修改Kibana的配置文件,把elasticsearch所在行的内容替换成如下:

[plain] view plain copy
1. # vim /var/www/html/kibana/config.js  
2. elasticsearch: "http://10.111.121.22:9200",

启动一下HTTP服务:

[plain] view plain copy
1. # service httpd start

修改ElasticSearch的配置文件,追加一行内容,并重启ElasticSearch服务:

[plain] view plain copy

1. # vim /usr/local/elasticsearch/config/elasticsearch.yml  
2. http.cors.enabled: true  
3. # /usr/local/elasticsearch/bin/service/elasticsearch restart

然后就可以通过浏览器访问Kibana了:

[plain] view plain copy

  1. http://10.111.121.22/kibana  

现在,在之前的Logstash会话中输入任意字符,就可以在Kibana中查看到日志情况。

7 配置Logstash

再次创建Logstash配置文件,这里将HTTP日志和文件系统日志作为输入,输出直接传给ElasticSearch,不再打印在界面上:

[plain] view plain copy
1. # vim /usr/local/logstash/etc/logstash_agent.conf  
2. 
3. input {
4.          file {
5.                 type => "http.access"
6.                 path => ["/var/log/httpd/access_log"]
7.         }
8. 
9.         file {
10.                 type => "http.error"
11.                 path => ["/var/log/httpd/error_log"]
12.         }
13. 
14.         file {
15.                 type => "messages"
16.                 path => ["/var/log/messages"]
17.         }
18. }
19. 
20. output {
21.         elasticsearch {
22.                 host => "123.206.211.52"
23.                 port => 9300
24.         }
25. } 
26. #/usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/logstash_agent.conf &

现在,一个简单的日志分析和监控平台就搭建好了,可以使用Kibana进行查看。


转载于:https://blog.51cto.com/joy1991/1903930