elk安装包 elk安装顺序

Linux安装ELK

废话不多说，直接上手

我这次准备安装的有ElasticSearch，Kibana，filebeat，apm，metricbeat.

先下载需要的压缩包，具体下载链接自己百度，因为我这是前辈留下的😳.

两台服务器，一台作为资源服务器（监控装在这），一台项目服务器（信息采取在这）.

1. ElasticSearch安装

这里提一句，安装ELK需要首先安装jdk环境，这个很简单，网上有安装步骤，或者参考：。

1.1 解压安装包

#-C为指定解压目录
[ect@localhost software]$ tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz -C /home/ect/elasticsearch/

解压后

[ect@localhost elasticsearch]$ ll
total 568
drwxr-xr-x.  2 ect ect   4096 Jun 15 03:38 bin
drwxr-xr-x.  3 ect ect    169 Nov  6 09:48 config
drwxr-xr-x.  9 ect ect    107 Jun 15 03:38 jdk
drwxr-xr-x.  3 ect ect   4096 Jun 15 03:38 lib
-rw-r--r--.  1 ect ect  13675 Jun 15 03:34 LICENSE.txt
drwxr-xr-x.  2 ect ect      6 Jun 15 03:37 logs
drwxr-xr-x. 47 ect ect   4096 Jun 15 03:39 modules
-rw-r--r--.  1 ect ect 544318 Jun 15 03:37 NOTICE.txt
drwxr-xr-x.  2 ect ect      6 Jun 15 03:37 plugins
-rw-r--r--.  1 ect ect   8165 Jun 15 03:34 README.asciidoc

1.2 修改配置文件

[ect@localhost elasticsearch]$ vim config/elasticsearch.yml

修改内容包括node.name、path.data、path.logs、network.host、http.port等配置

node.name: node-121
path.logs: /home/ect/elasticsearch/logs
path.data: /home/ect/elasticsearch/data
network.host: 0.0.0.0
http.port: 9200

1.3 启动

[ect@localhost elasticsearch]$ ./bin/elasticsearch

1.4 处理异常

出现异常，心塞

ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ERROR: Elasticsearch did not exit normally - check the logs at /home/ect/elasticsearch/logs/elasticsearch.log

三个问题

问题1

## max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

[root@localhost ~]# vim /etc/security/limits.conf 
## 65535修改为65536
* soft nofile 65536
* hard nofile 65536
## 文件末尾追加
* soft nproc 4096
* hard nproc 4096

问题2

## max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[root@localhost ~]# vim /etc/sysctl.conf
## 在文件末尾追加：vm.max_map_count=655360
[root@localhost ~]# sysctl -p
## vm.max_map_count = 655360

问题3

## the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[ect@localhost elasticsearch]$ vim config/elasticsearch.yml
# 取消注释，并保留一个节点
cluster.initial_master_nodes: ["node-121"]

解决后，断开连接，重新登录服务器，不然问题一不起作用

测试是否启动成功

[ect@localhost elasticsearch]$ ./bin/elasticsearch

http://ip:port

success！

做个sh启动命令，后台启动！

#!/bin/bash
./bin/elasticsearch -d

1.5 设置初始密码

[ect@localhost elasticsearch]$ ./bin/elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/home/ect/java/jre] does not meet this requirement

gg，自带了jdk11和本地jdk8冲突了，该咋办嘞，我让他别用我本地jdk，就用他自带的

[ect@localhost elasticsearch]$ vim bin/elasticsearch-env
###能发现这一段
# now set the path to java
if [ ! -z "$JAVA_HOME" ]; then
  JAVA="$JAVA_HOME/bin/java"
  JAVA_TYPE="JAVA_HOME"
else
  if [ "$(uname -s)" = "Darwin" ]; then
    # macOS has a different structure
    JAVA="$ES_HOME/jdk.app/Contents/Home/bin/java"
  else
    JAVA="$ES_HOME/jdk/bin/java"
  fi
  JAVA_TYPE="bundled jdk"
fi
###大概意思就是本地有全局jdk就用全局的，没有就用自带的，我来个偷梁换柱，不管有没有，全用es自带的（其他服务器不知道，windows和Linux应该是没问题的）
if [ ! -z "$JAVA_HOME" ]; then
  JAVA="$ES_HOME/jdk/bin/java"
  #JAVA="$JAVA_HOME/bin/java"
  JAVA_TYPE="JAVA_HOME"
else
  if [ "$(uname -s)" = "Darwin" ]; then
    # macOS has a different structure
    JAVA="$ES_HOME/jdk.app/Contents/Home/bin/java"
  else
    JAVA="$ES_HOME/jdk/bin/java"
  fi
  JAVA_TYPE="bundled jdk"
fi

得，还需要加配置

Unexpected response code [500] from calling GET http://10.4.60.121:9200/_security/_authenticate?pretty
It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.
Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.

ERROR: X-Pack Security is disabled by configuration.

再在elasticsearch.yml添加上这段，然后重启，重启后莫急，稍等一会

xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true

然后就是愉快的设置密码了

[ect@localhost elasticsearch]$ ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana_system]: 
Reenter password for [kibana_system]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

2. kibana安装

2.1 同款解压

2.2 修改配置文件

根据实际情况设置server.port、server.host、server.name、elasticsearch.hosts、elasticsearch.username、elasticsearch.password、i18n.locale等配置

搞这个有点迷，一直以为没部署成功，结果好像一直是地址输错了，贴出我elasticsearch.yml和kibana.yml

elasticsearch.yml；

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: node-121
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-121
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /home/ect/elasticsearch/data
#
# Path to log files:
#
path.logs: /home/ect/elasticsearch/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["127.0.0.1"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-121"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
http.cors.enabled: true
http.cors.allow-origin: http://localhost:5601

kibana.yml；

# Kibana is served by a back end server. This setting specifies the port to use.
#端口
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#地址
server.host: "10.4.60.121"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name.  This is used for display purposes.
#服务名称
server.name: "CT"

# The URLs of the Elasticsearch instances to use for all your queries.
#elasticsearch地址
elasticsearch.hosts: ["http://10.4.60.121:9200"]


# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch用户名和密码
elasticsearch.username: "elastic"
elasticsearch.password: "elastic"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000

# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false

# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid

# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
i18n.locale: "zh-CN"
xpack.security.encryptionKey: "122333444455555666666777777788888888"
xpack.reporting.encryptionKey: "122333444455555666666777777788888888"
xpack.encryptedSavedObjects.encryptionKey: "122333444455555666666777777788888888"

后面有机会补一次

3. apm安装

apm是项目监控，安装相对简单

3.1 同款解压

3.2 修改apm-server.yml

修改elasticsearch地址，及将账号密码填入

apm-server:
  host: "10.4.60.121:8200"
output.elasticsearch:
  hosts: ["localhost:9200"]
  username: "elastic"
  password: "elastic"

3.3 启动

后台启动

nohub ./apm-server -e >> /dev/null 2>&1 &

查看端口是否在运行

[ect@localhost apm-server-7.8.0-linux-x86_64]$ netstat -anp | grep 8200

3.4 下载apm-agent.jar

https://search.maven.org/#search%7Cga%7C1%7Ca%3Aelastic-apm-agent

这个需要放到需要启动的项目同服务器下，我将这个jar放到根目录下

3.5 项目启动

在项目启动命令中添加apm的参数

-javaagent:/home/ect/elastic-apm-agent-1.12.0.jar -Delastic.apm.service_name=eureka-java-application -Delastic.apm.server_url=http://10.4.60.121:8200 -Delastic.apm.secret_token= -Delastic.apm.application_packages=org.example

如

java -javaagent:/home/ect/elastic-apm-agent-1.12.0.jar -Delastic.apm.service_name=eureka-java-application -Delastic.apm.server_url=http://10.4.60.121:8200 -Delastic.apm.secret_token=  -Delastic.apm.application_packages=org.example  -Xms1024m -Xmx1024m -Xss256k -XX:NewSize=768m -XX:MaxNewSize=768m -XX:MetaspaceSize=128m -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -jar eureka.jar > /dev/null 2>&1 &

显示这个就说明已经成功监控到了

2020-11-09 14:06:34.429 [apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: {  "build_date": "2020-06-14T17:10:16Z",  "build_sha": "06c58bf4e5b675d04314bf44961ffd6b0e13f544",  "version": "7.8.0"}

4. metricbeat安装

4.1 同款解压

4.2 修改metricbeat.yml

output.elasticsearch:
  hosts: ["<es_url>"]
  username: "elastic"
  password: "<password>"
setup.kibana:
  host: "<kibana_url>"

4.3 启动你想监控的部分

./metricbeat modules enable system

如服务器系统监控

4.4 启动metricbeat

./metricbeat setup
./metricbeat -e

5 filebeat安装

不想写了，和metricbeat一模一样

“build_date”: “2020-06-14T17:10:16Z”, “build_sha”: “06c58bf4e5b675d04314bf44961ffd6b0e13f544”, “version”: “7.8.0”}

### 4. metricbeat安装

#### 4.1 同款解压

#### 4.2  修改metricbeat.yml

```shell
output.elasticsearch:
  hosts: ["<es_url>"]
  username: "elastic"
  password: "<password>"
setup.kibana:
  host: "<kibana_url>"

4.3 启动你想监控的部分

./metricbeat modules enable system

如服务器系统监控

4.4 启动metricbeat

./metricbeat setup
./metricbeat -e

5 filebeat安装

不想写了，和metricbeat一模一样