es7 集群配置账号密码 es7集群部署

转载

mob64ca14196783 2024-05-22 16:19:38

文章标签 es7 集群配置账号密码 elasticsearch elastic JAVA 文章分类 架构后端开发

es7.12集群部署
环境:
192.168.56.111
192.168.56.112
192.168.56.113

--------------------------------------------基础安装-----------------------------------

系统配置

每个机器上都要执行

系统参数配置
修改limits.conf配置文件
vi /etc/security/limits.conf
root用户下添加如下2两项,然后退出使用elasticsearch用户登陆,使其生效
*        hard    nofile           65536
*        soft    nofile           65536

修改sysctl.conf文件
vi /etc/sysctl.conf
vm.max_map_count=262144

然后执行如下命令:
[root@localhost ~]# sysctl -p

关闭防火墙
systemctl status firewalld.service
systemctl stop firewalld.service
systemctl disable firewalld.service

1.安装java
安装连接:
确保java版本在1.8以上

[root@rac01 soft]# java -version
java version "1.8.0_291"
Java(TM) SE Runtime Environment (build 1.8.0_291-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.291-b10, mixed mode)2.下载需要的安装版本
我这里下载的是7.12.1,elasticsearch-7.12.1-linux-x86_64.tar.gz
下载地址:
https://www.elastic.co/cn/downloads/past-releases#elasticsearch 
3.创建中间件安装目录和数据文件、日志文件目录
每台机器上都要执行
[root@es soft]# mkdir -p /usr/local/services
[root@es soft]# mkdir -p /home/middle/elasticsearch7/data
[root@es soft]# mkdir -p /home/middle/elasticsearch7/logs 
3.创建用户和用户组
每台机器上都要执行
groupadd -g 1500 elasticsearch
useradd -u 1500 -g elasticsearch elasticsearch
passwd elasticsearch 
4.上传到服务器
每台机器上都要执行
解压缩并创建数据目录
[root@rac01 soft]# cd /soft
[root@rac01 soft]# tar -xvf elasticsearch-7.12.1-linux-x86_64.tar.gz
[root@rac01 soft]# mv elasticsearch-7.12.1 /usr/local/services/elasticsearch7 
5.将elasticsearch目录权限修改为elasticsearch
每台机器上都要执行
[root@es config]# cd /usr/local/services
[root@es services]# chown -R elasticsearch.elasticsearch ./elasticsearch7同时修改数据文件和日志文件目录给到elasticsearch
[root@es services]# cd /home/middle
[root@es middle]#chown -R elasticsearch.elasticsearch ./elasticsearch7 
6.创建备份目录
每台机器上都要执行
[root@rac01 home]#mkdir -p /home/middle/esbak7
[root@rac01 home]#cd /home/middle
[root@rac01 home]#chown -R elasticsearch.elasticsearch ./esbak7 
7.修改配置文件
每台机器上都要执行
[root@rac01 middle]# su - elasticsearch
[elasticsearch@rac01 ~]$ cd /usr/local/services/elasticsearch7/config
[elasticsearch@es config]$ vi elasticsearch.ymlcluster.name: escluster_ysd
node.name: node-111
path.data: /home/middle/elasticsearch7/data
path.logs: /home/middle/elasticsearch7/logs
network.host: 192.168.1.111
http.port: 19200
discovery.zen.minimum_master_nodes: 2
discovery.seed_hosts: ["192.168.56.111", "192.168.56.112","192.168.56.113"]
cluster.initial_master_nodes: ["node-111","node-112","node-113"]
path.repo: /home/middle/esbak7
http.cors.enabled: true
http.cors.allow-origin: "*"其他机器的配置文件:
将配置文件scp到另外的机器,然后相应修改红色部分
node.name分别修改为node_112和node_113
network.host分别修改为对应机器的ip地址 
8.修改jvm参数(/usr/local/services/elasticsearch7/config/jvm.options)
每台机器上都要执行
 
## JVM configuration
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space-Xms4g
-Xmx4g9.修改/usr/local/services/elasticsearch7/bin/elasticsearch(已经不生效,可以不配置)
#   ES_JAVA_OPTS="-Xms8g -Xmx8g" ./bin/elasticsearch
export ES_HEAP_SIZE=4g10.启动
每个节点上都要执行,这里确保每台机器都能启动
[root@rac01 middle]# su - elasticsearch
[elasticsearch@es ~]$ cd /usr/local/services/elasticsearch7/bin
./elasticsearch -d[elasticsearch@rac02 bin]$ ./elasticsearch -d
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/java/jdk1.8.0_291/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/java/jdk1.8.0_291/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.这里提示是不需要操作系统的java了,使用es自带的jkd也可以启动的
curl 'http://192.168.56.111:19200/_cat/nodes?v'
curl http://192.168.56.111:19200/?pretty
curl http://192.168.56.112:19200/?pretty
curl -X GET 'http://192.168.56.111:19200/_cat/indices?v'----------------------------配置集群内部通信--------------------------------
1.生成证书
1.执行命令创建ca 执行:
su - elasticsearch
[elasticsearch@rac01 bin]$ cd /usr/local/services/elasticsearch7/bin
[elasticsearch@rac01 bin]$ ./elasticsearch-certutil ca
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/java/jdk1.8.0_291/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authorityBy default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private keyIf you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private keyPlease enter the desired output file [elastic-stack-ca.p12]:      ##直接回车
Enter password for elastic-stack-ca.p12 :   ##直接回车这个时候会生成elastic-stack-ca.p12文件
[elasticsearch@rac01 elasticsearch7]$ pwd
/usr/local/services/elasticsearch7
[elasticsearch@rac01 elasticsearch7]$ ls -1
bin
config
elastic-stack-ca.p12
jdk
lib
LICENSE.txt
logs
modules
NOTICE.txt
plugins
README.asciidoc然后按照提示输入Please enter the desired output file [elastic-stack-ca.p12] 此时提示输入文件名默认为:elastic-stack-ca.p12,输入完敲回车，或者直接回车默认。
接下来会提示输入Enter password for elastic-stack-ca.p12 :密码可以为空 直接回车 此时ca 创建OK 文件会在执行目录的根目录2.根据elastic-stack-ca.p12文件 生成elastic-certificates.p12
执行命令为:elasticsearch-certutil cert --ca elastic-stack-ca.p12[elasticsearch@rac01 bin]$./elasticsearch-certutil cert --ca elastic-stack-ca.p12
Enter password for CA (elastic-stack-ca.p12) : 
Please enter the desired output file [elastic-certificates.p12]: 
Enter password for elastic-certificates.p12 : 接下来会提示 输入Enter password for CA (elastic-stack-ca.p12) :上一个ca 文件的密码 如果没有则直接回车即可，
接下来会提示Please enter the desired output file [elastic-certificates.p12]:给当前生成的文件取名默认为elastic-certificates.p12
接下来会提示给当前文件设置密码Enter password for elastic-certificates.p12 : 设置完成后回车。
至此我们有了elastic-stack-ca.p12和elastic-certificates.p12两个文件将这两个文件拷贝到config目录下面
[elasticsearch@rac01 elasticsearch7]$ cd /usr/local/services/elasticsearch7
[elasticsearch@rac01 elasticsearch7]$ mv elastic-certificates.p12 ./config/
[elasticsearch@rac01 elasticsearch7]$ mv elastic-stack-ca.p12 ./config/3.将节点1上的两个文件拷贝到另外的节点
[elasticsearch@rac01 elasticsearch7]$ cd /usr/local/services/elasticsearch7/config
[elasticsearch@rac01 elasticsearch7]$ scp elastic-certificates.p12 192.168.56.112:/usr/local/services/elasticsearch7/config/
[elasticsearch@rac01 elasticsearch7]$ scp elastic-stack-ca.p12 192.168.56.112:/usr/local/services/elasticsearch7/config/[elasticsearch@rac01 elasticsearch7]$ scp elastic-certificates.p12 192.168.56.113:/usr/local/services/elasticsearch7/
[elasticsearch@rac01 elasticsearch7]$ scp elastic-stack-ca.p12 192.168.56.113:/usr/local/services/elasticsearch7/ 
4.修改配置文件
每台机器上的配置文件在最后面添加如下内容:[root@rac01 middle]# su - elasticsearch
vi /usr/local/services/elasticsearch7/config/elasticsearch.yml
添加如下配置项
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p125.重新启动
将原来的进程杀掉后重新启动
kill -9 进程号[root@rac01 middle]# su - elasticsearch
[elasticsearch@es ~]$ cd /usr/local/services/elasticsearch7/bin
./elasticsearch -d这个时候使用就需要密码访问了
curl 'http://192.168.56.111:19200/_cat/nodes?pretty'[elasticsearch@rac01 bin]$ curl 'http://192.168.56.111:19200/_cat/nodes?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication token for REST request [/_cat/nodes?pretty]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_cat/nodes?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}下面进行密码设置
6.设置密码
在其中一台机器上执行,我这里在 192.168.56.111 这台机器上执行,我这里密码全部设置为elastic
[elasticsearch@rac01 bin]$ cd /usr/local/services/elasticsearch7/bin
[elasticsearch@rac01 bin]$ ./elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]yEnter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
[elasticsearch@rac01 bin]$ 17.验证
curl -u elastic:elastic 'http://192.168.56.111:19200/_cat/nodes?v'
curl -u elastic:elastic 'http://192.168.56.112:19200/_cat/nodes?v'
curl -u elastic:elastic 'http://192.168.56.113:19200/_cat/nodes?v'
curl -u elastic:elastic 'http://192.168.56.111:19200/_cat/health?v'18.数据验证
查看索引:
curl -u elastic:elastic -X GET 'http://192.168.56.111:19200/_cat/indices?v'在节点1上创建索引和写入数据
curl -u elastic:elastic -XPUT 'http://192.168.56.111:19200/db_customer'
curl -u elastic:elastic -H "Content-Type: application/json" -XPUT 'http://192.168.56.111:19200/db_customer/tb_test/1' -d '{"name": "huangxueliang"}'查看数据
curl -u elastic:elastic -XGET 'http://192.168.56.111:19200/db_customer/tb_test/1?pretty'在其他的节点上查看该数据
curl -u elastic:elastic -XGET 'http://192.168.56.112:19200/db_customer/tb_test/1?pretty'
curl -u elastic:elastic -XGET 'http://192.168.56.113:19200/db_customer/tb_test/1?pretty' 
es7之后以及没有type的概念,所有的type都是_doc表示,下面的查询也可以查到数据
curl -u elastic:elastic -XGET 'http://192.168.56.112:19200/db_customer/_doc/1?pretty'
 
------------------------配置备份-------------------------------------------------------
--------服务端安装----------------
1.在做备份的机器上安装nfs 服务端
[root@rac01 ios]# yum install -y nfs-utils2.配置输出
$ more /etc/exports
/home/middle/esbak7 192.168.56.112(insecure,rw,no_root_squash,sync,anonuid=1500,anongid=1500)
/home/middle/esbak7 192.168.56.113(insecure,rw,no_root_squash,sync,anonuid=1500,anongid=1500)3.启动服务
先为rpcbind和nfs做开机启动：(必须先启动rpcbind服务)
[root@rac01 ios]# systemctl enable rpcbind.service
[root@rac01 ios]# systemctl enable nfs-server.service
然后分别启动rpcbind和nfs服务：
systemctl start rpcbind.service
systemctl start nfs-server.servicesystemctl restart rpcbind.service
systemctl restart nfs-server.service4.检查是否生效
配置生效
exportfs -r
exportfs---------客户端安装---------
首先是安裝nfs，同上，然后启动rpcbind服务
[root@rac02 ios]# yum install -y nfs-utils先为rpcbind做开机启动：
[root@rac02 ios]# systemctl enable rpcbind.service然后启动rpcbind服务：
[root@rac02 ios]# systemctl start rpcbind.service
注意:客户端不需要启动nfs服务检查 NFS 服务器端是否有目录共享：showmount -e nfs服务器的IP
showmount -e 192.168.56.111
Export list for 192.168.56.111:
/home/middle/esbak 192.168.56.113,192.168.56.112mount到指定的目录
另外的2个节点执行如下命令:
mount -t nfs -o proto=tcp -o nolock 192.168.56.111:/home/middle/esbak7 /home/middle/esbak7使用 elasticsearch 用户看是否可以写入数据
[root@rac02 ios]# su - elasticsearch
[elasticsearch@rac02 esbak]$ cd /home/middle/esbak
[elasticsearch@rac02 esbak]$ echo "112">aa.txt另外一台客户端
[elasticsearch@rac02 esbak]$ echo "113">bb.txt这个时候在任何一个节点都会看到上面创建的两个文件,同时也可以进行编辑
开始备份,备份其中一个节点(在nfs服务那台机器上)执行即可
curl -u elastic:elastic -H "Content-Type: application/json" -XPUT http://192.168.56.111:19200/_snapshot/esbackup -d'{
    "type": "fs",
    "settings": {
        "location": "/home/middle/esbak7"
    }
}'##备份
curl -u elastic:elastic -H "Content-Type: application/json" -XPUT http://192.168.56.111:19200/_snapshot/esbackup/snapshot_20210518查看备份设置
curl -u elastic:elastic -X GET "192.168.56.111:19200/_snapshot/esbackup?pretty"
查看所有的备份
curl -u elastic:elastic -X GET "192.168.56.111:19200/_snapshot/esbackup/_all?pretty"
curl -u elastic:elastic -X GET "192.168.56.112:19200/_snapshot/esbackup/_all?pretty"删除快照
curl -u elastic:elastic -X DELETE "192.168.56.111:19200/_snapshot/esbackup/snapshot_20210517"

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。