ElasticSearch常用配置（内置账号密码修改、自定义角色自定义账号，日志定期删除等)

转载

饭桶也得吃饭 2019-01-15 12:16:04 博主文章分类：日志管理

自定义内置账号

账户elastic为elasticsearch超级管理员，拥有所有权限
账户kibana用于kibana组件获取相关信息用于web展示
账户logstash_system用于logstash服务获取elasticsearch的监控数据
注意：此步骤需先启动elasticsearch服务

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/x-pack/setup-passwords interactive
Initiating the setup of reserved user elastic,kibana,logstash_system passwords.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

验证内置账户访问

若不提供用户名密码则返回401

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_cat/indices?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}

提供相应用户信息后可访问，若用户权限不足则返回403

使用logstash_system用户访问

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u logstash_system:logstash_system
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [indices:monitor/stats] is unauthorized for user [logstash_system]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用kibana用户访问

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定义角色

添加角色接口为 POST /_xpack/security/role/

下述示例为添加超级管理员角色的方法

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty' -d '{
>   "run_as": [ "elastic" ],
>   "cluster": [ "all" ],
>   "indices": [
>     {
>       "names": [ "*" ],
>       "privileges": [ "all" ]
>     }
>   ]
> }'
{
  "role" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role/admin?pretty'
{
  "admin" : {
    "cluster" : [
      "all"
    ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "all"
        ]
      }
    ],
    "run_as" : [
      "elastic"
    ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

添加自定义账户

添加用户接口为 POST /_xpack/security/user/

下述为添加martin账户并添加至admin角色操作方法

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/martin?pretty' -d '{
>   "password" : "123456",
>   "full_name" : "Martin Lei",
>   "roles" : ["admin"],
>   "email" : "martin@martin.com"
> }'
{
  "user" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/martin?pretty'
{
  "rocshen" : {
    "username" : "martin",
    "roles" : [
      "admin"
    ],
    "full_name" : "Martin Lei",
    "email" : "martin@martin.com",
    "metadata" : { },
    "enabled" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u martin:123456 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 4883 88   2.5mb   2.5mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0  24.2kb  24.2kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                        
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  630  0 703.3kb 703.3kb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0  33.3kb  33.3kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0   6.5kb   6.5kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

修改账户密码

修改密码需使用超级管理员权限即elastic账户，接口为POST _xpack/security/user//_password curl参数含义如下

-XPOST 使用post方法传递参数
-H 指定http协议的header信息
-u 指定用于认证的用户信息用户名与密码使用冒号分隔
-d 指定具体要传递的参数信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/user/kibana/_password?pretty' -d '{"password": "123456"}'
{ }

密码修改后使用老密码访问则返回401，使用更新后的密码则正常

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:kibana
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "failed to authenticate user [kibana]",
        "header" : {
          "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "failed to authenticate user [kibana]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl 'http://10.59.30.96:9200/_cat/indices?pretty' -u kibana:123456
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1    
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0    
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1    
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

配置ldap帐号认证

ldap服务安装可参考：https://segmentfault.com/a/11...

添加下述ldap相关述配置 bind_dn为ldap的管理DN

bind_password为管理dn的密码
user_search.base_dn为linux系统账户信息导入ldap的信息
user_search.attribute为账户在ldap中的标识信息
group_search.base_dn为linux系统组信息导入ldap的信息

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ vim config/elasticsearch.yml 

......

network.host: 10.59.30.96
bootstrap.system_call_filter: false

xpack.ssl.key: elasticsearch/elasticsearch.key
xpack.ssl.certificate: elasticsearch/elasticsearch.crt
xpack.ssl.certificate_authorities: ca/ca.crt
xpack.security.transport.ssl.enabled: true

xpack:
  security:
    authc:
      realms:
        ldap1:
          type: ldap
          order: 0
          url: "ldap://10.59.30.95"
          bind_dn: "cn=Manager, dc=martin, dc=com"
          bind_password: 123456
          user_search:
            base_dn: "ou=People,dc=martin,dc=com"
            attribute: uid
          group_search:
            base_dn: "ou=Group,dc=martin,dc=com"
          unmapped_groups_as_roles: false

配置AD域帐号认证 添加下ldap相关述配置至elasticsearch.yml，此处为接着上述LDAP配置添加，如果只需配置AD认证请将ldap相关配置删除即可；

domain_name为AD域的域名
url为AD域的地址
bind_dnw为随意的域账户名称（格式为user@domain）
bind_password为上述账户的密码

xpack:
  security:
    authc:
      realms:
        ldap1:
          type: ldap
          order: 0
          url: "ldap://10.59.30.94"
          bind_dn: "cn=Manager, dc=martin, dc=com"
          bind_password: 123456
          user_search:
            base_dn: "ou=People,dc=martin,dc=com"
            attribute: uid
          group_search:
            base_dn: "ou=Group,dc=martin,dc=com"
          unmapped_groups_as_roles: false
        active_directory:
          type: active_directory
          order: 1
          domain_name: martin.com
          url: ldap://ad.martin.com
          bind_dn: martin@martin.com
          bind_password: AD.123456

重启elasticsearch服务并使用ldap域账户user01登录

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ killall java
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ ./bin/elasticsearch -d
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat?pretty'
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

使用AD域账户martin登录

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl http://10.59.30.96:9200/_cat?pretty -u martin:AD.123456
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

为域账户信息映射角色

接口为：POST /_xpack/security/role_mapping/

下述为映射user1*账户为管理员角色的操作步骤

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XPOST -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty' -d '{
>   "roles": [ "admin" ],
>   "enabled": true,
>   "rules": {
>     "any": [
>       {
>         "field": {
>           "username": "/user1*/"
>         }
>       }
>     ]
>   }
> }'
{
  "role_mapping" : {
    "created" : true
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -H 'Content-type: application/json' -u elastic:elastic 'http://10.59.30.96:9200/_xpack/security/role_mapping/ldap_user_admin?pretty'
{
  "ldap_user_admin" : {
    "enabled" : true,
    "roles" : [
      "admin"
    ],
    "rules" : {
      "any" : [
        {
          "field" : {
            "username" : "/user1*/"
          }
        }
      ]
    },
    "metadata" : { }
  }
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

验证域账户权限，使用user01无权访问indices接口，使用user11可以访问；

[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user01:user01 'http://10.59.30.96:9200/_cat/indices?pretty'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [cluster:monitor/state] is unauthorized for user [user01]"
  },
  "status" : 403
}
[elasticsearch@elasticsearch elasticsearch-6.0.0]$ curl -XGET -u user11:user11 'http://10.59.30.96:9200/_cat/indices?pretty'
yellow open .monitoring-es-6-2018.01.10   nND6-i_rR5iLEYVccBGj8w 1 1 6178 44  5.9mb  5.9mb
yellow open .triggered_watches            BtygGZisSDqiL3Y2TaQGqQ 1 1    0  0 11.7kb 11.7kb
green  open .security-6                   QVRL1mcFSAilryHGEhen7Q 1 0                      
yellow open .watcher-history-6-2018.01.10 SBGiHDAnTPiXFoHU65VY_g 1 1  777  0  1.1mb  1.1mb
yellow open .watches                      kMzN4j5cQySZQQSDVPww8w 1 1    5  0 40.2kb 40.2kb
yellow open .monitoring-alerts-6          VygY6VN9R3S0PR_jrGy50Q 1 1    1  0 12.8kb 12.8kb
[elasticsearch@elasticsearch elasticsearch-6.0.0]$

ES数据定期删除

如果不删除ES数据，将会导致ES存储的数据越来越多，磁盘满了之后将无法写入新的数据。这时可以使用脚本定时删除过期数据。

#/bin/bash
#es-index-clear
#只保留15天内的日志索引
LAST_DATA=`date -d "-15 days" "+%Y.%m.%d"`
#删除上个月份所有的索引
curl -XDELETE 'http://ip:port/*-'${LAST_DATA}'*'

可以视个人情况调整保留的天数，这里的ip和port同样设置为不存储数据的那台机器。该脚本只需要在ES中一台机器定时运行即可。

crontab -e添加定时任务： 0 1 * * * /search/odin/elasticsearch/scripts/es-index-clear.sh 每天的凌晨一点清除索引。

常见报错

No subject alternative names matching IP address

[2018-01-10T19:19:35,483][WARN ][o.e.x.s.t.n.SecurityNetty4Transport] [fzP4t-4] exception caught on transport layer [[id: 0x5d97fe48, L:/0:0:0:0:0:0:0:1:49121 ! R:/0:0:0:0:0:0:0:1:9300]], closing connection
    io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
......
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found

解决方案为一种是关闭IPv6地址，另一种是修改ES_HOME/config/elasticsearch.yml中的network.host值为本机eth0的IP

原文：https://www.leiyawu.com/2018/05/07/es/

参考文档官方安装步骤：https://www.elastic.co/guide/... 配置内置账户密码： https://www.elastic.co/guide/... 修改账户密码： https://www.elastic.co/guide/... 用户相关操作： https://www.elastic.co/guide/... 使用LDAP认证： https://www.elastic.co/guide/... 用户角色映射： https://www.elastic.co/guide/...