HDFS Kerberos配置 hdfs kerberos认证

Kerberos原理介绍

    kerberos主要是用来做网络通信时候的身份认证，最主要的特点就是“复杂”。所以在入坑kerberos之前，最好先熟悉一下其原理。这里推荐一些别人写的文章内容来进行简单汇总：

1.链接：
 kerberos认证原理

 用对话场景来解释kerbeors的设计过程

 Kerberos的组件和术语(翻译和注解)

2.Kerberos认证流程：

 KDC服务安装及配置

本文档中将KDC服务安装在worker3节点服务器上（KDC服务可根据自己需要安装在其他服务器，最好安装在Cloudera Manager Server所在的服务器上）

1.在worker3节点上安装KDC服务

[root@worker3 ~]# yum -y install krb5-serverkrb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置(标红部分为需要修改的信息)

[root@worker3 ~]# vim /etc/krb5.conf 

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 dns_lookup_kdc = false
 clockskew = 600
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = WANWEI.COM
#  default_ccache_name = KEYRING:persistent:%{uid}

[realms]

 WANWEI.COM = {
  kdc = worker3
  admin_server = worker3
 }

[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM

 .worker3 = WANWEI.COM
 worker3 = WANWEI.COM

这个配置kdc，kerberos客户端，以及调用kerberos api时都会使用到。
几个重要的配置项目包括：

• ticket_lifetime和renew_lifetime：指定了kdc授权ticket的过期时长，和允许更新现有ticket的时长。
• realms的section：指定了kdc和admin_server的路径

3.修改/var/kerberos/krb5kdc/kadm5.acl配置 

[root@worker3 ~]# vi /var/kerberos/krb5kdc/kadm5.acl

*/[e-mail]      *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@worker3 ~]# vim /var/kerberos/krb5kdc/kdc.conf 

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 WANWEI.COM = {
#  #  master_key_type = aes256-cts
  max_renewable_life= 7d
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-c
ts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

这是kdc的专属配置，可以根据自己的需求修改下kdc数据库的存放目录。

5.创建Kerberos数据库

[root@worker3 krb5kdc]# kdb5_util create -r WANWEI.COM –s                              
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'WANWEI.COM',
master key name 'K/[e-mail]'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: 
Re-enter KDC database master key to verify:

红框处需要输入Kerberos数据库的密码。

6.创建Kerberos的管理账号

[root@worker3 krb5kdc]# kadmin.local
Authenticating as principal root/[e-mail] with password.
kadmin.local:  addprinc admin/[e-mail]
WARNING: no policy specified for admin/[e-mail]; defaulting to no policy
Enter password for principal "admin/[e-mail]": 
Re-enter password for principal "admin/[e-mail]": 
Principal "admin/[e-mail]" created.
kadmin.local:  exit

标红部分为Kerberos管理员账号，需要输入管理员密码

7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务

[root@worker3 ~]# systemctl enable krb5kdc.service
[root@worker3 ~]# systemctl enable kadmin.service
[root@worker3 ~]# systemctl start krb5kdc.service
Starting Kerberos 5 KDC:                                   [  OK  ]
[root@worker3 ~]# systemctl start kadmin.service
Starting Kerberos 5 Admin Server:                          [  OK  ]

8.测试Kerberos的管理员账号

[root@worker3 ~]# kinit admin/[e-mail]
Password for admin/[e-mail]: 
[root@worker3 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/[e-mail]

Valid starting       Expires              Service principal
2019-03-01T11:53:20  2019-03-02T11:53:20  krbtgt/[e-mail]
        renew until 2019-03-08T11:53:20

9.为集群安装所有Kerberos客户端，包括Cloudera Manager

yum -y install krb5-libs krb5-workstation

10.在Cloudera Manager Server服务器上安装额外的包

yum -y install openldap-clients

11.将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端

scp -r /etc/krb5.conf root@master:/etc/

CDH集群启用Kerberos

1.在KDC中给Cloudera Manager添加管理员账号

[root@worker3 ~]# kadmin.local
Authenticating as principal admin/[e-mail] with password.
kadmin.local:  addprinc cloudera-scm/[e-mail]
WARNING: no policy specified for cloudera-scm/[e-mail]; defaulting to no policy
Enter password for principal "cloudera-scm/[e-mail]": 
Re-enter password for principal "cloudera-scm/[e-mail]": 
Principal "cloudera-scm/[e-mail]" created.
kadmin.local:  exit

2.进入Cloudera Manager的“管理”-> “安全”界面 -> “启用Kerberos”

3.选择“启用Kerberos”，进入如下界面确保如下列出的所有检查项都已完成

4.点击“继续”，配置相关的KDC信息，包括类型、KDC服务器、KDC Realm、加密类型以及待创建的Service Principal（hdfs，yarn,，hbase，hive等）的更新生命期等。

5.取消“通过Cloudera manager管理”，不建议让Cloudera Manager来管理krb5.conf.点击“继续”：

 

 6.输入Cloudera Manager的Kerbers管理员账号，必须和之前创建的账号一致。点击“继续”:

7.等待启用Kerberos完成，点击“继续” 

8.重启集群，点击“继续” 

 9.等待集群重启成功，点击“继续”

至此已成功启用Kerberos。

Kerberos使用 

1.使用root用户运行MapReduce任务及操作Hive，需要在集群所有节点创建root用户。

[root@worker3 ~]# kadmin.local
Authenticating as principal admin/[e-mail] with password.
kadmin.local:  addprinc [e-mail]  
WARNING: no policy specified for [e-mail]; defaulting to no policy
Enter password for principal "[e-mail]": 
Re-enter password for principal "[e-mail]": 
Principal "[e-mail]" created.
kadmin.local:  exit

2.使用root用户登录Kerberos

[root@worker3 ~]# kdestroy
[root@worker3 ~]# kinit root
Password for [e-mail]: 
[root@worker3 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [e-mail]

Valid starting       Expires              Service principal
2019-03-01T15:51:09  2019-03-02T15:51:09  krbtgt/[e-mail]
        renew until 2019-03-08T15:51:09

3.验证namenode是否正确启动

两种方法：

（1）访问http://machine:50070

（2）

[root@worker3 ~]# hadoop fs -ls /

注：如果在你的凭据缓存中没有有效的kerberos ticket，执行hadoop fs -ls将会失败。

可以使用klist来查看是否有有有效的ticket。

可以通过kinit来获取ticket.

kinit -k -t /usr/local/hadoop/conf/hdfs.ketab hdfs/nn.hadoop.local@for_hadoop