kafka的kerberos认证 kafka开启kerberos

1.启动内置的zookeeper
bin/ config/zookeeper.properties

2.启动kafka服务
bin/ config/server.properties

3.创建topic话题
bin/kafka-topics.sh --create --topic test --bootstrap-server 192.168.218.128:9092 -partitions 3 -replication-factor 1

查看所有topic话题
bin/kafka-topics.sh --list --bootstrap-server 192.168.218.128:9092

查看指定话题的详情
bin/kafka-topics.sh --bootstrap-server 192.168.230.128:9092 --describe --topic test

3.创建生产者
bin/ --broker-list 192.168.218.128:9092 --topic test

4.创建消费者
bin/kafka-console-consumer.sh --bootstrap-server 192.168.218.128:9092 --topic test

kdb5_util create -s -r [e-mail]

删除凭据
delprinc nsh/111

kadmin.local: addprinc -pw 111 nsh/111
WARNING: no policy specified for nsh/[e-mail]; defaulting to no policy
Principal “nsh/[e-mail]” created.
kadmin.local: listprincs
K/[e-mail]
hdfs/[e-mail]
jack/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kiprop/[e-mail]
krbtgt/[e-mail]
nsh/[e-mail]
root/[e-mail]
kadmin.local: delprinc nsh/111 ##删除凭据
Are you sure you want to delete the principal “nsh/[e-mail]”? (yes/no): yes
Principal “nsh/[e-mail]” deleted.
Make sure that you have removed this principal from all ACLs before reusing.
kadmin.local: listprincs
K/[e-mail]
hdfs/[e-mail]
jack/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kiprop/[e-mail]
krbtgt/[e-mail]
root/[e-mail]
kadmin.local:

导出某个用户的keytab证书（使用xst命令或者ktadd命令）
ktadd -k /root/node3.keytab hdfs/node3

xst -k /root/node4.keytab jack/node4

xst -norandkey -k /root/node34.keytab hdfs/node3 jack/node4

kadmin.local:  listprincs 
K/[e-mail]
hdfs/[e-mail]
jack/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kiprop/[e-mail]
krbtgt/[e-mail]
root/[e-mail]
kadmin.local:  
kadmin.local:  
kadmin.local:  ktadd -k /root/node3.keytab hdfs/node3 ##导出凭据
Entry for principal hdfs/node3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/root/node3.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/root/node3.keytab.
kadmin.local: 
kadmin.local:  xst -k /root/node4.keytab jack/node4 ##导出凭据
Entry for principal jack/node4 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/root/node4.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/root/node4.keytab.
kadmin.local:
kadmin.local:  xst -norandkey -k /root/node34.keytab hdfs/node3 jack/node4 ###将多个principal生产一个keytab
Entry for principal hdfs/node3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/root/node34.keytab.
Entry for principal hdfs/node3 with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/root/node34.keytab.
Entry for principal jack/node4 with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/root/node34.keytab.
kadmin.local:

查看当前客户端认证用户

[root@node1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: root/[e-mail]

Valid starting       Expires              Service principal
2020-01-16T14:43:16  2020-01-17T14:43:16  krbtgt/[e-mail]
[root@node1 ~]#

删除当前的认证的缓存

[root@node1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: root/[e-mail]

Valid starting       Expires              Service principal
2020-01-16T14:43:16  2020-01-17T14:43:16  krbtgt/[e-mail]
[root@node1 ~]# 
[root@node1 ~]# 
[root@node1 ~]# kdestroy
[root@node1 ~]# 
[root@node1 ~]# klist
klist: No credentials cache found (filename: /tmp/krb5cc_0)
[root@node1 ~]#

认证用户
基于密钥认证

[root@node1 ~]# kadmin.local 
Authenticating as principal root/[e-mail] with password.
kadmin.local:  
kadmin.local:  
kadmin.local:  listprincs 
K/[e-mail]
hdfs/[e-mail]
jack/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kadmin/[e-mail]
kiprop/[e-mail]
krbtgt/[e-mail]
root/[e-mail]
kadmin.local:  q
[root@node1 ~]# 
[root@node1 ~]# kinit -kt node3.keytab hdfs/node3 ##基于密钥进行认证
[root@node1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hdfs/[e-mail]

Valid starting       Expires              Service principal
2020-01-16T15:12:10  2020-01-17T15:12:10  krbtgt/[e-mail]
[root@node1 ~]# kinit -kt node3.keytab hdfs/[e-mail] ##基于密钥进行认证
[root@node1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hdfs/[e-mail]

Valid starting       Expires              Service principal
2020-01-16T15:12:48  2020-01-17T15:12:48  krbtgt/[e-mail]
[root@node1 ~]#

基于密码认证
要求

导出密钥时，需要指定-norandkey参数
创建凭据时，需要指定key

[root@node1 ~]# klist
klist: No credentials cache found (filename: /tmp/krb5cc_0)
[root@node1 ~]# kinit tom/node5
Password for tom/[e-mail]: 
[root@node1 ~]# 
[root@node1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: tom/[e-mail]

Valid starting       Expires              Service principal
2020-01-16T15:29:15  2020-01-17T15:29:15  krbtgt/[e-mail]
[root@node1 ~]#

修改Kerberos用户的密码

[root@node1 ~]# kpasswd tom/node5
Password for tom/[e-mail]: 
Enter new password: 
Enter it again: 
Password changed.

下面这种方式不需要知道旧密码

[root@node1 ~]# kadmin.local
Authenticating as principal tom/[e-mail] with password.
kadmin.local:  
kadmin.local:  change_password tom/node5
Enter password for principal "tom/[e-mail]": 
Re-enter password for principal "tom/[e-mail]": 
Password for "tom/[e-mail]" changed.
kadmin.local:  

获取凭据信息
kadmin.local:  getprinc tom/node5
Principal: tom/[e-mail]
Expiration date: [never]
Last password change: 四 1月 16 15:40:12 CST 2020
Password expiration date: [never]
Maximum ticket life: 1 day 00:00:00
Maximum renewable life: 0 days 00:00:00
Last modified: 四 1月 16 15:40:12 CST 2020 (tom/[e-mail])
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 8
Key: vno 3, aes256-cts-hmac-sha1-96
Key: vno 3, aes128-cts-hmac-sha1-96
Key: vno 3, des3-cbc-sha1
Key: vno 3, arcfour-hmac
Key: vno 3, camellia256-cts-cmac
Key: vno 3, camellia128-cts-cmac
Key: vno 3, des-hmac-sha1
Key: vno 3, des-cbc-md5
MKey: vno 1
Attributes:
Policy: [none]
kadmin.local: 

查看keytab文件中的帐号列表
[root@node1 ~]# klist -kt node34.keytab 
Keytab name: FILE:node34.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 hdfs/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
   2 2020-01-16T15:05:53 jack/[e-mail]
[root@node1 ~]#

. centos安装kerberos
下载安装kerberos
yum install krb5-libs krb5-server krb5-workstation krb5-libs
1
安装成功后etc下应该有krb5.conf文件，编辑该文件vi /etc/krb5.conf

Configuration snippets may be placed in this directory as well

includedir /etc/krb5.conf.d/
[logging]
 default = FILE:/var/log/krb5libs.log[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = /etc/pki/tls/certs/ca-bundle.crt
 default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}[realms]
 EXAMPLE.COM = {
 kdc = 192.168.1.237:88
 admin_server = 192.168.1.237
 }[domain_realm]

.example.com = EXAMPLE.COM

example.com = EXAMPLE.COM

这个配置kdc，kerberos客户端，以及调用kerberos api时都会使用到。

realms 中 添加一个realms，并修改realms的kdc鱼admin_sercer地址
libdefaults 中 添加default_realm
编辑vi /var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88[realms]
 EXAMPLE.COM = {
 #master_key_type = aes256-cts
 acl_file = /var/kerberos/krb5kdc/kadm5.acl
 dict_file = /usr/share/dict/words
 admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
 supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

这是kdc的专属配置，可以根据自己的需求修改下kdc数据库的存放目录,我用的默认的，之后生成的文件可以来这里配置过的目录下找

你需要下载java版本的JCE策略文件，将它们复制到 $JAVA_HOME/jre/lib/security中（注意：必须替换！！）
jce下载地址
找到java home

ll $(whereis java)
 ll /etc/alternatives/java

然后拷贝两个jar到你的java home /lib/security中（/jre/lib/security）

创建数据库和principal
使用kdb5_util创建数据库，从而可以存放principal相关的信息(输入两次password)
kdb5_util create -r EXAMPLE.COM -s
使用kadmin.local来添加principal
kadmin.local

创建一个新用户（输入两次密码）

add_principal test-server/@EXAMPLE.COM

导出用户加密配置到krb5.keytab中(先生成一下keytab文件)

xst -k ~/krb5.keytab test-server/[e-mail]
在这里插入图片描述
设置kadmin和krb5kdc服务开机启动(一个kdc的服务一个kadmin管理服务)

chkconfig krb5kdc on
 chkconfig kadmin on
 service krb5kdc start
 service kadmin start

用kinit验证KDC是否启动成功

kinit -k -t krb5.keytab test-server/[e-mail]
 klist

kinit对应的是向kdc获取TGT的步骤。它会向/etc/krb5.conf中指定的kdc server来发送请求。
如果TGT请求成功，你就可以用klist看到它。
在这里插入图片描述
至此，centos就安装完kerberos，并添加一个principal进行验证没有问题了。

2. kafka启用kerberos进行权限控制
   安装kafka,下载地址
   上传kafka，解压，直接启动发送端测试消息发送，启动consumer，测试消息接收，再都没问题的情况下继续权限处理。
   创建kafka 的 Kerberos Principals

添加principal（这时注意，admin就是访问principal的hostname，等下需要配置到hosts文件中）

add_principal kafka/[e-mail]

导出证书

xst -k krb5.keytab kafka/[e-mail]

验证principal是否生效

kinit -k -t krb5.keytab kafka/[e-mail]
klist
配置Broker
添加一个JAAS文件，类似下面的每个kafka broker的配置目录。这个例子我们姑且命名为kafka_server_jaas.conf（注意，每个broker都应该有自己的密钥表）。
KafkaServer {

com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 storeKey=true
 keyTab=“/root/krb5.keytab”
 principal=“kafka/[e-mail]”;
 };// Zookeeper client authentication
 Client {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 storeKey=true
 keyTab=“/root/krb5.keytab”
 principal=“kafka/[e-mail]”;

通过JAAS和krb5文件位置作为JVM参数传递到每个broker
-Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
1
将该内容添加到中的JAVA_OPTS中
在这里插入图片描述
在server.properties中配置SASL端口和SASL机制

listeners=SASL_PLAINTEXT://admin:9002
 security.inter.broker.protocol=SASL_PLAINTEXT
 sasl.mechanism.inter.broker.protocol=GSSAPI
 sasl.enabled.mechanisms=GSSAPI

还必须在server.properties配置服务器名称，应与broker的principal名匹配
=kafka
此时，kafka的服务器端就完成了SASL的配置