版本说明
Released July 30, 2018
Release Notes
Source download: kafka-2.0.0-src.tgz (asc, sha512)
Binary downloads:We build for multiple versions of Scala. This only matters if you are using Scala and you want a version built for the same Scala version you use. Otherwise any version should work (2.11 is recommended).
Scala 2.11 - kafka_2.11-2.0.0.tgz (asc, sha512)
Scala 2.12 - kafka_2.12-2.0.0.tgz (asc, sha512)
软件下载地址 http://kafka.apache.org/downloads.html
下载安装包 kafka_2.11-2.0.0.tgz
wget https://archive.apache.org/dist/kafka/2.4.1/kafka_2.11-2.4.1.tgz
部署
tar -zxvf kafka_2.11-2.4.1.tgz
cd kafka_2.11-2.0.0
chmod a+x bin/* //赋予服务器脚本运行需要的执行权限
vi config/server.properties //配置服务器的IP地址，如果是默认的localhost只能够在本机器使用，其他的外网链接不上
listeners=PLAINTEXT://192.168.11.173:9092 //修改内容如左
firewall-cmd --zone=public --add-port=9092/tcp --permanent# 开放9092端口
firewall-cmd --reload# 配置立即生效
启动
必须先启动zookeeper,再启动kafka缺一不可
bin/zookeeper-server-start.sh config/zookeeper.properties &
bin/kafka-server-start.sh config/server.properties
可能出现以下错误/opt/kafka_2.11-2.0.0/bin/kafka-run-class.sh: line 306: exec: java not found 解决方案 yum install java（原因为Linux缺少java）
//另外启动两个控制台程序，作为生成者发送数据
bin/kafka-console-producer.sh --broker-list 192.168.11.173:9092 --topic test//生产者往主题test发送数据,出现>可以输入发送数据
>
//消费者消费数据
bin/kafka-console-consumer.sh --bootstrap-server 192.168.11.173:9092 --topic test --from-beginning //接收数据
zookeeper跟broker的关系
1）如何将kafka注册到zookeeper上,然后可以查询到kafka的节点信息
kafka 节点启动后，会向 zookeeper 注册自己，同时告诉 zookeeper 自身的通信地址，这个地址就是配置文件中的 advertised.listeners，如果没有配置 advertised.listeners，就会使用listeners。同时从 zookeeper 中获取兄弟节点的这个地址，以便与兄弟节点通信。即 kafka 节点是从 zookeeper 获取的其他节点的通信地址。
我们使用客户端以一个 ip 地址首次连接 kafka 节点后，节点返回给客户端的 kafka 集群地址就是从 zookeeper 中获得的这些地址，也就是各个节点配置的 advertised.listeners，包括当前连接的节点。所以可能客户端后续访问当前节点的 ip 地址有可能和首次连接的 ip 地址并不一样。
显示zookeeper根目录下的子节点，其中kafka的broker在zookeeper根目录注册了brokers节点
kafka对外提供的meta info中的节点地址信息是直接读取zookeeper的注册信息，该注册信息是在集群启动时写入的，信息的最终来源为server.properties。依据Kafka官方文档http://kafka.apache.org/0101/documentation.html#brokerconfigs的描述，配置的优先级如下：
1.首先读取advertised.listeners的值
2.若1中值为空，使用advertised.host.name & advertised.port注册到zookeeper
连接上了broker，找到了zookeeper上对应的kafka的topic信息，接下来要往topic上发送数据的时候，zk上注册的topic对应的信息是局域网的机器名称和ip
[2020-05-18 16:33:58,912] WARN Session 0x0 for server ls:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1021)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1064)
注意
1）
[2020-05-18 11:52:07,986] WARN [Producer clientId=console-producer] Connection to node 0 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
本地环境测试启动生产者说明服务broker没有启动
2）检测服务器是否配置正确的IP地址和开放端口
%3|1589772697.411|FAIL|rdkafka#consumer-1| [thrd:192.168.11.173:9092/bootstrap]: 192.168.58.121:9092/bootstrap: Connect to ipv4#192.168.11.173:9092 failed: 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。
%3|1589772699.528|ERROR|rdkafka#consumer-1| [thrd:192.168.58.121:9092/bootstrap]: 1/1 brokers are down
3）bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
zookeeper is not a recognized option
Kafka 从 2.2 版本开始将 kafka-topic.sh 脚本中的 −−zookeeper 参数标注为 “过时”，推荐使用 −−bootstrap-server 参数。若读者依旧使用的是 2.1 及以下版本，请将下述的 --bootstrap-server 参数及其值手动替换为 --zookeeper zk1:2181,zk2:2181,zk:2181。一定要注意两者参数值所指向的集群地址是不同的。
4)thrd:localhost:9092/0
%3|1702952967.809|FAIL|rdkafka#consumer-1| [thrd:localhost:9092/0]: localhost:9092/0: Connect to ipv4#127.0.0.1:9092 failed: Unknown error
没有正确配置Kafka服务器IP地址，
否则，Kafka 代理将告诉任何连接的人，可以在 localhost:9092 上找到它
导致错误提示出现localhost的错误
vi config/server.properties //配置服务器的IP地址，如果是默认的localhost只能够在本机器使用，其他的外网链接不上
listeners=PLAINTEXT://192.168.11.173:9092 //修改内容如左
5 必须先启动zookeeper，否则提示如下错误
[2023-12-19 13:00:07,808] INFO Socket error occurred: localhost/127.0.0.1:2181: 拒绝连接 (org.apache.zookeeper.ClientCnxn)
[2023-12-19 13:00:08,301] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-12-19 13:00:08,909] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-19 13:00:09,019] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2023-12-19 13:00:09,024] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2023-12-19 13:00:09,027] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply$mcV$sp(ZooKeeperClient.scala:259)
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:255)
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:255)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
at kafka.zookeeper.ZooKeeperClient.kafka$zookeeper$ZooKeeperClient$$waitUntilConnected(ZooKeeperClient.scala:255)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1857)
at kafka.server.KafkaServer.kafka$server$KafkaServer$$createZkClient$1(KafkaServer.scala:374)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:399)
at kafka.server.KafkaServer.startup(KafkaServer.scala:207)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:84)
at kafka.Kafka.main(Kafka.scala)
[2023-12-19 13:00:09,042] INFO shutting down (kafka.server.KafkaServer)
[2023-12-19 13:00:09,047] INFO shut down completed (kafka.server.KafkaServer)
[2023-12-19 13:00:09,048] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2023-12-19 13:00:09,030] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2023-12-19 13:00:09,058] INFO shutting down (kafka.server.KafkaServer)
