安装单机Kafka

步骤

安装kafka需要安装:JDK+Zookeeper+Kafka。

1.下载

下载地址: ​​Apache Kafka​

2.解压

3.修改配置

安装目录/config/server.properties

log.dirs=D:\Kafka\kafka_2.12-0.11.0.0\kafka-logs    //日志路径
zookeeper.connect=localhost:2181 //zookeeper地址
broker.id=0 //broker的唯一id

4.启动

1.启动zookeeper

2.启动kafka

.\bin\windows\kafka-server-start.bat ..\..\config\server.properties

5.验证

可以通过查看zookeeper中的节点来确认kafka是否启动。

启动zookeeper的client(在命令行执行.\zkCli.cmd -server 127.0.0.1:2181),然后输入ls /brokers/ids

Kafka--安装与设置_apache

这里的0就是broker.id,可见我们刚才配置的kafka启动成功了。

问题解决

Kafka启动失败,查看kafka失败日志:Kafka安装目录/logs/server.log

ID不匹配

[2021-06-19 17:09:45,737] INFO Cluster ID = Gp_FULuuSTSKwBzkGtD1DQ (kafka.server.KafkaServer)
[2021-06-19 17:09:45,745] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID Gp_FULuuSTSKwBzkGtD1DQ doesn't match stored clusterId Some(Ra3BVZxZSiqW7JomU_PWQA) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
at kafka.server.KafkaServer.startup(KafkaServer.scala:252)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
[2021-06-19 17:09:45,747] INFO shutting down (kafka.server.KafkaServer)
[2021-06-19 17:09:45,749] INFO [feature-zk-node-event-process-thread]: Shutting down (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2021-06-19 17:09:45,750] INFO [feature-zk-node-event-process-thread]: Shutdown completed (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2021-06-19 17:09:45,750] INFO [feature-zk-node-event-process-thread]: Stopped (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2021-06-19 17:09:45,751] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2021-06-19 17:09:45,865] INFO Session: 0x1002df2aca80000 closed (org.apache.zookeeper.ZooKeeper)
[2021-06-19 17:09:45,865] INFO EventThread shut down for session: 0x1002df2aca80000 (org.apache.zookeeper.ClientCnxn)
[2021-06-19 17:09:45,866] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2021-06-19 17:09:45,871] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2021-06-19 17:09:45,871] INFO shut down completed (kafka.server.KafkaServer)
[2021-06-19 17:09:45,872] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2021-06-19 17:09:45,873] INFO shutting down (kafka.server.KafkaServer)

主要是这句:kafka.common.InconsistentClusterIdException: The Cluster ID Gp_FULuuSTSKwBzkGtD1DQ doesn't match stored clusterId Some(Ra3BVZxZSiqW7JomU_PWQA) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.

解决方案 

kafka配置日志路径,用来保存执行过程中的各种信息,当kafka异常关闭时,日志记录就会出现异常,会把当时的情况记录到meta.properties文件中,重新启动时此文件会对启动造成影响,kafka重启报错的原因就在这里。 

a.直接删除日志:Kafka安装目录/logs。不要怕丢失,日志还会自动生成

b.删除meta-properties文件。对于windows:Kafka安装目录\bin\windows\devkafka_2.13-2.7.0kafka-logs

安装集群Kafka