首先需要搭建zookeeper环境,详见:

下载并解压kafka 1.0.0版本:

wget http://archive.apache.org/dist/kafka/1.0.0/kafka_2.11-1.0.0.tgz
tar -zxvf kafka_2.11-1.0.0.tgz
cd kafka_2.11-1.0.0

后面的命令都是在 kafka_2.11-1.0.0 文件夹下操作:

cp config/server.properties config/server-1.properties 
cp config/server.properties config/server-2.properties

编辑 config/server.properties:

broker.id=0
listeners=PLAINTEXT://localhost:9092
log.dirs=/root/kafka-logs/kafka00
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183

编辑 config/server-1.properties:

broker.id=1
listeners=PLAINTEXT://localhost:9093
log.dirs=/root/kafka-logs/kafka01
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183

编辑 config/server-2.properties:

broker.id=2
listeners=PLAINTEXT://localhost:9094
log.dirs=/root/kafka-logs/kafka02
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183

启动3个kafka:

./bin/kafka-server-start.sh -daemon /root/kafka_2.11-1.0.0/config/server.properties; #daemon表示静态,如果不加这个参数需要打开单独的窗口来启动
./bin/kafka-server-start.sh -daemon /root/kafka_2.11-1.0.0/config/server-1.properties;
./bin/kafka-server-start.sh -daemon /root/kafka_2.11-1.0.0/config/server-2.properties;

查看是否启动成功:

netstat -nltp | grep 9092
netstat -nltp | grep 9093
netstat -nltp | grep 9094

创建一个3副本,3分区的topic:

./bin/kafka-topics.sh --create --zookeeper localhost:2181,localhost:2182,localhost:2183 --replication-factor 3 --partitions 3 --topic test5

查看刚创建的test5主题:

./bin/kafka-topics.sh --describe --zookeeper  localhost:2181 --topic test5

结果如图:

kafka搭建伪集群_hive

关闭kafka:

./bin/kafka-server-stop.sh