ubuntu下安装kafka集群connector

原创

软件测试日常记录 2023-01-26 18:59:23 博主文章分类：ubuntu ©著作权

文章标签 kafka maven mysql 文章分类 运维

©著作权归作者所有：来自51CTO博客作者软件测试日常记录的原创作品，请联系作者获取转载授权，否则将追究法律责任

1.首先安装kafka集群，安装步骤参考链接如下：

2.创建安装connector安装目录
mkdir -p /kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin
mkdir -p /kafka/kafka-2/kafka_2.12-2.2.1/connector-plugin
mkdir -p /kafka/kafka-3/kafka_2.12-2.2.1/connector-plugin
3.下载插件
下载链接： https://debezium.io/docs/install/stable/

cd /kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin
wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-mysql/0.9.5.Final/debezium-connector-mysql-0.9.5.Final-plugin.tar.gz wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-postgres/0.9.5.Final/debezium-connector-postgres-0.9.5.Final-plugin.tar.gz wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-mongodb/0.9.5.Final/debezium-connector-mongodb-0.9.5.Final-plugin.tar.gz wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-sqlserver/0.9.5.Final/debezium-connector-sqlserver-0.9.5.Final-plugin.tar.gz wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-oracle/0.9.5.Final/debezium-connector-oracle-0.9.5.Final-plugin.tar.gz

下载完后解压
tar -zxvf debezium-connector-mysql-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-mongodb-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-oracle-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-postgres-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-sqlserver-0.9.5.Final-plugin.tar.gz

4.下载后配置文件connect-distributed.properties
cd /kafka/kafka-1/kafka_2.12-2.2.1/config
vim connect-distributed.properties
修改1：bootstrap.servers=192.168.0.79:9092,192.168.0.79:9093,192.168.0.79:9094
修改2：rest.port=8083（端口依次增加，8084,8085）
修改3：plugin.path=/kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin（修改目录编号）

cp -r debezium-connector-mysql /kafka/kafka-2/kafka_2.12-2.2.1/connector-plugin
cp /kafka/kafka-1/kafka_2.12-2.2.1/config/connect-distributed.properties /kafka/kafka-2/kafka_2.12-2.2.1/config/connect-distributed.properties

cp -r debezium-connector-mysql /kafka/kafka-3/kafka_2.12-2.2.1/connector-plugin
cp /kafka/kafka-1/kafka_2.12-2.2.1/config/connect-distributed.properties /kafka/kafka-3/kafka_2.12-2.2.1/config/connect-distributed.properties

cd /kafka/kafka-1/kafka_2.12-2.2.1/config
vim connect-distributed.properties

cd /kafka/kafka-2/kafka_2.12-2.2.1/config
vim connect-distributed.properties

cd /kafka/kafka-3/kafka_2.12-2.2.1/config
vim connect-distributed.properties

继续修改如上参数
5.启动
cd /kafka/kafka-1/kafka_2.12-2.2.1/bin
nohup sh connect-distributed.sh …/config/connect-distributed.properties &

cd /kafka/kafka-2/kafka_2.12-2.2.1/bin
nohup sh connect-distributed.sh …/config/connect-distributed.properties &

cd /kafka/kafka-3/kafka_2.12-2.2.1/bin
nohup sh connect-distributed.sh …/config/connect-distributed.properties &