kafka分区不均衡 kafka分区再均衡

1.再均衡1表示，新增一个消费者3；再均衡2表示，去掉一个消费者3；每次再均衡都会从数据库读取最新的偏移量开始消费

2.再均衡监听器

• 消费者在退出和进行分区再均衡之前,会做一些清理工作比如，提交偏移量、关闭文件句柄、数据库连接等。
• 在为消费者分配新分区或移除旧分区时,可以通过消费者 API 执行一些应用程序代码，在调用 subscribe()方法时传进去一个 ConsumerRebalancelistener 实例就可以了。
• ConsumerRebalancelistener 有两个需要实现的方法。

1. public void onPartitionsRevoked( Collection< TopicPartition> partitions)方法会在 再均衡开始之前和消费者停止读取消息之后被调用。如果在这里提交偏移量，下一个接管分区的消费者就知道该从哪里开始读取了
2. public void onPartitionsAssigned( Collection< TopicPartition> partitions)方法会在重新分配分区之后和消费者开始读取消息之前被调用。
   具体使用，我们先创建一个 3 分区的主题rebalance-topic

3.从特定偏移量处开始记录

• 如果想从分区的起始位置开始读取消息,或者直接跳到分区的末尾开始读取消息,可以使 seekToBeginning(Collection tp)和 seekToEnd( Collectiontp)这两个方法。
• 如果需要从特定的偏移量处开始读取消息。使用Kafka 提供的查找特定偏移量的 API。它有很多用途,比如向后回退几个消息或者向前跳过几个消息(对时间比较敏感的应用程序在处 理滞后的情况下希望能够向前跳过若干个消息)。在使用 Kafka 以外的系统来存储偏移量时,它将给我们带来更大的惊喜–让消息的业务处理和偏移量的提 交变得一致
• 场景:应用程序从 Kaka 读取事件(可能是网站的用户点击事件流),对它们进行处理(可能是使用自动程序清理点击操作并添加会话信息), 然后把结果保存到数据库。假设我们真的不想丢失任何数据,也不想在数据库里多次保存相同的结果。
• 我们可能会，毎处理一条记录就提交一次偏移量。尽管如此,在记录被保存到数据库之后以及偏移量被提交之前,应用程序仍然有可能发生崩溃,导致重复处理数据,数据库里就会出现重复记录。
• 如果保存记录和偏移量可以在一个原子操作里完成,就可以避免出现上述情况。记录和偏移量要么都被成功提交,要么都不提交。如果记录是保存在数 据库里而偏移量是提交到Kafka上,那么就无法实现原子操作不过,如果在同一个事务里把记录和偏移量都写到数据库里会怎样呢?那么我们就会知道记录和偏移量要么都成功提交,要么都没有,然后重新处理记录。
• 现在的问题是:如果偏移量是保存在数据库里而不是 Kafka 里,那么消费者在得到新分区时怎么知道该从哪里开始读取?这个时候可以使用 seek()方法。 在消费者启动或分配到新分区时,可以使用 seck()方法查找保存在数据库里的偏移量。我们可以使用使用 Consumer Rebalancelistener 和 seek()方法确保我们 是从数据库里保存的偏移量所指定的位置开始处理消息的。

4.添加常量rebalance-topic

package org.example.config;

public class BusiConst {
    public static final String HELLO_TOPIC  = "hello-kafka";
    public static final String CONCURRENT_TOPIC  = "concurrent-test";
    public static final int CONCURRENT_PARTITION_COUNT = 2;
    public static final String SELF_PARTITION  = "self-partition";
    public static final String CONSUMER_COMMIT_TOPIC = "simple";
    public static final String REBALANCE_TOPIC = "rebalance-topic";
}

5.创建生产者RebalanceProducer，生产50条消息

package org.example.rebalance;

import org.apache.kafka.clients.producer.Callback;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import org.apache.kafka.common.serialization.StringSerializer;
import org.example.config.BusiConst;

import java.util.Properties;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * 多线程下使用生产者
 **/
public class RebalanceProducer {
    private static final int MSG_SIZE = 50;
    private static ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
    private static CountDownLatch countDownLatch = new CountDownLatch(MSG_SIZE);

    private static class ProducerWorker implements Runnable {
        private ProducerRecord<String, String> record;
        private KafkaProducer<String, String> producer;

        public ProducerWorker(ProducerRecord<String, String> record, KafkaProducer<String, String> producer) {
            this.record = record;
            this.producer = producer;
        }

        @Override
        public void run() {
            final String id = Thread.currentThread().getId() + "-" + System.identityHashCode(producer);
            try {
                producer.send(record, new Callback() {
                    @Override
                    public void onCompletion(RecordMetadata recordMetadata, Exception e) {
                        if (e != null) {
                            e.printStackTrace();
                        }
                        if (recordMetadata != null) {
                            System.out.println("偏移量：" + recordMetadata.offset() + ";分区:" + recordMetadata.partition());
                        }
                    }
                });
                System.out.printf(id + "数据[" + record + "]已发出");
                countDownLatch.countDown();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    public static void main(String[] args) {
        Properties properties = new Properties();
        properties.put("bootstrap.servers", "192.168.42.111:9092");
        properties.put("key.serializer", StringSerializer.class);
        properties.put("value.serializer", StringSerializer.class);
        KafkaProducer<String, String> producer = new KafkaProducer<String, String>(properties);
        try {
            for (int i = 0; i < MSG_SIZE; i++) {
                ProducerRecord<String, String> record = new ProducerRecord<String, String>(BusiConst.REBALANCE_TOPIC, null,
                        System.currentTimeMillis(), String.valueOf(i), "Fisher" + i);
                executorService.submit(new ProducerWorker(record, producer));
                Thread.sleep(600);
            }
            countDownLatch.await();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            producer.close();
            executorService.shutdown();
        }
    }
}

6.创建再均衡监听器HandlerRebalance，实现ConsumerRebalanceListener接口

package org.example.rebalance;

import org.apache.kafka.clients.consumer.ConsumerRebalanceListener;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.consumer.OffsetAndMetadata;
import org.apache.kafka.common.TopicPartition;

import java.util.Collection;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
 * 再均衡监听器
 **/
public class HandlerRebalance implements ConsumerRebalanceListener {
    //模拟一个保存分区偏移量的数据库表
    public final static ConcurrentHashMap<TopicPartition, Long> partitionOffsetMap = new ConcurrentHashMap<>();
    private final Map<TopicPartition, OffsetAndMetadata> currOffset;
    private final KafkaConsumer<String, String> consumer;

    public HandlerRebalance(Map<TopicPartition, OffsetAndMetadata> currOffset, KafkaConsumer<String, String> consumer) {
        this.currOffset = currOffset;
        this.consumer = consumer;
    }

    //分区在均衡之前
    @Override
    public void onPartitionsRevoked(Collection<TopicPartition> collection) {
        final String id = String.valueOf(Thread.currentThread().getId());
        System.out.println(id + "-Partition参数值为" + collection);
        System.out.println(id + "服务器准备分区再均衡，提交偏移量。当前偏移量为：" + currOffset);
        //我们可以不使用consumer.commitSync(currOffsets);
        //提交偏移量到kafka,由我们自己维护*/
        //开始事务
        //偏移量写入数据库
        System.out.println("分区偏移量表中：" + partitionOffsetMap);
        for (TopicPartition topicPartition : collection) {
            partitionOffsetMap.put(topicPartition, currOffset.get(topicPartition).offset());
        }
        consumer.commitSync(currOffset);
        //提交业务数和偏移量入库  tr.commit
    }

    //分区再均衡完成之后
    @Override
    public void onPartitionsAssigned(Collection<TopicPartition> collection) {
        final String id = String.valueOf(Thread.currentThread().getId());
        System.out.println(id + "-再均衡完成，Partition参数为：" + collection);
        System.out.println("分区偏移量表中：" + partitionOffsetMap);
        for (TopicPartition topicPartition : collection) {
            System.out.println(id + "-topicPartition:" + topicPartition);
            //模拟从数据库中取得上次的偏移量
            Long offset = partitionOffsetMap.get(topicPartition);
            if (offset == null) continue;
            //从特定偏移量处开始记录（从指定分区中的指定偏移量开始消费）
            //这样就可以确保分区在均衡中的记录不错乱
            consumer.seek(topicPartition, partitionOffsetMap.get(topicPartition));
        }
    }
}

7.创建ConsumerWorker实现Runnable接口

package org.example.rebalance;

import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.TopicPartition;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.example.config.BusiConst;

import java.time.Duration;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

public class ConsumerWork implements Runnable {
    private final KafkaConsumer<String, String> consumer;
    //todo 用来保存每个消费者当前读取分区的偏移量，解决分区再均衡的关键
    private final Map<TopicPartition, OffsetAndMetadata> currOffsets;
    private final boolean isStop;
    //todo 事务类可以送入（tr）
//    private final Transaction tr;事务类实例
    public ConsumerWork(boolean isStop) {
        //消费者配置
        Properties properties = new Properties();
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.42.111:9092");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, "rebalanceconsumer");
        //todo 取消自动提交
        properties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
        this.isStop = isStop;
        this.consumer = new KafkaConsumer<String, String>(properties);
        //todo 偏移量
        this.currOffsets = new HashMap<>();
        //todo 消费者订阅时加入再均衡监听器（HandlerRebalance）
        consumer.subscribe(Collections.singletonList(BusiConst.REBALANCE_TOPIC), new HandlerRebalance(currOffsets, consumer));
    }

    @Override
    public void run() {
        final String id = String.valueOf(Thread.currentThread().getId());
        int count = 0;
        TopicPartition topicPartition = null;
        long offset = 0;
        try {
            while (true) {
                ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
                //todo 业务处理
                //开始事务 tr.begin
                for (ConsumerRecord<String, String> record : records) {
                    System.out.println(id + "处理主题：" + record.topic() + ";分区：" + record.partition() +
                            ";偏移量：" + record.offset() + ";key:" + record.key() + ";value:" + record.value());
                    topicPartition = new TopicPartition(record.topic(), record.partition());
                    offset = record.offset() + 1;
                    //消费者消费时把偏移量提交到统一HashMap
                    currOffsets.put(topicPartition, new OffsetAndMetadata(offset, "no"));
                    count++;
                    //执行业务sql
                }
                //todo 同时将业务和偏移量入库
                if (currOffsets.size() > 0) {
                    for (TopicPartition topicPartitionkey : currOffsets.keySet()) {
                        HandlerRebalance.partitionOffsetMap.put(topicPartitionkey, currOffsets.get(topicPartitionkey).offset());
                    }
                }
                //todo 提交事务
                //todo 如果stop参数为true，这个消费者消费到第5个时自动关闭
                if (isStop && count >= 5) {
                    System.out.println(id + "-将关闭，当前偏移量为：" + currOffsets);
                    consumer.commitSync();
                    break;
                }
                consumer.commitSync();
            }
        } finally {
            consumer.close();
        }
    }
}

8.创建消费者RebalanceConsumer

package org.example.rebalance;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
/**
 * 设置了再均衡监听器的消费者
 **/
public class RebalanceConsumer {
    //使用线程池，使用2个线程（2个消费者）
    private static ExecutorService executorService = Executors.newFixedThreadPool(2);

    public static void main(String[] args) throws InterruptedException {
        //todo 1.起2个消费者，消费3个分区
        for (int i = 0; i < 2; i++) {
            executorService.submit(new ConsumerWork(false));
        }
        Thread.sleep(500);
        //用来被停止观察消费情况
        //todo 2.再起一个消费者，消费5条数据之后关闭
        //todo 这里会触发2次分区再均衡，一次是新消费者加入，另一次是这个消费者消费5条数据之后离开
        new Thread(new ConsumerWork(true)).start();
    }
}

9.先启动消费者，再启动生产者

生产者打印

13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=0, value=Fisher0, timestamp=1595077712653)]已发出偏移量：0;分区:2
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=1, value=Fisher1, timestamp=1595077713259)]已发出偏移量：0;分区:0
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=2, value=Fisher2, timestamp=1595077713859)]已发出偏移量：1;分区:2
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=3, value=Fisher3, timestamp=1595077714460)]已发出偏移量：2;分区:2
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=4, value=Fisher4, timestamp=1595077715060)]已发出偏移量：0;分区:1
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=5, value=Fisher5, timestamp=1595077715661)]已发出偏移量：1;分区:0
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=6, value=Fisher6, timestamp=1595077716262)]已发出偏移量：1;分区:1
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=7, value=Fisher7, timestamp=1595077716862)]已发出偏移量：2;分区:0
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=8, value=Fisher8, timestamp=1595077717462)]已发出偏移量：3;分区:0
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=9, value=Fisher9, timestamp=1595077718062)]已发出偏移量：3;分区:2
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=10, value=Fisher10, timestamp=1595077718663)]已发出偏移量：2;分区:1
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=11, value=Fisher11, timestamp=1595077719263)]已发出偏移量：4;分区:0
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=12, value=Fisher12, timestamp=1595077719864)]已发出偏移量：3;分区:1
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=13, value=Fisher13, timestamp=1595077720464)]已发出偏移量：4;分区:1
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=14, value=Fisher14, timestamp=1595077721064)]已发出偏移量：5;分区:1
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=15, value=Fisher15, timestamp=1595077721665)]已发出偏移量：5;分区:0
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=16, value=Fisher16, timestamp=1595077722265)]已发出偏移量：4;分区:2
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=17, value=Fisher17, timestamp=1595077722865)]已发出偏移量：6;分区:0
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=18, value=Fisher18, timestamp=1595077723466)]已发出偏移量：6;分区:1
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=19, value=Fisher19, timestamp=1595077724066)]已发出偏移量：7;分区:1
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=20, value=Fisher20, timestamp=1595077724666)]已发出偏移量：8;分区:1
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=21, value=Fisher21, timestamp=1595077725267)]已发出偏移量：7;分区:0
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=22, value=Fisher22, timestamp=1595077725867)]已发出偏移量：8;分区:0
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=23, value=Fisher23, timestamp=1595077726468)]已发出偏移量：9;分区:0
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=24, value=Fisher24, timestamp=1595077727068)]已发出偏移量：9;分区:1
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=25, value=Fisher25, timestamp=1595077727669)]已发出偏移量：10;分区:0
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=26, value=Fisher26, timestamp=1595077728269)]已发出偏移量：10;分区:1
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=27, value=Fisher27, timestamp=1595077728869)]已发出偏移量：11;分区:0
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=28, value=Fisher28, timestamp=1595077729470)]已发出偏移量：12;分区:0
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=29, value=Fisher29, timestamp=1595077730071)]已发出偏移量：5;分区:2
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=30, value=Fisher30, timestamp=1595077730671)]已发出偏移量：13;分区:0
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=31, value=Fisher31, timestamp=1595077731272)]已发出偏移量：11;分区:1
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=32, value=Fisher32, timestamp=1595077731872)]已发出偏移量：6;分区:2
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=33, value=Fisher33, timestamp=1595077732472)]已发出偏移量：14;分区:0
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=34, value=Fisher34, timestamp=1595077733073)]已发出偏移量：15;分区:0
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=35, value=Fisher35, timestamp=1595077733673)]已发出偏移量：12;分区:1
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=36, value=Fisher36, timestamp=1595077734274)]已发出偏移量：7;分区:2
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=37, value=Fisher37, timestamp=1595077734874)]已发出偏移量：16;分区:0
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=38, value=Fisher38, timestamp=1595077735475)]已发出偏移量：13;分区:1
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=39, value=Fisher39, timestamp=1595077736075)]已发出偏移量：14;分区:1
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=40, value=Fisher40, timestamp=1595077736676)]已发出偏移量：8;分区:2
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=41, value=Fisher41, timestamp=1595077737277)]已发出偏移量：9;分区:2
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=42, value=Fisher42, timestamp=1595077737877)]已发出偏移量：15;分区:1
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=43, value=Fisher43, timestamp=1595077738477)]已发出偏移量：17;分区:0
15-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=44, value=Fisher44, timestamp=1595077739078)]已发出偏移量：18;分区:0
16-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=45, value=Fisher45, timestamp=1595077739679)]已发出偏移量：19;分区:0
17-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=46, value=Fisher46, timestamp=1595077740279)]已发出偏移量：16;分区:1
18-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=47, value=Fisher47, timestamp=1595077740880)]已发出偏移量：20;分区:0
13-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=48, value=Fisher48, timestamp=1595077741480)]已发出偏移量：21;分区:0
14-608597659数据[ProducerRecord(topic=rebalance-topic, partition=null, headers=RecordHeaders(headers = [], isReadOnly = true), key=49, value=Fisher49, timestamp=1595077742081)]已发出偏移量：10;分区:2

Process finished with exit code 0

10.查看消费者打印

12-Partition参数值为[]
12服务器准备分区再均衡，提交偏移量。当前偏移量为：{}
分区偏移量表中：{}
13-Partition参数值为[]
13服务器准备分区再均衡，提交偏移量。当前偏移量为：{}
分区偏移量表中：{}
16-Partition参数值为[]
16服务器准备分区再均衡，提交偏移量。当前偏移量为：{}
分区偏移量表中：{}
13-再均衡完成，Partition参数为：[rebalance-topic-1]
分区偏移量表中：{}
12-再均衡完成，Partition参数为：[rebalance-topic-0]
分区偏移量表中：{}
12-topicPartition:rebalance-topic-0
16-再均衡完成，Partition参数为：[rebalance-topic-2]
分区偏移量表中：{}
16-topicPartition:rebalance-topic-2
13-topicPartition:rebalance-topic-1
16处理主题：rebalance-topic;分区：2;偏移量：0;key:0;value:Fisher0
12处理主题：rebalance-topic;分区：0;偏移量：0;key:1;value:Fisher1
16处理主题：rebalance-topic;分区：2;偏移量：1;key:2;value:Fisher2
16处理主题：rebalance-topic;分区：2;偏移量：2;key:3;value:Fisher3
13处理主题：rebalance-topic;分区：1;偏移量：0;key:4;value:Fisher4
12处理主题：rebalance-topic;分区：0;偏移量：1;key:5;value:Fisher5
13处理主题：rebalance-topic;分区：1;偏移量：1;key:6;value:Fisher6
12处理主题：rebalance-topic;分区：0;偏移量：2;key:7;value:Fisher7
12处理主题：rebalance-topic;分区：0;偏移量：3;key:8;value:Fisher8
16处理主题：rebalance-topic;分区：2;偏移量：3;key:9;value:Fisher9
13处理主题：rebalance-topic;分区：1;偏移量：2;key:10;value:Fisher10
12处理主题：rebalance-topic;分区：0;偏移量：4;key:11;value:Fisher11
13处理主题：rebalance-topic;分区：1;偏移量：3;key:12;value:Fisher12
13处理主题：rebalance-topic;分区：1;偏移量：4;key:13;value:Fisher13
13处理主题：rebalance-topic;分区：1;偏移量：5;key:14;value:Fisher14
12处理主题：rebalance-topic;分区：0;偏移量：5;key:15;value:Fisher15
16处理主题：rebalance-topic;分区：2;偏移量：4;key:16;value:Fisher16
16-将关闭，当前偏移量为：{rebalance-topic-2=OffsetAndMetadata{offset=5, leaderEpoch=null, metadata='no'}}
12处理主题：rebalance-topic;分区：0;偏移量：6;key:17;value:Fisher17
13处理主题：rebalance-topic;分区：1;偏移量：6;key:18;value:Fisher18
13处理主题：rebalance-topic;分区：1;偏移量：7;key:19;value:Fisher19
13-Partition参数值为[rebalance-topic-1]
13服务器准备分区再均衡，提交偏移量。当前偏移量为：{rebalance-topic-1=OffsetAndMetadata{offset=8, leaderEpoch=null, metadata='no'}}
分区偏移量表中：{rebalance-topic-0=7, rebalance-topic-1=8, rebalance-topic-2=5}
12-Partition参数值为[rebalance-topic-0]
12服务器准备分区再均衡，提交偏移量。当前偏移量为：{rebalance-topic-0=OffsetAndMetadata{offset=7, leaderEpoch=null, metadata='no'}}
分区偏移量表中：{rebalance-topic-0=7, rebalance-topic-1=8, rebalance-topic-2=5}
13-再均衡完成，Partition参数为：[rebalance-topic-2]
分区偏移量表中：{rebalance-topic-0=7, rebalance-topic-1=8, rebalance-topic-2=5}
13-topicPartition:rebalance-topic-2
12-再均衡完成，Partition参数为：[rebalance-topic-0, rebalance-topic-1]
分区偏移量表中：{rebalance-topic-0=7, rebalance-topic-1=8, rebalance-topic-2=5}
12-topicPartition:rebalance-topic-0
12-topicPartition:rebalance-topic-1
12处理主题：rebalance-topic;分区：1;偏移量：8;key:20;value:Fisher20
12处理主题：rebalance-topic;分区：0;偏移量：7;key:21;value:Fisher21
12处理主题：rebalance-topic;分区：0;偏移量：8;key:22;value:Fisher22
12处理主题：rebalance-topic;分区：0;偏移量：9;key:23;value:Fisher23
12处理主题：rebalance-topic;分区：1;偏移量：9;key:24;value:Fisher24
12处理主题：rebalance-topic;分区：0;偏移量：10;key:25;value:Fisher25
12处理主题：rebalance-topic;分区：1;偏移量：10;key:26;value:Fisher26
12处理主题：rebalance-topic;分区：0;偏移量：11;key:27;value:Fisher27
12处理主题：rebalance-topic;分区：0;偏移量：12;key:28;value:Fisher28
13处理主题：rebalance-topic;分区：2;偏移量：5;key:29;value:Fisher29
12处理主题：rebalance-topic;分区：0;偏移量：13;key:30;value:Fisher30
12处理主题：rebalance-topic;分区：1;偏移量：11;key:31;value:Fisher31
13处理主题：rebalance-topic;分区：2;偏移量：6;key:32;value:Fisher32
12处理主题：rebalance-topic;分区：0;偏移量：14;key:33;value:Fisher33
12处理主题：rebalance-topic;分区：0;偏移量：15;key:34;value:Fisher34
12处理主题：rebalance-topic;分区：1;偏移量：12;key:35;value:Fisher35
13处理主题：rebalance-topic;分区：2;偏移量：7;key:36;value:Fisher36
12处理主题：rebalance-topic;分区：0;偏移量：16;key:37;value:Fisher37
12处理主题：rebalance-topic;分区：1;偏移量：13;key:38;value:Fisher38
12处理主题：rebalance-topic;分区：1;偏移量：14;key:39;value:Fisher39
13处理主题：rebalance-topic;分区：2;偏移量：8;key:40;value:Fisher40
13处理主题：rebalance-topic;分区：2;偏移量：9;key:41;value:Fisher41
12处理主题：rebalance-topic;分区：1;偏移量：15;key:42;value:Fisher42
12处理主题：rebalance-topic;分区：0;偏移量：17;key:43;value:Fisher43
12处理主题：rebalance-topic;分区：0;偏移量：18;key:44;value:Fisher44
12处理主题：rebalance-topic;分区：0;偏移量：19;key:45;value:Fisher45
12处理主题：rebalance-topic;分区：1;偏移量：16;key:46;value:Fisher46
12处理主题：rebalance-topic;分区：0;偏移量：20;key:47;value:Fisher47
12处理主题：rebalance-topic;分区：0;偏移量：21;key:48;value:Fisher48
13处理主题：rebalance-topic;分区：2;偏移量：10;key:49;value:Fisher49

• 消费者在没有启动生产者的时候，已经开始发生分区再均衡，结果为（消费1对应线程12与分区0，消费者2对应线程13与分区1，消费者3对应线程16与分区2）
• 生产者生产消息之后，消费者开始消费者，直到消费者3消费了5条消息停止之后，发生第2次分区再均衡，此时分区2的偏移量为5
• 此时消费者1和消费者2重新分配分区，分区0的偏移量为7，分区1的偏移量为8
• 再均衡完成，消费者1消费分区0和1，消费者2消费分区2
• 消费者1消费分区1从偏移量8开始消费，消费分区0从偏移量7开始消费；消费者2消费分区2从偏移量5开始消费