kafka消费者超时解决方案_kafka消费超时

背景

  kafka消费者采用批量拉取的方式，一次拉取一批记录来消费，如果消费者线程消费超时会导致整批消息的回滚，从而导致已经消费过的数据再消费一遍，消费者不幂等会出大问题，这也是为啥要强调使用消费队列一定要考虑幂等性的原因。
  细思极恐的是：这次在规定的时间内消费不完，你能保证下次就能消费完吗？有可能进入拉取-》超时-》回滚-》拉取的无限循环中。

消费超时会发生什么？

Kafka Handle Error, Client Will Seek Soon: org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

报错信息非常良心，简单解释下：