centos 8 Kafka 集群搭建（kafka_2.13-2.6.0）_安装 your local environment must have java 8+ instal

一、准备jdk环境

NOTE: Your local environment must have Java 8+ installed.
注意：jdk版本必须是1.8+

二、准备zookeeper集群环境

Note: Soon, ZooKeeper will no longer be required by Apache Kafka.
注意：很快，kafka将不再需要zookeeper
意思是现在还需要喽

安装zookeeper集群环境并启动

三、kafka集群搭建

• 下载kafka_2.13-2.6.0.tgz解压到/opt目录

$ cd /opt
$ wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
$ tar -zxvf kafka_2.13-2.6.0.tgz
$ cd kafka_2.13-2.6.0/config
1
2
3
4

• 修改配置：主要修改server.properties中以下三个配置项，三台机器对应的broker.id分别为0 1 2，其他两项保持一致即可

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0 # kafka实例的唯一标识，用整数表示，使用不同的整数值将集群中的kafka实例区分即可
# A comma separated list of directories under which to store log files
log.dirs=/opt/kafka_2.13-2.6.0/data # 这个日志文件目录是用来保存消息数据的，注意与/opt/kafka_2.13-2.6.0/logs目录进行区分
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=node2:2181,node3:2181,node4:2181 # 配置zookeeper集群
1
2
3
4
5
6
7
8
9
10

• 启动三个服务

$ cd kafka_2.13-2.6.0
$ bin/kafka-server-start.sh -daemon config/server.properties
1
2

四、基本操作

• 添加主题topic，查看数据文件存储位置

  $ bin/kafka-topics.sh --create --topic HELLO --bootstrap-server localhost:9092
  1

  

• 查看topic详细信息

  $ bin/kafka-topics.sh --describe --topic HELLO --bootstrap-server localhost:9092
  
  Topic: HELLO	PartitionCount: 1	ReplicationFactor: 1	Configs: segment.bytes=1073741824
  	Topic: HELLO	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
  1
  2
  3
  4

  Topic ：主题

  PartitionCount ： 分区数

  ReplicationFactor ： 副本数

  Configs: segment.bytes=1073741824 ： 配置项，segment.bytes表示分割日志文件的阈值，即片段日志文件最大1GB，当达到该值时，新建片段
  

• 写消息到主题，即生产消息

  $ bin/kafka-console-producer.sh --topic HELLO --bootstrap-server localhost:9092
  >hello, this is my first message
  >hello, this is my second message
  1
  2
  3

• 从主题读取消息，即消费消息

  $ bin/kafka-console-consumer.sh --topic HELLO --from-beginning --bootstrap-server localhost:9092
  hello, this is my first message
  hello, this is my second message
  1
  2
  3

参考资料

http://kafka.apache.org/documentation/#quickstart