(一.zookeeper伪集群搭建)
1.创建3个zoo.cfg(zoo1.cfg,zoo2.cfg,zoo3.cfg)
zoo1.cfg 设置:
# 数据文件夹
dataDir=/usr/local/zookeeper/data/data1
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs1
clientPort=2181
server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889
zoo2.cfg 设置:
# 数据文件夹
dataDir=/usr/local/zookeeper/data/data2
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs2
clientPort=2182
server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889
# 数据文件夹
dataDir=/usr/local/zookeeper/data/data3
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs2
clientPort=2183
server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889
2.创建3个myid
在usr/local/zookeeper/data/datax下闯进myid文件作为标识符,对应server.x数字
bin/zkServer.sh start conf/zoo1.cfg
bin/zkServer.sh start conf/zoo2.cfg
bin/zkServer.sh start conf/zoo3.cfg
4.查看伪集群服务每个zookeeper状态
[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo1.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo1.cfg
Mode: follower
[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo2.cfg
Mode: leader
[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo3.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo3.cfg
Mode: follower
(二.kafka伪集群搭建)
1. 将配置文件拷贝多份
[root@master zookeeper-3.4.11]$cp config/server.properties config/server-1.properties
[root@master zookeeper-3.4.11]$cp config/server.properties config/server-2.properties
[root@master zookeeper-3.4.11]$cp config/server.properties config/server-3.properties
2.修改每个server-x.properties
server-1.properties设置:
broker.id=1
port=9092
host.name=192.168.94.132
log.dirs=/usr/local/kafka/kafka-logs-1
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true
server-2.properties设置:
broker.id=2
port=9093
host.name=192.168.94.132
log.dirs=/usr/local/kafka/kafka-logs-2
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true
server-3.properties设置:
broker.id=3
port=9093
host.name=192.168.94.132
log.dirs=/usr/local/kafka/kafka-logs-3
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true
每个kafka的broker.id不一样,作为唯一标识符
(三)启动集群测试
注：启动时：先启动 zookeeper，后启动 kafka；关闭时：先关闭 kafka，后关闭zookeeper
1. 分别在每个节点上启动zookeeper
[root@master zookeeper-3.4.11]$ bin/zkServer.sh start conf/zoo1.cfg
[root@worker1 zookeeper-3.4.11]$ bin/zkServer.sh startconf/zoo2.cfg
[root@worker2 zookeeper-3.4.11]$ bin/zkServer.sh start conf/zoo3.cfg
2.验证zookeeper集群
# 在master节点上
[root@master zookeeper-3.4.11]$ bin/zkServer.sh statusconf/zoo1.cfg
# 在worker1节点上
[root@worker1 zookeeper-3.4.11]$ bin/zkServer.sh statusconf/zoo2.cfg
# 在worker2节点上
[root@worker2 zookeeper-3.4.11]$ bin/zkServer.sh status conf/zoo3.cfg
显示结果为：有一个是leader，剩下的都是follower
3.启动kafka集群
后台启动方式:
[root@master kafka_2.12-1.1.0]$ bin/kafka-server-start.sh -daemon config/server-1.properties
[root@master kafka_2.12-1.1.0]$ bin/kafka-server-start.sh config/server.properties config/server-1.properties
[root@worker1 kafka_2.12-1.1.0]$ bin/kafka-server-start.sh config/server.properties config/server-1.properties
[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-start.sh config/server.properties config/server-1.properties
4.测试kafka集群
4.1 创建 topic 和 显示 topic 信息
# 创建topic
bin/kafka-topics.sh --create --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --replication-factor 3 --partitions 3 --topic test
#显示topic信息
bin/kafka-topics.sh --describe --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test
# 列出topic
bin/kafka-topics.sh --list --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
4.2 创建 producer
bin/kafka-console-producer.sh --broker-list master:9092 -topic test
4.3 创建 consumer
# 测试消费 也可分别做测试
bin/kafka-console-consumer.sh --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 -topic test --from-beginning
然后在 producer 里输入消息，consumer 中就会显示出同样的内容，表示消费成功
4.3删除节点
bin/kafka-topics.sh --delete --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test
(四)关闭集群服务
#删除topic
bin/kafka-topics.sh --delete --zookeeper192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test
# 关闭kafka
[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-1.properties
[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-2.properties
[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-3.properties
# 关闭zookeeper
[root@master zookeeper-3.4.11]$ bin/zkServer.sh stop conf/zoo1.cfg
[root@worker1 zookeeper-3.4.11]$ bin/zkServer.shstop conf/zoo2.cfg
[root@worker2 zookeeper-3.4.11]$ bin/zkServer.shstop conf/zoo3.cfg