设为首页 加入收藏

TOP

flume负载均衡版配置
2019-03-14 14:06:58 】 浏览:8
Tags:flume 负载 均衡 配置

flume负载均衡的意思:由node01 发送数据给node02 和node03 ,由他们俩分别轮询接收消息

设置node01,在flume安装包的conf目录下新建load_balancer.conf

#命名
a1.sources = r1
a1.channels = c1
a1.sinks = k1 k2

#设置资源
a1.sources.r1.type = exec
a1.sources.r1.command = tail -F /export/servers/taillogs/access_log

#设置通道channels
a1.channels.c1.type = memory
a1.channels.c1.capatity = 1000
a1.channels.c1.transactionCapacity = 100

#设置sink1
a1.sinks.k1.type=avro
a1.sinks.k1.hostname=node02
a1.sinks.k1.port=52020

#设置sink2
a1.sinks.k2.type=avro
a1.sinks.k2.hostname=node03
a1.sinks.k2.port=52020

#关系
a1.sources.r1.channels=c1
a1.sinks.k1.channel=c1
a1.sinks.k2.channel=c1

#设置proceecing
a1.sinkgroups=g1
a1.sinkgroups.g1.sinks=k1 k2
a1.sinkgroups.g1.processor.type=load_balance
a1.sinkgroups.g1.processor.backoff=true
a1.sinkgroups.g1.processor.selector=round_robin
a1.sinkgroups.g1.processor.selector.maxTimeOut=1000

设置node02 ,新建load_banlancer_server.conf

#命名
a1.sources = r1
a1.sinks = k1
a1.channels = c1

#资源
a1.sources.r1.type = avro
a1.sources.r1.bind = node02
a1.sources.r1.port = 52020

# sink
a1.sinks.k1.type = logger

#通道
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# 连接
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

设置node02 ,新建load_banlancer_server.conf

#命名
a1.sources = r1
a1.sinks = k1
a1.channels = c1

#资源
a1.sources.r1.type = avro
a1.sources.r1.bind = node03
a1.sources.r1.port = 52020

# sink
a1.sinks.k1.type = logger

#通道
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# 连接
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

分别开启node02 node03 和node01

bin/flume-ng agent -n a1 -c conf -f conf/load_banlancer_server.conf -Dflume.root.logger=DEBUG,console

往node01的/export/servers/taillogs/access_log追加数据看node02 和node03的变化

结果如下,数字都是字节流


编程开发网
】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
上一篇Flume实时读取目录文件到HDFS案例 下一篇flume配置文件和实际场景理解

评论

帐  号: 密码: (新用户注册)
验 证 码:
表  情:
内  容:

array(4) { ["type"]=> int(8) ["message"]=> string(24) "Undefined variable: jobs" ["file"]=> string(32) "/mnt/wp/cppentry/do/bencandy.php" ["line"]=> int(214) }