设为首页 加入收藏

TOP

Flume自定义开发常见异常及解决办法
2019-04-14 02:10:26 】 浏览:13
Tags:Flume 定义 开发 常见 异常 解决 办法
版权声明:欢迎转载,转载请说明出处. 大数据Github项目地址https://github.com/SeanYanxml/bigdata。 https://blog.csdn.net/u010416101/article/details/80604460

前言

前些日子,对Flume进行了自定义sink和source的开发。遇到一些异常,以后关于Flume的异常都统一记录与此处。(除了个别较为麻烦的异常处理)


正文

  • Flume source has been removed due to an error in configuration
    中文含义: Flume的source部分因为配置异常被移除。
    表象: Flume中source没有启动。
    解决办法:查看上下文,查看诸如java.lang.IllegalStateException: The parameter command must be specified类似的具体异常,具体定位问题。(一般是配置文件或者自定义source未写正确导致,可以try-catch捕获处理某些异常。)
05 六月 2018 16:52:57,528 ERROR [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadSinks:426)  - Sink epp has been removed due to an error during configuration
org.apache.flume.conf.ConfigurationException: Bootstrap Servers must be specified
	at org.apache.flume.sink.kafka.KafkaSink.translateOldProps(KafkaSink.java:353)
	at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:295)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
  • Flume agent unable to deliver event.
    分析: 和上个异常类似。 一般跟有ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.Sink类似的报错。

Reference

[1]. (stackoverflow)flume-source-has-been-removed-due-to-an-error-in-configuration
[2] (stackover)flume-agent-unable-to-deliver-event


编程开发网
】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
上一篇ICC副本>>>>(logback.. 下一篇nginx ---->flume ----->ka..

评论

帐  号: 密码: (新用户注册)
验 证 码:
表  情:
内  容:

array(4) { ["type"]=> int(8) ["message"]=> string(24) "Undefined variable: jobs" ["file"]=> string(32) "/mnt/wp/cppentry/do/bencandy.php" ["line"]=> int(214) }