admin管理员组

文章数量:1642237

前言

前些日子，对Flume进行了自定义sink和source的开发。遇到一些异常，以后关于Flume的异常都统一记录与此处。(除了个别较为麻烦的异常处理)

正文

  • Flume source has been removed due to an error in configuration
    中文含义: Flume的source部分因为配置异常被移除。
    表象: Flume中source没有启动。
    解决办法：查看上下文，查看诸如java.lang.IllegalStateException: The parameter command must be specified类似的具体异常，具体定位问题。(一般是配置文件或者自定义source未写正确导致，可以try-catch捕获处理某些异常。)
05 六月 2018 16:52:57,528 ERROR [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadSinks:426)  - Sink epp has been removed due to an error during configuration
org.apache.flume.conf.ConfigurationException: Bootstrap Servers must be specified
	at org.apache.flume.sink.kafka.KafkaSink.translateOldProps(KafkaSink.java:353)
	at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:295)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
  • Flume agent unable to deliver event.
    分析: 和上个异常类似。 一般跟有ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.Sink类似的报错。

Reference

[1]. (stackoverflow)flume-source-has-been-removed-due-to-an-error-in-configuration
[2] (stackover)flume-agent-unable-to-deliver-event

本文标签： 自定义解决办法异常常见flume

版权声明：本文标题：Flume自定义开发常见异常及解决办法 内容由热心网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：https://m.elefans.com/dianzi/1729332833a1196612.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。