设为首页 加入收藏

TOP

Flume学习12_Flume Spooling directory source读取文件格式的要求
2018-11-28 17:45:49 】 浏览:48
Tags:Flume 学习 12_Flume Spooling directory source 读取 文件 格式 要求

把Flume的Source设置为Spooling directory source,在设定的目录下放置需要读取的文件,一些文件在读取过程中会报错。

文件格式和报错如下:

实验一

读取汉子和“;"符号的TXT文档

报一下错误

2015-11-06 22:16:02,386 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-2 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)

实验二

读取数字和”;“符号的TXT文档

报错

2015-11-06 22:22:49,012 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-3 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
此报错与前面的报错相同。


实验三

读取数字和”回车符“的TXT文档

报错:

2015-11-06 22:28:06,889 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-7 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)


实验四

读取数字,字母,汉字组成的csv文件

报错:

2015-11-06 22:40:34,781 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-6 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
	at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)
	at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)


实验五

读取数字和汉字组成的TXT文件

报错

	2015-11-06 23:34:29,455 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-10 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
<span style="white-space:pre">	</span>at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
<span style="white-space:pre">	</span>at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
<span style="white-space:pre">	</span>at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)
<span style="white-space:pre">	</span>at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)
<span style="white-space:pre">	</span>at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)
<span style="white-space:pre">	</span>at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)
<span style="white-space:pre">	</span>at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
<span style="white-space:pre">	</span>at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
<span style="white-space:pre">	</span>at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
<span style="white-space:pre">	</span>at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)


成功的实验一

一排数字,什么符号都没有


成功的实验二

如上所示,不缺定是否使用了”回车符“。


】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
上一篇自定义flume拦截器-实现了多种功能 下一篇flume 自定义Sink之kafkaSink

最新文章

热门文章

Hot 文章

Python

C 语言

C++基础

大数据基础

linux编程基础

C/C++面试题目