flume

来源：15-2 -流处理语义详解

慕先生4044781

2019-08-23

老师您好，我用SpoolDir监控一个文件夹，放进去一个一万条日志的文件，报这种错。导致sparkStreaming一直不停重复消费。望老师帮忙看一下，谢谢！
flume报错如下：
2019-08-23 16:07:59,580 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource $SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source avro-source: { spoolDir: /root/data }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing. java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:281) at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195) at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:134) at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:72) at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:91) at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:238) at org.apache.flume.source.SpoolDirectorySource$ SpoolDirectoryRunnable.run(SpoolDirectorySource.java:227)
at java.util.concurrent.Executors $R u n n a b l e A d a p t e r . c a l l (E x e c u t o r s . j a v a : 511) a t j a v a . u t i l . c o n c u r r e n t . F u t u r e T a s k . r u n A n d R e s e t (F u t u r e T a s k . j a v a : 308) a t j a v a . u t i l . c o n c u r r e n t . S c h e d u l e d T h r e a d P o o l E x e c u t o r$ ScheduledFutureTask.access $301 (S c h e d u l e d T h r e a d P o o l E x e c u t o r . j a v a : 180) a t j a v a . u t i l . c o n c u r r e n t . S c h e d u l e d T h r e a d P o o l E x e c u t o r$ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)