SparkStreaming PULL方式整合Flume,在启动IDEA报错: Connection refused: /0.0.0.0:41414

来源:9-10 -Pull方式整合之本地环境联调

hwbaker

2018-02-04


18/02/04 14:56:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - java.io.IOException: Error connecting to /0.0.0.0:41414

at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)

at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)

at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:138)

at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$1.apply(FlumePollingInputDStream.scala:83)

at org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$1.apply(FlumePollingInputDStream.scala:82)

at scala.collection.immutable.List.foreach(List.scala:381)

at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:82)

at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)

at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)

at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:607)

at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:597)

at org.apache.spark.SparkContext$$anonfun$34.apply(SparkContext.scala:2173)

at org.apache.spark.SparkContext$$anonfun$34.apply(SparkContext.scala:2173)

at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)

at org.apache.spark.scheduler.Task.run(Task.scala:108)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.net.ConnectException: Connection refused: /0.0.0.0:41414


写回答

1回答

Michael_PK

2018-02-04

414!4端口是否通?

0
1
hwbaker
端口是通的。 telnet localhost 41414是通的。
2018-02-05
共1条回复

Spark Streaming实时流处理项目实战

Flume+Kafka+Spark Streaming 构建通用实时流处理平台

1404 学习 · 571 问题

查看课程