运行完一个批次之后就直接报错
来源:9-10 -Pull方式整合之本地环境联调

qq_windybeyondw_0
2019-02-28
运行完一个批次之后就直接报错
ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0:
这个是啥原因,下面直接就报了连接不上,flush测试pull的时候
19/02/28 14:04:40 INFO scheduler.JobScheduler: Finished job streaming job 1551333880000 ms.0 from job set of time 1551333880000 ms
19/02/28 14:04:40 INFO scheduler.JobScheduler: Total delay: 0.400 s for time 1551333880000 ms (execution: 0.336 s)
19/02/28 14:04:40 INFO scheduler.ReceivedBlockTracker: Deleting batches:
19/02/28 14:04:40 INFO scheduler.InputInfoTracker: remove old batch metadata:
19/02/28 14:04:40 INFO receiver.ReceiverSupervisorImpl: Stopping receiver with message: Error starting receiver 0: java.io.IOException: Error connecting to /192.168.137.200:41414
19/02/28 14:04:40 INFO flume.FlumePollingReceiver: Shutting down Flume Polling Receiver
19/02/28 14:04:40 INFO receiver.ReceiverSupervisorImpl: Called receiver onStop
19/02/28 14:04:40 INFO receiver.ReceiverSupervisorImpl: Deregistering receiver 0
19/02/28 14:04:40 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - java.io.IOException: Error connecting to /192.168.137.200:41414
at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
at org.apache.avro.ipc.NettyTransceiver.(NettyTransceiver.java:203)
at org.apache.avro.ipc.NettyTransceiver.(NettyTransceiver.java:138)
at org.apache.spark.streaming.flume.FlumePollingReceiverKaTeX parse error: $ within math modeanonfun$onStart1.apply(FlumePollingInputDStream.scala:82)atscala.collection.immutable.List.foreach(List.scala:381)atorg.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:82)atorg.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)atorg.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)atorg.apache.spark.streaming.scheduler.ReceiverTracker1.apply(FlumePollingInputDStream.scala:82)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:82)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
at org.apache.spark.streaming.scheduler.ReceiverTracker1.apply(FlumePollingInputDStream.scala:82)atscala.collection.immutable.List.foreach(List.scala:381)atorg.apache.spark.streaming.flume.FlumePollingReceiver.onStart(FlumePollingInputDStream.scala:82)atorg.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)atorg.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)atorg.apache.spark.streaming.scheduler.ReceiverTrackerReceiverTrackerEndpointKaTeX parse error: $ within math modeanonfun9.apply(ReceiverTracker.scala:597)atorg.apache.spark.SparkContext9.apply(ReceiverTracker.scala:597)
at org.apache.spark.SparkContext9.apply(ReceiverTracker.scala:597)atorg.apache.spark.SparkContext$anonfun34.apply(SparkContext.scala:2173)atorg.apache.spark.SparkContext34.apply(SparkContext.scala:2173)
at org.apache.spark.SparkContext34.apply(SparkContext.scala:2173)atorg.apache.spark.SparkContext$anonfun34.apply(SparkContext.scala:2173)atorg.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)atorg.apache.spark.scheduler.Task.run(Task.scala:108)atorg.apache.spark.executor.Executor34.apply(SparkContext.scala:2173)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor34.apply(SparkContext.scala:2173)atorg.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)atorg.apache.spark.scheduler.Task.run(Task.scala:108)atorg.apache.spark.executor.ExecutorTaskRunner.run(Executor.scala:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused: no further information: /192.168.137.200:41414
1回答
-
Michael_PK
2019-02-28
你机器的网络是否正常?这种连不上两个可能:一是网络,而是端口是否了
022019-02-28
相似问题