执行日志清理时退出

来源:9-25 -数据清洗作业运行到YARN上

慕粉2110073833

2017-11-10

老师下面是,打出的错误是指信息,是由于资源不够吗?  使用的是阿里云的服务器2核4G的, hadoop后台显示的Memory Total是8G

17/11/09 23:57:33 ERROR scheduler.TaskSetManager: Task 0 in stage 1.0 failed 4 times; aborting job

17/11/09 23:57:33 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool 

17/11/09 23:57:33 INFO storage.BlockManagerMaster: Removal of executor 4 requested

17/11/09 23:57:33 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asked to remove non-existent executor 4

17/11/09 23:57:33 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 4 from BlockManagerMaster.

17/11/09 23:57:33 INFO cluster.YarnScheduler: Cancelling stage 1

17/11/09 23:57:33 INFO scheduler.DAGScheduler: ResultStage 1 (save at SparkStatCleanJobYARN.scala:29) failed in 82.394 s due to Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4, hadoop001, executor 4): ExecutorLostFailure (executor 4 exited caused by one of the running tasks) Reason: Container marked as failed: container_1510122812790_0009_02_000002 on host: hadoop001. Exit status: 1. Diagnostics: Exception from container-launch.

Container id: container_1510122812790_0009_02_000002

Exit code: 1

Stack trace: ExitCodeException exitCode=1: 

at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)

at org.apache.hadoop.util.Shell.run(Shell.java:478)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)



Container exited with a non-zero exit code 1


写回答

1回答

Michael_PK

2017-11-10

从这日志看应该是yarn有问题,你到失败作业的yarn页面上,仔细查看下各个链接的日志信息

0
2
Michael_PK
回复
慕粉2110073833
这个错你搜索下看看
2017-11-10
共2条回复

以慕课网日志分析为例 进入大数据Spark SQL的世界

快速转型大数据:Hadoop,Hive,SparkSQL步步为赢

1644 学习 · 1129 问题

查看课程