老师,您好,提交wordCount作业的时候报错,我想让您帮我看看该如何解决

来源:1-20 -YARN环境搭建及提交作业到YARN上运行

陈素素

2019-03-20

[hadoop@hadoop001 bin]$ hadoop jar /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar wordcount /input/wc/hello.txt /output/wc/
/home/hadoop/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop/hadoop-env.sh: line 2: or: command not found
19/03/19 23:35:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
19/03/19 23:35:05 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/03/19 23:35:05 WARN security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hadoop001:8020/output/wc already exists
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hadoop001:8020/output/wc already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:270)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:143)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)

写回答

2回答

Michael_PK

2019-03-20

你现在的yarn能提交作业正常运行吗?提交一个自带的wordcount试试。

InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist

看这错,像是yarn的配置有问题

0
3
Michael_PK
回复
陈素素
这是啥操作引起的呢?描述好上下文。。。
2019-03-20
共3条回复

陈素素

提问者

2019-03-20

像是提交上了,但是就感觉是配置有问题。我该如何修改呢?

0
4
陈素素
回复
Michael_PK
Ok,谢谢老师
2019-03-20
共4条回复

以慕课网日志分析为例 进入大数据Spark SQL的世界

快速转型大数据:Hadoop,Hive,SparkSQL步步为赢

1644 学习 · 1129 问题

查看课程