spark-submit yarn报错

来源:2-17 YARN模式下提交Spark应用程序

慕九州3016327

2020-08-21

[hadoop@hadoop000 bin]$ ./spark-submit \

–class com.imooc.bigdata.chapter01.SparkWordCountAppV2
–master yarn \
–name SparkWordCountAppV2
/home/hadoop/lib/sparksql-train-1.0.jar
hdfs://hadoop000:8020/meng/wc.data hdfs://hadoop000:8020/meng/out20/08/22 06:51:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

20/08/22 06:51:42 WARN deploy.DependencyUtils: Local jar /home/hadoop/app/spark-2.4.6-bin-hadoop2.6/bin/ does not exist, skipping.
20/08/22 06:51:42 WARN deploy.SparkSubmitKaTeX parse error: $ within math moderunMain(SparkSubmit.scala:806)
at org.apache.spark.deploy.SparkSubmit.doRunMain1(SparkSubmit.scala:161)atorg.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)atorg.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)atorg.apache.spark.deploy.SparkSubmit1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit1(SparkSubmit.scala:161)atorg.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)atorg.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)atorg.apache.spark.deploy.SparkSubmit$anon2.doSubmit(SparkSubmit.scala:920)atorg.apache.spark.deploy.SparkSubmit2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit2.doSubmit(SparkSubmit.scala:920)atorg.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/08/22 06:51:42 INFO util.ShutdownHookManager: Shutdown hook called
20/08/22 06:51:42 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-497fb498-88bf-48d1-a44e-a631d1f43e3f

老师我这个是哪地方弄错了,研究了好久也没找到问题。

写回答

1回答

Michael_PK

2020-08-21

–class com.imooc.bigdata.chapter01.SparkWordCountAppV2
–master yarn \
–name SparkWordCountAppV2
/home/hadoop/lib/sparksql-train-1.0.jar


这些换行的都需要斜杠的,你看你写的脚本,一会有一会没有

0
2
Michael_PK
回复
慕九州3016327
只要是一行内就不用斜杠,只要多行就需要
2020-08-22
共2条回复

SparkSQL入门 整合Kudu实现广告业务数据分析

大数据工程师干货课程 带你从入门到实战掌握SparkSQL

535 学习 · 192 问题

查看课程