按照教学视频里的流程运行,报错如下
来源:8-4 快速开发第一个Structured Streaming应用程序

酱汁怪兽
2022-03-18
一、依赖完全按照视频里的配置
log-sss的pom.xml中
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.12</artifactId>
</dependency>
</dependencies>
二、运行WordCount.scala
(1)报错1
Symbol 'type org.apache.spark.internal.Logging' is missing from the classpath.
This symbol is required by 'class org.apache.spark.sql.SparkSession'.
Make sure that type Logging is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'SparkSession.class' was compiled against an incompatible version of org.apache.spark.internal.
val spark = SparkSession
(2)报错2
Symbol 'type org.apache.spark.internal.Logging' is missing from the classpath.
This symbol is required by 'class org.apache.spark.sql.streaming.DataStreamReader'.
Make sure that type Logging is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'DataStreamReader.class' was compiled against an incompatible version of org.apache.spark.internal.
val lines = spark.readStream
(3)报错3
Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'value org.apache.spark.sql.SQLImplicits.evidence$7'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'SQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
.option("port", 9999)
(4)报错4
Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'method org.apache.spark.sql.LowPrioritySQLImplicits.newProductEncoder'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'LowPrioritySQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
.option("port", 9999)
(5)报错5
Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'value org.apache.spark.sql.Dataset.evidence$2'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'Dataset.class' was compiled against an incompatible version of org.apache.spark.sql.
val words = lines.as[String].flatMap(_.split(" "))
(6)报错6
could not find implicit value for evidence parameter of type org.apache.spark.sql.Encoder[String]
val words = lines.as[String].flatMap(_.split(" "))
代码部分依赖部分
写回答
1回答
-
Michael_PK
2022-03-19
你看你的代码那个图,代码都抛红的,编译都不过呢。。。肯定运行不起来。
把你的代码对比git上提供的代码,看看有啥区别
00
相似问题