按照教学视频里的流程运行,报错如下

来源:8-4 快速开发第一个Structured Streaming应用程序

酱汁怪兽

2022-03-18

一、依赖完全按照视频里的配置
log-sss的pom.xml中

    <dependencies>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.12</artifactId>
        </dependency>
     </dependencies>

二、运行WordCount.scala
(1)报错1

Symbol 'type org.apache.spark.internal.Logging' is missing from the classpath.
This symbol is required by 'class org.apache.spark.sql.SparkSession'.
Make sure that type Logging is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'SparkSession.class' was compiled against an incompatible version of org.apache.spark.internal.
    val spark = SparkSession

(2)报错2

Symbol 'type org.apache.spark.internal.Logging' is missing from the classpath.
This symbol is required by 'class org.apache.spark.sql.streaming.DataStreamReader'.
Make sure that type Logging is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'DataStreamReader.class' was compiled against an incompatible version of org.apache.spark.internal.
    val lines = spark.readStream

(3)报错3

Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'value org.apache.spark.sql.SQLImplicits.evidence$7'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'SQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
      .option("port", 9999)

(4)报错4

Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'method org.apache.spark.sql.LowPrioritySQLImplicits.newProductEncoder'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'LowPrioritySQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
      .option("port", 9999)

(5)报错5

Symbol 'type org.apache.spark.sql.Encoder' is missing from the classpath.
This symbol is required by 'value org.apache.spark.sql.Dataset.evidence$2'.
Make sure that type Encoder is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'Dataset.class' was compiled against an incompatible version of org.apache.spark.sql.
    val words = lines.as[String].flatMap(_.split(" "))

(6)报错6

could not find implicit value for evidence parameter of type org.apache.spark.sql.Encoder[String]
    val words = lines.as[String].flatMap(_.split(" "))

图片描述

图片描述

图片描述


代码部分
图片描述
图片描述依赖部分

写回答

1回答

Michael_PK

2022-03-19

你看你的代码那个图,代码都抛红的,编译都不过呢。。。肯定运行不起来。

把你的代码对比git上提供的代码,看看有啥区别

0
0

Spark3实时处理-Streaming+StructuredStreaming实战

实战Spark3实时处理,掌握两套企业级处理方案

340 学习 · 238 问题

查看课程