连接未能建立，broker可能无效

来源：8-10 对接Kafka数据源数据

酱汁怪兽

2022-04-19

描述：

 spark
        .readStream
        .format("kafka")
        .option("kafka.bootstrap.servers", "192.168.50.166:9092, 192.168.50.166:9093, 192.168.50.166:9094")
        .option("subscribe", "localtest-replicated-topic")
        .load()
        .selectExpr("CAST(value AS STRING)")
        .as[String].flatMap(_.split("/t"))
        .writeStream
        .format("console")
        .start()
        .awaitTermination()

kafka的consumer是正常接收数据，scala的控制台还是未展示数据。
代码中，云主机名字改为了ip地址，就报如下警告：

22/04/19 10:17:16 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0-1, groupId=spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0] Connection to node 2 (localhost/127.0.0.1:9094) could not be established. Broker may not be available.
22/04/19 10:17:19 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0-1, groupId=spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
22/04/19 10:17:21 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0-1, groupId=spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0] Connection to node 1 (localhost/127.0.0.1:9093) could not be established. Broker may not be available.
22/04/19 10:17:23 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0-1, groupId=spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0] Connection to node 2 (localhost/127.0.0.1:9094) could not be established. Broker may not be available.
22/04/19 10:17:25 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0-1, groupId=spark-kafka-source-c277856e-75cc-4616-87fa-fba913ddf4cc--799407624-driver-0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

写回答

1回答