不能成功运行spark-submit

来源：5-4 -standalone模式spark-submit运行

ByronC

2019-11-16

运行了下面这句命令，出现了如下error，我在想是不是因为没有成功调用wc.txt文件（wc.txt文件已经成功上传到服务器，并且spark0403.py能正常运行）

命令：
spark-submit --master spark://bangrencs-MacBook-Pro.local:7077 --name spark-standalone spark0403.py hdfs://localhost:50070/wc.txt hdfs://localhost:50070/output

出现的Error：
py4j.protocol.Py4JJavaError: An error occurred while calling o18.partitions.
: java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: “bangrencs-MacBook-Pro.local/127.0.0.1”; destination host is: “localhost”:50070;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776)
at org.apache.hadoop.ipc.Client.call(Client.java:1479)
at org.apache.hadoop.ipc.Client.call(Client.java:1412)
at org.apache.hadoop.ipc.ProtobufRpcEngine $I n v o k e r . i n v o k e (P r o t o b u f R p c E n g i n e . j a v a : 229) a t c o m . s u n . p r o x y .$ Proxy22.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy23.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
at org.apache.hadoop.hdfs.DistributedFileSystem $22 . d o C a l l (D i s t r i b u t e d F i l e S y s t e m . j a v a : 1301) a t o r g . a p a c h e . h a d o o p . f s . F i l e S y s t e m L i n k R e s o l v e r . r e s o l v e (F i l e S y s t e m L i n k R e s o l v e r . j a v a : 81) a t o r g . a p a c h e . h a d o o p . h d f s . D i s t r i b u t e d F i l e S y s t e m . g e t F i l e S t a t u s (D i s t r i b u t e d F i l e S y s t e m . j a v a : 1317) a t o r g . a p a c h e . h a d o o p . f s . G l o b b e r . g e t F i l e S t a t u s (G l o b b e r . j a v a : 57) a t o r g . a p a c h e . h a d o o p . f s . G l o b b e r . g l o b (G l o b b e r . j a v a : 252) a t o r g . a p a c h e . h a d o o p . f s . F i l e S y s t e m . g l o b S t a t u s (F i l e S y s t e m . j a v a : 1676) a t o r g . a p a c h e . h a d o o p . m a p r e d . F i l e I n p u t F o r m a t . s i n g l e T h r e a d e d L i s t S t a t u s (F i l e I n p u t F o r m a t . j a v a : 259) a t o r g . a p a c h e . h a d o o p . m a p r e d . F i l e I n p u t F o r m a t . l i s t S t a t u s (F i l e I n p u t F o r m a t . j a v a : 229) a t o r g . a p a c h e . h a d o o p . m a p r e d . F i l e I n p u t F o r m a t . g e t S p l i t s (F i l e I n p u t F o r m a t . j a v a : 315) a t o r g . a p a c h e . s p a r k . r d d . H a d o o p R D D . g e t P a r t i t i o n s (H a d o o p R D D . s c a l a : 204) a t o r g . a p a c h e . s p a r k . r d d . R D D$ $a n o n f u n$ partitions $2 . a p p l y (R D D . s c a l a : 253) a t o r g . a p a c h e . s p a r k . r d d . R D D$ $a n o n f u n$ partitions $2 . a p p l y (R D D . s c a l a : 251) a t s c a l a . O p t i o n . g e t O r E l s e (O p t i o n . s c a l a : 121) a t o r g . a p a c h e . s p a r k . r d d . R D D . p a r t i t i o n s (R D D . s c a l a : 251) a t o r g . a p a c h e . s p a r k . r d d . M a p P a r t i t i o n s R D D . g e t P a r t i t i o n s (M a p P a r t i t i o n s R D D . s c a l a : 49) a t o r g . a p a c h e . s p a r k . r d d . R D D$ $a