datanode出问题,不能用open,create读文件,写文件

来源:3-20 HDFS API编程之创建文件并写入内容

慕粉4125737

2019-05-27

和另一个同学的问题相似
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hdfsapi/test/a.txt could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

但是我是在VM上实验的。也是idea里面运行create代码,然后就报了上面的错误。
我参考了baidu到的一些解决方案,感觉和他们的情况不一样。
hadoop dfsadmin -report 运行这句命令后看到如下:

DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

19/05/27 08:02:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Configured Capacity: 18700304384 (17.42 GB)
Present Capacity: 16119177216 (15.01 GB)
DFS Remaining: 16119148544 (15.01 GB)
DFS Used: 28672 (28 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

-------------------------------------------------
Live datanodes (1):

Name: 10.0.0.14:50010 (hadoop000)
Hostname: localhost
Decommission Status : Normal
Configured Capacity: 18700304384 (17.42 GB)
DFS Used: 28672 (28 KB)
Non DFS Used: 2581127168 (2.40 GB)
DFS Remaining: 16119148544 (15.01 GB)
DFS Used%: 0.00%
DFS Remaining%: 86.20%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Mon May 27 08:02:26 CST 2019

另外还有一点很奇怪,我mkdir是成功的。

写回答

2回答

慕粉4125737

提问者

2019-05-27

这个问题后来解决了。50010 这个端口没有打开。

2
1
慕标7323975
你是用的云服务器吗?
2019-06-17
共1条回复

Michael_PK

2019-05-27

你的hadoop环境已经不能正常工作了,一个副本都找不到地方存了。这问题贴Google,一堆解决方案

0
0

Hadoop 系统入门+核心精讲

从Hadoop核心技术入手,掌握数据处理中ETL应用,轻松进军大数据

2408 学习 · 908 问题

查看课程