hdfs fsck命令检查文件系统健康状态、处理missing blocks问题、hdfs fsck / -delete执行失败_hadoop there are missing block

前言

  hadoop集群由于磁盘损坏、节点挂机、机架掉电难免造成一些文件块损坏，通过hdfs fsck命令，检查HDFS上⽂件和⽬录的健康状态、获取⽂件的block信息和位置信息

如，出现报错There are 41 missing blocks. The following files may be corrupted

---

1. 命令详解

命令：

 hdfs fsck <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-includeSnapshots] [-storagepolicies] [-blockId <blk_Id>]
1

参数含义：

	<path>	检测的目录
	-move	把损坏的文件移动到/lost+found
	-delete	直接删除损坏的文件
	-files	打印被检测的文件
	-openforwrite	print out files opened for write
	-includeSnapshots	include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it
	-list-corruptfileblocks	print out list of missing blocks and files they belong to
	-blocks	打印 block 的信息
	-locations	打印 block 的位置，即在哪个节点
	-racks	print out network topology for data-node locations
	-storagepolicies	打印 block 存储的策略信息
	-blockId	打印指定blockId所属块的状况
	-replicaDetails	print out each replica details 
1
2
3
4
5
6
7
8
9
10
11
12
13

---

2. 使用

  首先执行hdfs fsck / -delete，发现执行hdfs fsck / -delete失败，查看hdfs状态

Safe mode is ON. Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
1

执行hdfs dfsadmin -safemode leave，再运行hdfs fsck / -delete

检查状态hdfs fsck /

---