hadoop 3.x 案例3:datanode 异常_hdfs中如果datanode损坏怎么办

一. 问题描述

hdfs web页面显示有的文件是损坏的，需要使用fsck来检查

二. 解决方案

通过fsck检查根目录，看是否有损坏的文件

hdfs fsck /
1

将检查到的损坏的文件进行删除

hdfs fsck /spark-jars -move
hdfs fsck /spark-jars -delete
hdfs fsck /spark-jars -files
1
2
3

测试记录:

[root@hadoop1 shell]# hdfs fsck /spark-jars -move
Connecting to namenode via http://hadoop1:9870/fsck?ugi=root&move=1&path=%2Fspark-jars
FSCK started by root (auth:SIMPLE) from /192.168.0.150 for path /spark-jars at Sun Nov 20 21:15:14 CST 2022


/spark-jars/JLargeArrays-1.5.jar: MISSING 1 blocks of total size 232470 B.
/spark-jars/JTransforms-3.1.jar: MISSING 1 blocks of total size 1175798 B.
/spark-jars/RoaringBitmap-0.9.0.jar: MISSING 1 blocks of total size 386529 B.
/spark-jars/aircompressor-0.21.jar: MISSING 1 blocks of total size 183588 B.
/spark-jars/algebra_2.12-2.0.1.jar: MISSING 1 blocks of total size 1168417 B.
/spark-jars/annotations-17.0.0.jar: MISSING 1 blocks of total size 19014 B.
/spark-jars/antlr4-runtime-4.8.jar: MISSING 1 blocks of total size 337864 B.
/spark-jars/aopalliance-repackaged-2.6.1.jar: MISSING 1 blocks of total size 27006 B.
/spark-jars/arpack-2.2.1.jar: MISSING 1 blocks of total size 77537 B.
/spark-jars/arpack_combined_all-0.1.jar: MISSING 1 blocks of total size 1194003 B.
/spark-jars/arrow-format-2.0.0.jar: MISSING 1 blocks of total size 72668 B.
......
......
......
/spark-jars/tink-1.6.0.jar: MISSING 1 blocks of total size 1323184 B.
/spark-jars/univocity-parsers-2.9.1.jar: MISSING 1 blocks of total size 447005 B.
/spark-jars/xbean-asm9-shaded-4.20.jar: MISSING 1 blocks of total size 294763 B.
/spark-jars/xz-1.8.jar: MISSING 1 blocks of total size 108555 B.
/spark-jars/zjsonpatch-0.3.0.jar: MISSING 1 blocks of total size 35518 B.
/spark-jars/zstd-jni-1.5.0-4.jar: MISSING 1 blocks of total size 6768681 B.
Status: CORRUPT
 Number of data-nodes:  1
 Number of racks:               1
 Total dirs:                    1
 Total symlinks:                0

Replicated Blocks:
 Total size:    199191055 B
 Total files:   170
 Total blocks (validated):      170 (avg. block size 1171712 B)
  ********************************
  UNDER MIN REPL'D BLOCKS:      170 (100.0 %)
  MINIMAL BLOCK REPLICATION:    1
  CORRUPT FILES:        170
  MISSING BLOCKS:       170
  MISSING SIZE:         199191055 B
  ********************************
 Minimally replicated blocks:   0 (0.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                170
 Corrupt blocks:                0
 Missing replicas:              0
 Blocks queued for replication: 0

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:   0
 Total block groups (validated):        0
 Minimally erasure-coded block groups:  0
 Over-erasure-coded block groups:       0
 Under-erasure-coded block groups:      0
 Unsatisfactory placement block groups: 0
 Average block group size:      0.0
 Missing block groups:          0
 Corrupt block groups:          0
 Missing internal blocks:       0
 Blocks queued for replication: 0
FSCK ended at Sun Nov 20 21:15:15 CST 2022 in 1408 milliseconds


The filesystem under path '/spark-jars' is CORRUPT
[root@hadoop1 shell]# hdfs fsck /spark-jars -delete
Connecting to namenode via http://hadoop1:9870/fsck?ugi=root&delete=1&path=%2Fspark-jars
FSCK started by root (auth:SIMPLE) from /192.168.0.150 for path /spark-jars at Sun Nov 20 21:15:59 CST 2022


/spark-jars/JLargeArrays-1.5.jar: MISSING 1 blocks of total size 232470 B.
......
/spark-jars/zstd-jni-1.5.0-4.jar: MISSING 1 blocks of total size 6768681 B.
Status: CORRUPT
 Number of data-nodes:  1
 Number of racks:               1
 Total dirs:                    1
 Total symlinks:                0

Replicated Blocks:
 Total size:    199191055 B
 Total files:   170
 Total blocks (validated):      170 (avg. block size 1171712 B)
  ********************************
  UNDER MIN REPL'D BLOCKS:      170 (100.0 %)
  MINIMAL BLOCK REPLICATION:    1
  CORRUPT FILES:        170
  MISSING BLOCKS:       170
  MISSING SIZE:         199191055 B
  ********************************
 Minimally replicated blocks:   0 (0.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                170
 Corrupt blocks:                0
 Missing replicas:              0
 Blocks queued for replication: 0

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:   0
 Total block groups (validated):        0
 Minimally erasure-coded block groups:  0
 Over-erasure-coded block groups:       0
 Under-erasure-coded block groups:      0
 Unsatisfactory placement block groups: 0
 Average block group size:      0.0
 Missing block groups:          0
 Corrupt block groups:          0
 Missing internal blocks:       0
 Blocks queued for replication: 0
FSCK ended at Sun Nov 20 21:16:00 CST 2022 in 755 milliseconds


The filesystem under path '/spark-jars' is CORRUPT
[root@hadoop1 shell]# 
[root@hadoop1 shell]# 
[root@hadoop1 shell]# 
[root@hadoop1 shell]# hdfs fsck /spark-jars -files
Connecting to namenode via http://hadoop1:9870/fsck?ugi=root&files=1&path=%2Fspark-jars
FSCK started by root (auth:SIMPLE) from /192.168.0.150 for path /spark-jars at Sun Nov 20 21:16:35 CST 2022

/spark-jars <dir>

Status: HEALTHY
 Number of data-nodes:  1
 Number of racks:               1
 Total dirs:                    1
 Total symlinks:                0

Replicated Blocks:
 Total size:    0 B
 Total files:   0
 Total blocks (validated):      0
 Minimally replicated blocks:   0
 Over-replicated blocks:        0
 Under-replicated blocks:       0
 Mis-replicated blocks:         0
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                0
 Corrupt blocks:                0
 Missing replicas:              0
 Blocks queued for replication: 0

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:   0
 Total block groups (validated):        0
 Minimally erasure-coded block groups:  0
 Over-erasure-coded block groups:       0
 Under-erasure-coded block groups:      0
 Unsatisfactory placement block groups: 0
 Average block group size:      0.0
 Missing block groups:          0
 Corrupt block groups:          0
 Missing internal blocks:       0
 Blocks queued for replication: 0
FSCK ended at Sun Nov 20 21:16:35 CST 2022 in 0 milliseconds


The filesystem under path '/spark-jars' is HEALTHY
[root@hadoop1 shell]# 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171

参考:

1. https://blog.csdn.net/aohuang8877/article/details/101116099