数仓工具—Hive进阶之MSCK(14)

MSCK

MSCK REPAIR TABLE命令主要是用来解决通过hdfs dfs -put或者hdfs api写入hive分区表的数据在hive中无法被查询到的问题。

我们知道hive有个服务叫metastore，这个服务主要是存储一些元数据信息，比如数据库名，表名或者表的分区等等信息。如果不是通过hive的insert等插入语句，很多分区信息在metastore中是没有的。

MSCK命令

Hive stores a list of partitions for each table in its metastore. If, however, new partitions are directly added to HDFS (say by using hadoop fs -put command) or removed from HDFS, the metastore (and hence Hive) will not be aware of these changes to partition information unless the user runs ALTER TABLE table_name ADD/DROP PARTITION commands on each of the newly added or removed partitions, respectively.

However, users can run a metastore check command with the repair table option:MSCK [REPAIR] TABLE table_name [ADD/DROP/SYNC PARTITIONS];

which will update metadata about partitions to the Hive metastore for partitions for which such metadata doesn't already exis
1
2
3
4