赞
踩
DML: 会按字段顺序插入,select出的前面的字段会放入非分区字段中,后面的几个字段则会写到分区字段中。
- use bdm;
- set mapred.output.compress=true;
- set hive.exec.compress.output=true;
- set mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;
- set hive.exec.dynamic.partition=true; -- 可以使用动态分区
- set hive.exec.dynamic.partition.mode=nonstrict; -- 所有分区字段都可以由查询得出。
- set hive.exec.max.dynamic.partitions=100000;
- set hive.exec.max.dynamic.partitions.pernode=100000;
- set hive.exec.reducers.bytes.per.reducer = 400000000;
- set hive.exec.parallel=true;
- set hive.auto.convert.join=true;
- set mapreduce.job.running.map.limit=50000;
- set yarn.app.mapreduce.am.command-opts=-Xmx8192m;
- set yarn.app.mapreduce.am.resource.mb=9216;
- set hive.lzo.paralle.read.index.thread = 50;
-
- INSERT OVERWRITE TABLE tmp.test_dym_par_1121 PARTITION (dt,data_type,dh)
- SELECT
- report_time
- ,'2019-11-21' as dt
- ,'TIME' a
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。