已解决：FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

问题描述

1. 今天在跑hive sql的时候，报错;FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask【从hive log找到的错误提示】，详细的报错如下所示：时隔近一年，又遇到这个报错了，蛮无语的，看不出啥原因导致的。。。

   org.apache.hive.service.cli.HiveSQLException: 
   
   Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask 
   
   at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) 
   
   at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257) 
   
   at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) 
   
   at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348) 
   
   at java.security.AccessController.doPrivileged(Native Method) 
   
   at javax.security.auth.Subject.doAs(Subject.java:422) 
   
   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) 
   
   at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362) 
   
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
   
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
   
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
   
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27

2. 在网上查了蛮久的，找到一个解决方案

解决方案

【参考方案】

1. 参考错误原因：

Cause:
The above issue occurs when there are multiple jobs triggered and Hive removes a session directory for some application failure while Tez Application Master is still using it. The Tez Application Master staging directory is part of Hive Scratch directory which is controlled by the Hive Session.
Solution:
To resolve this issue, block the closing of sessions until tez AM shuts down


【中文描述就是这个样子：】
当触发了多个作业，并且当Tez application Master仍在使用时，Hive删除了某个应用程序故障的会话目录时，就会出现上述问题。
Tez application Master暂存目录是Hive Scratch目录的一部分，该目录由Hive session控制。

解决方案：要解决此问题，请阻止会话关闭，直到tez AM关闭
1
2
3
4
5
6
7
8
9
10
11

2. 在执行sql前设置：set tez.client.asynchronous-stop=false，然后就正常执行了。。。

【未完待续-…先正常能跑，有时间了再深入研究下】

快一年了，又遇到这个情况了，这次必须解决掉！