当前位置:   article > 正文

hive java.lang.OutOfMemoryError: Java heap space Vertex did not succeed due to OWN_TASK_FAILURE_hive vertex did not succeed due to own_task_failur

hive vertex did not succeed due to own_task_failure

当你使用的yarn,执行hive,将数据插入分桶表时,报如下错误时:

java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1534574591143_0017_235_00, diagnostics=[Task failed, taskId=task_1534574591143_0017_235_00_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.OutOfMemoryError: Java heap space
, errorMessage=Cannot recover from this error:java.lang.OutOfMemoryError: Java heap space
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:8, Vertex vertex_1534574591143_0017_235_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1534574591143_0017_235_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:3, Vertex vertex_1534574591143_0017_235_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
        at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:363)
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:256)
        at org.apache.hive.hplsql.Conn.executeSql(Conn.java:110)
        at org.apache.hive.hplsql.Exec.executeSql(Exec.java:592)
        at org.apache.hive.hplsql.Stmt.insertSelect(Stmt.java:725)
        at org.apache.hive.hplsql.Stmt.insert(Stmt.java:701)
        at org.apache.hive.hplsql.Exec.visitInsert_stmt(Exec.java:1101)
        at org.apache.hive.hplsql.Exec.visitInsert_stmt(Exec.java:52)
        at org.apache.hive.hplsql.HplsqlParser$Insert_stmtContext.accept(HplsqlParser.java:10161)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
        at org.apache.hive.hplsql.Exec.visitStmt(Exec.java:994)
        at org.apache.hive.hplsql.Exec.visitStmt(Exec.java:52)
        at org.apache.hive.hplsql.HplsqlParser$StmtContext.accept(HplsqlParser.java:1007)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
        at org.apache.hive.hplsql.HplsqlBaseVisitor.visitBlock(HplsqlBaseVisitor.java:28)
        at org.apache.hive.hplsql.HplsqlParser$BlockContext.accept(HplsqlParser.java:444)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
        at org.apache.hive.hplsql.Exec.visitBegin_end_block(Exec.java:911)
        at org.apache.hive.hplsql.Exec.visitBegin_end_block(Exec.java:52)
        at org.apache.hive.hplsql.HplsqlParser$Begin_end_blockContext.accept(HplsqlParser.java:539)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
        at org.apache.hive.hplsql.HplsqlBaseVisitor.visitBlock(HplsqlBaseVisitor.java:28)
        at org.apache.hive.hplsql.HplsqlParser$BlockContext.accept(HplsqlParser.java:444)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
        at org.apache.hive.hplsql.Exec.visitProgram(Exec.java:901)
        at org.apache.hive.hplsql.Exec.visitProgram(Exec.java:52)
        at org.apache.hive.hplsql.HplsqlParser$ProgramContext.accept(HplsqlParser.java:387)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:42)
        at org.apache.hive.hplsql.Exec.run(Exec.java:760)
        at org.apache.hive.hplsql.Exec.run(Exec.java:736)
        at org.apache.hive.hplsql.Hplsql.main(Hplsql.java:23)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
[hive@GXWQdatanode03 bin]$
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41

通过搜索,找到了解决的方案,来源下面这两篇文章,你自己针对情况做修改吧。
参考链接
参考链接

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/小丑西瓜9/article/detail/539700
推荐阅读
相关标签
  

闽ICP备14008679号