devbox
喵喵爱编程
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

flink checkpoint失败问题_org.apache.flink.runtime.rest.handler.resthandlere

作者：喵喵爱编程 | 2024-07-12 21:54:11

赞

踩

org.apache.flink.runtime.rest.handler.resthandlerexception: could not find c

1、flink ckpt失败的问题  

https://mp.weixin.qq.com/s/5o8rbz07Je17ZHnRz97-Rg

 

2、先去jobmanager去查询相关日志，说明flinksql有点问题

 

  1.  
  2. java.lang.Exception: Could not complete snapshot 1 for operator GroupAggregate(groupBy: (user_id), select: (user_id, KeyByFunction(properties) AS properties, KeyByFunction(event) AS event, KeyByFunction(current_page) AS current_page, KeyByFunction(block_type) AS block_type, KeyByFunction(content_tag_id) AS content_tag_id, KeyByFunction(author_id) AS author_id, KeyByFunction(category_id_new) AS category_id_new, KeyByFunction(cspu_id) AS cspu_id, KeyByFunction(sex) AS sex, KeyByFunction(sexy) AS sexy, KeyByFunction(cspu_ids) AS cspu_ids, KeyByFunction(content_id) AS content_id, KeyByFunction(content_type) AS content_type, KeyByFunction(time) AS time, KeyByFunction(circle_id) AS circle_id)) -> Calc(select: (properties, user_id, event, current_page, block_type, content_tag_id, author_id, category_id_new, cspu_id, sex, sexy, cspu_ids, content_id, content_type, ((event = 'community_content_click':VARCHAR(65536)) CASE CAST(CAST(((properties MULTI_JSON_VALUE '$.algorithm_channel_Id' MULTI_JSON_VALUE '$.feedback_type' MULTI_JSON_VALUE '$.community_content_i... (11/24).
  3. 	at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:422)
  4. 	at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.checkpointStreamOperator(StreamTask.java:1522)
  5. 	at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.executeCheckpointing(StreamTask.java:1445)
  6. 	at org.apache.flink.streaming.runtime.tasks.StreamTask.checkpointState(StreamTask.java:1034)
  7. 	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$5(StreamTask.java:880)
  8. 	at org.apache.flink.util.checkpointlock.SynchronizedLockHelper.lockAndRun(SynchronizedLockHelper.java:58)
  9. 	at org.apache.flink.util.checkpointlock.CheckpointLockDelegate.lockAndRun(CheckpointLockDelegate.java:57)
  10. 	at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:855)
  11. 	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:815)
  12. 	at org.apache.flink.streaming.runtime.io.BarrierBuffer.notifyCheckpoint(BarrierBuffer.java:428)
  13. 	at org.apache.flink.streaming.runtime.io.BarrierBuffer.processBarrier(BarrierBuffer.java:331)
  14. 	at org.apache.flink.streaming.runtime.io.BarrierBuffer.getNext(BarrierBuffer.java:204)
  15. 	at org.apache.flink.streaming.runtime.io.BarrierBuffer.getNextNonBlocked(BarrierBuffer.java:162)
  16. 	at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:248)
  17. 	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:113)
  18. 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:448)
  19. 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:758)
  20. 	at java.lang.Thread.run(Thread.java:834)
  21. Caused by: java.lang.NullPointerException
  22. 	at com.poizon.userportrait.handler.community.CommunityExposureEventHandler.doAttenuation(CommunityExposureEventHandler.java:209)
  23. 	at com.poizon.userportrait.handler.community.CommunityExposureEventHandler.doEvent(CommunityExposureEventHandler.java:113)
  24. 	at com.poizon.userportrait.sink.redis.community.UserPortraitCommunityRedisSink.doBatch(UserPortraitCommunityRedisSink.java:254)
  25. 	at com.poizon.userportrait.sink.redis.community.UserPortraitCommunityRedisSink.sync(UserPortraitCommunityRedisSink.java:347)
  26. 	at com.alibaba.blink.streaming.connector.custom.sink.CustomOutputFormat.sync(CustomOutputFormat.java:89)
  27. 	at com.alibaba.blink.streaming.connectors.common.output.TupleOutputFormatAdapterSink.snapshotState(TupleOutputFormatAdapterSink.java:88)
  28. 	at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:126)
  29. 	at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99)
  30. 	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:90)
  31. 	at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:400)
  32. 	... 17 more

 

  1. 2021-03-25 22:43:35,146 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  2. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
  3. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  4. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  5. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  6. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  7. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  8. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  9. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  10. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  11. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  12. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  13. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  14. 2021-03-25 22:43:35,174 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  15. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
  16. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  17. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  18. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  19. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  20. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  21. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  22. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  23. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  24. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  25. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  26. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  27. 2021-03-25 22:43:35,257 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  28. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  29. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  30. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  31. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  32. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  33. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  34. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  35. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  36. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  37. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  38. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  39. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  40. 2021-03-25 22:45:17,483 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  41. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
  42. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  43. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  44. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  45. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  46. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  47. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  48. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  49. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  50. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  51. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  52. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  53. 2021-03-25 22:45:17,493 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  54. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
  55. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  56. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  57. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  58. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  59. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  60. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  61. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  62. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  63. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  64. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  65. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  66. 2021-03-25 22:45:17,756 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  67. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  68. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  69. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  70. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  71. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  72. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  73. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  74. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  75. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  76. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  77. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  78. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  79. 2021-03-25 22:45:17,770 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  80. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
  81. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  82. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  83. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  84. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  85. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  86. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  87. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  88. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  89. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  90. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  91. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  92. 2021-03-25 22:45:18,197 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  93. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  94. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  95. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  96. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  97. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  98. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  99. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  100. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  101. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  102. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  103. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  104. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  105. 2021-03-25 22:45:18,339 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  106. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  107. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  108. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  109. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  110. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  111. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  112. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  113. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  114. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  115. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  116. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  117. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  118. 2021-03-25 22:45:20,494 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  119. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
  120. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  121. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  122. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  123. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  124. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  125. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  126. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  127. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  128. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  129. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  130. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  131. 2021-03-25 22:45:20,542 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  132. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
  133. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  134. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  135. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  136. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  137. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  138. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  139. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  140. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  141. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  142. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  143. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  144. 2021-03-25 22:45:20,672 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  145. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  146. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  147. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  148. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  149. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  150. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  151. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  152. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  153. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  154. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  155. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  156. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  157. 2021-03-25 22:45:20,786 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  158. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
  159. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  160. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  161. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  162. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  163. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  164. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  165. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  166. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  167. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  168. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  169. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  170. 2021-03-25 22:45:21,078 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  171. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  172. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  173. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  174. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  175. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  176. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  177. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  178. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  179. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  180. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  181. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  182. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  183. 2021-03-25 22:45:21,253 ERROR [flink-akka.actor.default-dispatcher-108] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  184. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  185. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  186. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  187. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  188. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  189. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  190. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  191. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  192. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  193. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  194. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  195. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  196. 2021-03-25 22:45:22,772 INFO  [flink-akka.actor.default-dispatcher-101] org.apache.flink.runtime.rest.handler.job.JobVertexDetailsHandler  - job vertex detail get log file from ResourceManager Api.
  197. 2021-03-25 22:45:22,800 INFO  [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.JobVertexDetailsHandler  - job vertex detail get log file from ResourceManager Api.
  198. 2021-03-25 22:45:24,923 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  199. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  200. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  201. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  202. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  203. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  204. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  205. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  206. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  207. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  208. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  209. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  210. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  211. 2021-03-25 22:45:24,926 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  212. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
  213. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  214. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  215. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  216. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  217. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  218. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  219. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  220. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  221. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  222. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  223. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  224. 2021-03-25 22:45:25,144 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  225. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
  226. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  227. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  228. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  229. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  230. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  231. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  232. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  233. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  234. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  235. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  236. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  237. 2021-03-25 22:45:25,168 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  238. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  239. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  240. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  241. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  242. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  243. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  244. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  245. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  246. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  247. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  248. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  249. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  250. 2021-03-25 22:45:25,193 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  251. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
  252. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  253. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  254. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  255. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  256. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  257. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  258. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  259. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  260. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  261. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  262. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  263. 2021-03-25 22:45:25,344 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
  264. org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
  265. 	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
  266. 	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
  267. 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
  268. 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
  269. 	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
  270. 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
  271. 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
  272. 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  273. 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  274. 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  275. 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

 

 

4、总结，一定要从jobmanager里查看相关日志信息。会有相关报错信息的

 

5、因为自己继承了自定义sink类CustomSinkBase。而该类中的sync方法，会在checkpoint的时候调用，如果里面有npe错误，则导致checkpoint失败的情况，所以在调用sync的方法还是要注意点，注意错误捕获。该方法主要是在一些保存使用的。

  1.  
  2. public abstract class CustomSinkBase implements Serializable {
  3.     protected Map<String, String> userParamsMap;
  4.     protected Set<String> primaryKeys;
  5.     protected List<String> headerFields;
  6.     protected RowTypeInfo rowTypeInfo;
  7.  
  8.     public CustomSinkBase() {
  9.     }
  10.  
  11.     public CustomSinkBase setUserParamsMap(Map<String, String> userParamsMap) {
  12.         this.userParamsMap = userParamsMap;
  13.         return this;
  14.     }
  15.  
  16.     public CustomSinkBase setPrimaryKeys(Set<String> primaryKeys) {
  17.         this.primaryKeys = primaryKeys;
  18.         return this;
  19.     }
  20.  
  21.     public CustomSinkBase setHeaderFields(List<String> headerFields) {
  22.         this.headerFields = headerFields;
  23.         return this;
  24.     }
  25.  
  26.     public CustomSinkBase setRowTypeInfo(RowTypeInfo rowTypeInfo) {
  27.         this.rowTypeInfo = rowTypeInfo;
  28.         return this;
  29.     }
  30.  
  31.     public abstract void open(int var1, int var2) throws IOException;
  32.  
  33.     public abstract void close() throws IOException;
  34.  
  35.     public abstract void writeAddRecord(Row var1) throws IOException;
  36.  
  37.     public abstract void writeDeleteRecord(Row var1) throws IOException;
  38.  
  39.     public abstract void sync() throws IOException;
  40.  
  41.     public abstract String getName();
  42. }

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/喵喵爱编程/article/detail/816206
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号