Redis报错-java.net.SocketException: Broken pipe (Write failed)

作者：盐析白兔 | 2024-02-28 16:18:34

踩

java.net.socketexception: broken pipe (write failed)

异常原因：

在SparkStreaming中使用Redis进行高并发写，发现有时候会出现以下异常：

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
   at redis.clients.jedis.Connection.flush(Connection.java:303)
   at redis.clients.jedis.Connection.getMany(Connection.java:317)
   at redis.clients.jedis.Pipeline.sync(Pipeline.java:98)
   at redis.clients.jedis.Pipeline.clear(Pipeline.java:84)
   at redis.clients.jedis.Pipeline.close(Pipeline.java:157)
   at com.amt.spark.streaming.UserBehaviorStreaming$1$12$1.call(UserBehaviorStreaming.java:353)
   at com.amt.spark.streaming.UserBehaviorStreaming$1$12$1.call(UserBehaviorStreaming.java:331)
   at org.apache.spark.api.java.JavaRDDLike

a n o n f u n $ f o r e a c h P a r t i t i o n $ 1. a p p l y (J a v a R D D L i k e . s c a l a : 225) a t o r g . a p a c h e . s p a r k . a p i . j a v a . J a v a R D D L i k e

$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225) at org.apache.spark.api.java.JavaRDDLike$ anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225)
at org.apache.spark.rdd.RDD

a n o n f u n $ f o r e a c h P a r t i t i o n $ 1

$anonfun$foreachPartition$1$ anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD

a n o n f u n $ f o r e a c h P a r t i t i o n $ 1

$anonfun$foreachPartition$1$ anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext

a n o n f u n $ r u n J o b $ 5. a p p l y (S p a r k C o n t e x t . s c a l a : 1858) a t o r g . a p a c h e . s p a r k . S p a r k C o n t e x t

$anonfun$runJob$5.apply(SparkContext.scala:1858)    at org.apache.spark.SparkContext$ anonfun$runJob$5.apply(SparkContext.scala:1858)
   at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
   at org.apache.spark.scheduler.Task.run(Task.scala:89)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
   at java.net.SocketOutputStream.socketWrite0(Native Method)
   at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
   at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
   at redis.clients.jedis.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:52)
   at redis.clients.jedis.util.RedisOutputStream.flush(RedisOutputStream.java:133)
   at redis.clients.jedis.Connection.flush(Connection.java:300)

解决过程：

百度谷歌尝试了各种方法，发现一点用都没有，异常原因给我引导到错误的排查方式去了。

解决方法：

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/盐析白兔/article/detail/160103