SparkException: Dynamic allocation of executors requires the external shuffle service解决办法

作者：AllinToyou | 2024-04-23 15:30:28

踩

dynamic allocation of executors requires the external shuffle service. you m

详细报错： 
ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Dynamic allocation of executors requires the external shuffle service. You may enable this through spark.shuffle.service.enabled.
...
ERROR ApplicationMaster: User class threw exception: org.apache.spark.SparkException: Dynamic allocation of executors requires the external shuffle service. You may enable this through spark.shuffle.service.enabled.
org.apache.spark.SparkException: Dynamic allocation of executors requires the external shuffle service. You may enable this through spark.shuffle.service.enabled.

将spark.shuffle.service.enabled 设置为true 即可

spark.shuffle.service.enabled=true

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/AllinToyou/article/detail/474633