当前位置:   article > 正文

spark-submit提交任务报错:java.sql.SQLException:No suitable driver_spark no suitable driver

spark no suitable driver

提交spark任务时一直报错:No suitable driver
提交任务命令如下

  1. spark-submit \
  2. --master yarn-client \
  3. --num-executors 1 \
  4. --executor-cores 2 \
  5. --executor-memory 4G \
  6. --conf spark.sql.shuffle.partitions=4 \
  7. --class com.dy.ads.ads_rk_ccxx_population_property ../../script/smart-city-1.0.jar

每次执行这个命令都会报错,经过查询资料,尝试了加入--jars /opt/sparkJob/mysql-connector-java-5.1.47.jar也不行,后来又看到相关资料,还需要加入--conf spark.executor.extraClassPath=/opt/sparkJob/mysql-connector-java-5.1.47.jar才可以
完整命令如下
 

  1. spark-submit \
  2. --jars ../../jars/mysql-connector-java-5.1.37.jar \
  3. --driver-class-path ../../jars/mysql-connector-java-5.1.37.jar \
  4. --conf spark.executor.extraClassPath=/opt/sparkJob/mysql-connector-java-5.1.37.jar \
  5. --master yarn-client \
  6. --num-executors 1 \
  7. --executor-cores 2 \
  8. --executor-memory 4G \
  9. --conf spark.sql.shuffle.partitions=4 \
  10. --class com.dy.ads.ads_rk_ccxx_population_property ../../script/smart-city-1.0.jar

这三个必须都加上才可以,即--jars--driver-class-path--conf

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/2023面试高手/article/detail/593396
推荐阅读
相关标签
  

闽ICP备14008679号