Spark 远程调试_spark远程调试

调试方式进入 spark-shell

spark-shell \
   --conf spark.driver.extraJavaOptions=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000
1
2

调试方式进入 spark-sql

spark-sql \
   --conf spark.driver.extraJavaOptions=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000
1
2

3. 调试方式执行指定程序

spark-submit \
  --name SparkByExamples.com \
  --class org.sparkbyexamples.SparkWordCountExample \
  --conf spark.driver.extraJavaOptions=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000
  spark-by-examples.jar
1
2
3
4
5