当前位置:   article > 正文

Kafka+Spark Streaming+Redis Spark streaming实时读取kafka中数据完成wordcount并写入redis中_sparkstreaming输出数据到redis pom文件怎么写

sparkstreaming输出数据到redis pom文件怎么写

集群或虚拟机提前安装好spark、kafka和redis

windows环境安装好scala

用idea创建maven工程

spark版本2.2.0

pom文件内容如下:

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <project xmlns="http://maven.apache.org/POM/4.0.0"
  3. xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  4. xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  5. <modelVersion>4.0.0</modelVersion>
  6. <groupId>spark</groupId>
  7. <artifactId>sparklearning</artifactId>
  8. <version>1.0-SNAPSHOT</version>
  9. <properties>
  10. <maven.compiler.source>1.7</maven.compiler.source>
  11. <maven.compiler.target>1.7</maven.compiler.target>
  12. <encoding>UTF-8</encoding>
  13. <scala.version>2.11.7</scala.version>
  14. <spark.version>2.2.0</spark.version>
  15. <!--<hadoop.version>2.6.0</hadoop.version>-->
  16. </properties>
  17. <dependencies>
  18. <dependency>
  19. <groupId>org.apache.spark</groupId>
  20. <artifactId>spark-core_2.11</artifactId>
  21. <version>${spark.version}</version>
  22. <!--<scope>provided</scope>-->
  23. </dependency>
  24. <dependency>
  25. <groupId>org.apache.spark</groupId>
  26. <artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
  27. <version>2.2.0</version>
  28. </dependency>
  29. <dependency>
  30. <groupId>org.apache.spark</groupId>
  31. <artifactId>spark-streaming_2.11</artifactId>
  32. <version>2.2.0</version>
  33. </dependency>
  34. <dependency>
  35. <groupId>org.apache.spark</groupId>
  36. <artifactId>spark-sql_2.11</artifactId>
  37. <version>2.2.0</version>
声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/IT小白/article/detail/1021846
推荐阅读
相关标签