大数据组件-Flink 常用命令_flink常用命令

使用Maven将自己的代码编译打包

打好的包一般放在工程目录的target子文件夹下

mvn clean package

flink run --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.0-SNAPSHOT.jar

sudo -u hdfs 
bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.0-SNAPSHOT.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.0.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.1.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.2.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.xxx.data.wordcount.WordCountKafkaInStdOut /data/cdh/test/flink-kafka-1.3.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.atguigu.hotitems_analysis.KafkaProducerUtil /data/cdh/test/HotItemsAnalysis-1.0.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 --class com.atguigu.hotitems_analysis.HotItems /data/cdh/test/HotItemsAnalysis-2.0.jar

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 /data/cdh/test/HotItemsAnalysis-3.0.jar --class com.atguigu.hotitems_analysis.HotItems
    
bin/flink run --class com.atguigu.hotitems_analysis.HotItems /data/cdh/test/HotItemsAnalysis-3.0.jar

bac56b51f24ae63c4bf484cb0868b09a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

启动jar包中指定的类

java -cp /data/cdh/test/HotItemsAnalysis-4.0.jar com.atguigu.hotitems_analysis.KafkaProducerUtil

bin/flink run --class com.atguigu.hotitems_analysis.HotItems /data/cdh/test/HotItemsAnalysis-4.0.jar
1
2
3

flink内置案例

第一种方式，永久集群

bin/flink run ./examples/batch/WordCount.jar
bin/flink run ./examples/batch/WordCount.jar
1
2

第二种方式 临时集群

export HADOOP_CLASSPATH=`hadoop classpath`

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 ./examples/batch/WordCount.jar

sudo -u hdfs bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 ./examples/batch/WordCount.jar

sudo -u hdfs hadoop fs -setrep -R 2 /

export HADOOP_CONF_DIR=/etc/hadoop/conf

Permission denied: user=cdh, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x 问题解决

sudo -u hdfs hadoop fs -mkdir /user/cdh

sudo -u hdfs  hdfs dfs -chown cdh:supergroup   /user/cdh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15