hadoop--hadoop集群常用脚本_hadoop集群脚本

集群启动/停止方式总结

各个模块分开启动/停止(前提配置ssh)常用
1). 整体启动/停止HDFS：

start-dfs.sh/stop-dfs.sh
1

2).整体启动/停止YARN：

start-yarn.sh/stop-yarn.sh
1

2.各个服务组建注意启动/停止

1). 分别启动/停止HDFS组件：

hdfs --daemon start/stop namenode/datanode/secondarynamenode
1

2). 启动/停止YARN：

yarn --daemon start/stop resourcemanager/nodemanager
1

编写Hadoop集群常用脚本

1.Hadoop集群启停脚本(包含HDFS、YARN、Historyserver):myhadoop.sh;

在/opt/module/hadoop-3.2.2/bin目录下创建myhadoop.sh：

[xiaobai@hadoop102 bin]$ vim myhadoop.sh
1

在文件里输入如下内容：

case $1 in
"start")
        echo "=========启动hadoop集群==========="

        echo "---------启动hdfs----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/sbin/start-dfs.sh"
        echo "----------启动yarn----------"
        ssh hadoop103 "/opt/module/hadoop-3.2.2/sbin/start-yarn.sh"
        echo "----------启动historyserver----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/bin/mapred --daemon start historyserver"
;;
"stop")
echo "=========关闭hadoop集群==========="

        echo "----------关闭historyserver----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/bin/mapred --daemon stop historyserver"
        echo "----------关闭yarn------------"
        ssh hadoop103 "/opt/module/hadoop-3.2.2/sbin/stop-yarn.sh"
        echo "-----------关闭hdfs------------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/sbin/stop-dfs.sh"
;;
*)

        echo "Input Args Error..."
;;
esac
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

保存退出，赋予脚本权限：

[xiaobai@hadoop102 bin]$ chmod 777 myhadoop.sh 
1

测试脚本是否可正常使用：

[xiaobai@hadoop102 hadoop-3.2.2]$ myhadoop.sh start
=========启动hadoop集群===========
---------启动hdfs----------
Starting namenodes on [hadoop102]
Starting datanodes
Starting secondary namenodes [hadoop104]
----------启动yarn----------
Starting resourcemanager
Starting nodemanagers
----------启动historyserver----------
1
2
3
4
5
6
7
8
9
10

2.查看3x服务器Java进程脚本：jpsall

在/opt/module/hadoop-3.2.2/bin目录下创建jpsall：

[xiaobai@hadoop102 bin]$ vim jpsall
1

输入如下内容：

#!/bin/bash

for host in hadoop102 hadoop103 hadoop104
do
        echo ============ $host ===========
        ssh $host jps
done
1
2
3
4
5
6
7

保存退出，赋予脚本执行权限：

[xiaobai@hadoop102 bin]$ chmod +x jpsall
1

测试脚本是否可正常使用：

[xiaobai@hadoop102 hadoop-3.2.2]$ jpsall
============ hadoop102 ===========
2624 JobHistoryServer
2020 NameNode
2132 DataNode
2694 Jps
2460 NodeManager
============ hadoop103 ===========
2533 Jps
1878 DataNode
2054 ResourceManager
2171 NodeManager
============ hadoop104 ===========
2070 NodeManager
1883 DataNode
1951 SecondaryNameNode
2191 Jps
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

3.分发/home/xiaobai/bin目录，保证自定义脚本在三台机器上都可以使用：

[xiaobai@hadoop102 hadoop-3.2.2]$ xsync /home/xiaobai/bin/
1