当前位置:   article > 正文

hadoop--hadoop集群常用脚本_hadoop集群脚本

hadoop集群脚本

集群启动/停止方式总结

各个模块分开启动/停止(前提配置ssh)常用
1). 整体启动/停止HDFS

start-dfs.sh/stop-dfs.sh
  • 1

2).整体启动/停止YARN:

start-yarn.sh/stop-yarn.sh
  • 1

2.各个服务组建注意启动/停止

1). 分别启动/停止HDFS组件:

hdfs --daemon start/stop namenode/datanode/secondarynamenode
  • 1

2). 启动/停止YARN:

yarn --daemon start/stop resourcemanager/nodemanager
  • 1

编写Hadoop集群常用脚本

1.Hadoop集群启停脚本(包含HDFS、YARN、Historyserver):myhadoop.sh;

在/opt/module/hadoop-3.2.2/bin目录下创建myhadoop.sh:

[xiaobai@hadoop102 bin]$ vim myhadoop.sh
  • 1

在文件里输入如下内容:

case $1 in
"start")
        echo "=========启动hadoop集群==========="

        echo "---------启动hdfs----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/sbin/start-dfs.sh"
        echo "----------启动yarn----------"
        ssh hadoop103 "/opt/module/hadoop-3.2.2/sbin/start-yarn.sh"
        echo "----------启动historyserver----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/bin/mapred --daemon start historyserver"
;;
"stop")
echo "=========关闭hadoop集群==========="

        echo "----------关闭historyserver----------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/bin/mapred --daemon stop historyserver"
        echo "----------关闭yarn------------"
        ssh hadoop103 "/opt/module/hadoop-3.2.2/sbin/stop-yarn.sh"
        echo "-----------关闭hdfs------------"
        ssh hadoop102 "/opt/module/hadoop-3.2.2/sbin/stop-dfs.sh"
;;
*)

        echo "Input Args Error..."
;;
esac
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26

保存退出,赋予脚本权限:

[xiaobai@hadoop102 bin]$ chmod 777 myhadoop.sh 
  • 1

测试脚本是否可正常使用:

[xiaobai@hadoop102 hadoop-3.2.2]$ myhadoop.sh start
=========启动hadoop集群===========
---------启动hdfs----------
Starting namenodes on [hadoop102]
Starting datanodes
Starting secondary namenodes [hadoop104]
----------启动yarn----------
Starting resourcemanager
Starting nodemanagers
----------启动historyserver----------
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10

2.查看3x服务器Java进程脚本:jpsall

在/opt/module/hadoop-3.2.2/bin目录下创建jpsall:

[xiaobai@hadoop102 bin]$ vim jpsall
  • 1

输入如下内容:

#!/bin/bash

for host in hadoop102 hadoop103 hadoop104
do
        echo ============ $host ===========
        ssh $host jps
done
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

保存退出,赋予脚本执行权限:

[xiaobai@hadoop102 bin]$ chmod +x jpsall
  • 1

测试脚本是否可正常使用:

[xiaobai@hadoop102 hadoop-3.2.2]$ jpsall
============ hadoop102 ===========
2624 JobHistoryServer
2020 NameNode
2132 DataNode
2694 Jps
2460 NodeManager
============ hadoop103 ===========
2533 Jps
1878 DataNode
2054 ResourceManager
2171 NodeManager
============ hadoop104 ===========
2070 NodeManager
1883 DataNode
1951 SecondaryNameNode
2191 Jps
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17

3.分发/home/xiaobai/bin目录,保证自定义脚本在三台机器上都可以使用:

[xiaobai@hadoop102 hadoop-3.2.2]$ xsync /home/xiaobai/bin/
  • 1
声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/运维做开发/article/detail/932033
推荐阅读
相关标签
  

闽ICP备14008679号