基于centos的hadoop3.1.3伪分布式安装脚本_cent0s7-hadoop3.1.3 伪分布式模式安装

使用说明：
1.需要创建一个用户（用户名按个人喜好）
2.赋予用户root的权限
3.hadoop的环境配置是当前用户的环境，不是全局
4.建议在window下载hadoop3.1.3的包传入linux，官网是国外网站，wget下载很慢，能等的可以把wget下载的命令解除注释，删除#即可；
5.使用脚本时需要确定当前目录下是否有安装包，脚本的wget默认注释！！因为很慢
脚本使用前置：
1.创建hadoop72用户，并修改hadoop72用户的密码
[root@hadoop100 ~]# useradd hadoop72
[root@hadoop100 ~]# passwd hadoop72
2.配置atguigu用户具有root权限，方便后期加sudo执行root权限的命令
[root@hadoop100 ~]# vim /etc/sudoers
修改/etc/sudoers文件，在%wheel这行下面添加一行，如下所示：

## Allow root to run any commands anywhere
root    ALL=(ALL)     ALL

## Allows people in group wheel to run all commands
%wheel  ALL=(ALL)       ALL
hadoop72   ALL=(ALL)     NOPASSWD:ALL
1
2
3
4
5
6

切换用户：su hadoop72
ps:以考虑到用户名习惯不一样，所以下面的脚本会自动获取当前用户名，无需手动更改；下面的脚本配置只有hdfs，yarn是没有配置，后续加上

#!/bin/bash
user=$(env | grep USER | cut -d "=" -f 2)
if [ "$user" == "root" ]
then
echo "当前用户是root，请更换用户"
exit
else
if(test -e /opt/moudle)
then
echo "----------------创建文件夹moudle---------------"
echo "/opt/moudle 文件夹存在"
chown $user:$user /opt/module
else
echo "----------------创建文件夹moudle---------------"
sudo mkdir /opt/moudle
sudo chown $user:$user /opt/moudle
echo "/opt/moudle 文件夹创建成功"
fi
if(test -e /opt/soft)
then
echo "----------------创建文件夹soft---------------"
echo "/opt/soft 文件夹存在"
else
echo "----------------创建文件夹soft---------------"
sudo mkdir /opt/soft
sudo chown $user:$user /opt/soft
echo "/opt/soft 文件夹创建成功"
fi
fi
#国外网站下载速度很慢，建议使用window下载传入linux
#wget http://archive.apache.org/dist/hadoop/core/hadoop-3.1.3/hadoop-3.1.3.tar.gz
echo "--------------------------解压----------------------------------"
tar -zxvf hadoop-3.1.3.tar.gz -C /opt/moudle
echo "-------------------------配置环境变量---------------------------"
echo "export HADOOP_HOME=/opt/moudle/hadoop-3.1.3" >> /home/hadoop72/.bashrc
echo "export PATH=\$PATH:\$HADOOP_HOME/sbin:\$HADOOP_HOME/bin" >> /home/hadoop72/.bashrc
cd ~
source .bashrc
echo "------------------hadoop版本-------------------------------------------"
hadoop version
echo "------------------配置ssh本地免密--------------------------------------"
cd ~
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys
echo "------------------配置hadoop伪分布式------------------"
echo "export JAVA_HOME=${JAVA_HOME}" >> /opt/moudle/hadoop-3.1.3/etc/hadoop/hadoop-env.sh
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<?xml-stylesheet type=\"text/xsl\" href=\"configuration.xsl\"?>
<!--Licensed under the Apache License, Version 2.0 (the \"License\");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>" > /opt/moudle/hadoop-3.1.3/etc/hadoop/core-site.xml
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<?xml-stylesheet type=\"text/xsl\" href=\"configuration.xsl\"?>
<!--Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>" > /opt/moudle/hadoop-3.1.3/etc/hadoop/hdfs-site.xml
cd /opt/moudle/hadoop-3.1.3
bin/hdfs namenode -format
#开启hdfs命令
#sbin/start-dfs.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

1

1

1

1