Hadoop伪分布式集群搭建

1.放入安装包

2.解压

tar -zxvf hadoop-2.7.3.tar.gz -C /usr/local/

 

3.导入JDK安装包

4.解压JDK安装包

tar -zxvf jdk-8u241-linux-x64.tar.gz -C /usr/local/

5.修改配置文件的路径

 cd /usr/local/hadoop-2.7.3/etc/hadoop

 4.修改hadoop-env.sh的文件

vi hadoop-env.sh

添加JAVA_HOME路径

 /usr/local/jdk1.8.0_241

5.修改core-site.xml

vi core-site.xml

<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop01:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-2.7.3/tmp</value>
</property>

 6.修改hdfs-site.xml

vi hdfs-site.xml

<property>
<name>dfs.namenode.name.dir</name>
<value>/usr/local/hadoop-2.7.3/data/name</value>
</property>
 
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/hadoop-2.7.3/data/data</value>
</property>
 
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
 
<property>
<name>dfs.secondary.http.address</name>
<value>hadoop01:50090</value>
</property>

7. 修改mapred-site.xml

mv mapred-site.xml.template mapred-site.xml
 
vi mapred-site.xml

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

8. 修改yarn-site.xml 

vi yarn-site.xml

<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop01</value>
</property>
 
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

 9.修改slaves

vi slaves

hadoop01 

10. 修改环境变量

vi /etc/profile

export JAVA_HOME=/usr/local/jdk1.8.0_241
export HADOOP_HOME=/usr/local/hadoop-2.7.3
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

 

 11.环境生效

source /etc/profile

12.配置免密登录

ssh-keygen

 13.配置映射

vi /etc/hosts

 192.168.70.111 hadoop01

14. 将公钥复制到远程机器中

 ssh-copy-id hadoop01

 15.初始化 (就一次)

hadoop namenode -format

 16.启动hadoop

start-all.sh

 17.查看启动状态

jps

18.查看防火墙状态

systemctl status firewalld

关了以后

 19.关闭并禁用防火墙

systemctl stop firewalld
systemctl disable firewalld

 

 18.启动成功后，可通过以下方式访问管理页面

HADOOP： 
http://node1 的 IP:8088/cluster 
HDFS： 
http://node1 的 IP:50070/dfshealth.html