devbox
木道寻08
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

Hadoop3.3.1 HA配置多个namenode（5个Namenode）详细_hadoop多个namenode

作者：木道寻08 | 2024-07-20 07:14:12

赞

踩

hadoop多个namenode

前期准备

所有主机关闭防火墙。

所有主机安装JDK1.8，并配置环境变量。

所有主机设置静态IP地址，修改主机名称。

设置所有主机selinux=disabled。

安装好zookeeper集群。并启动。

下载hadoop3.3.1包

集群配置

机器现状

六台虚拟机a2-a7

1. 配置hadoop-env.sh

export JAVA_HOME=/apps/jdk 可以只配置这其他的我这是后边有问题加上的

export HADOOP_PID_DIR=/apps/hadoop/hadoop_data/tmp/pids

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/apps/hadoop/share/hadoop/mapreduce/*

2.  配置core-site.xml文件

  1. <!-- 指定hdfs的nameservice为ns1 -->
  2. <property>
  3.   <name>fs.defaultFS</name>
  4.   <value>hdfs://test</value>
  5. </property>
  6. <!-- 指定hadoop hdfs目录 -->
  7. <property>
  8.   <name>hadoop.tmp.dir</name>
  9.   <value>/data/hadoop/</value>
  10. </property>
  11. <property>
  12.   <name>hadoop.native.lib</name>
  13.   <value>false</value>
  14.   <description>Should native hadoop libraries, if present, be used.</description>
  15. </property>
  16. <!-- 指定zookeeper地址 -->
  17. <property>
  18.   <name>ha.zookeeper.quorum</name>
  19.   <value>a1:2181,a2:2181,a3:2181,a4:2181,a5:2181,a6:2181</value>
  20. </property>
  21. <!--可选的配置QJM日志目录-->
  22. <property>
  23.   <name>dfs.journalnode.edits.dir</name>
  24.   <value>/data/hadoop/journal/local/data</value>
  25. </property>

 3.  配置hdfs-site.xml文件

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <!--
  4.   Licensed under the Apache License, Version 2.0 (the "License");
  5.   you may not use this file except in compliance with the License.
  6.   You may obtain a copy of the License at
  8.     http://www.apache.org/licenses/LICENSE-2.0
  10.   Unless required by applicable law or agreed to in writing, software
  11.   distributed under the License is distributed on an "AS IS" BASIS,
  12.   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13.   See the License for the specific language governing permissions and
  14.   limitations under the License. See accompanying LICENSE file.
  15. -->
  16.  
  17. <!-- Put site-specific property overrides in this file. -->
  18.  
  19. <configuration>
  20. <!--指定hdfs的nameservice为test，需要和core-site.xml中的保持一致 -->
  21. <property>
  22.   <name>dfs.nameservices</name>
  23.   <value>test</value>
  24. </property>
  25. <property>
  26.   <name>dfs.ha.namenodes.test</name>
  27.   <value>nn1,nn2,nn3,nn4,nn5</value>
  28. </property>
  29. <!--配置每一个NameNode的rpc通信地址-->
  30. <property>
  31.   <name>dfs.namenode.rpc-address.test.nn1</name>
  32.   <value>a2:8020</value>
  33. </property>
  34. <property>
  35.   <name>dfs.namenode.rpc-address.test.nn2</name>
  36.   <value>a3:8020</value>
  37. </property>
  38. <property>
  39.   <name>dfs.namenode.rpc-address.test.nn3</name>
  40.   <value>a4:8020</value>
  41. </property>
  42. <property>
  43.   <name>dfs.namenode.rpc-address.test.nn4</name>
  44.   <value>a5:8020</value>
  45. </property>
  46. <property>
  47.   <name>dfs.namenode.rpc-address.test.nn5</name>
  48.   <value>a6:8020</value>
  49. </property>
  50. <!--配置每一个NameNode的web http地址-->
  51. <property>
  52.   <name>dfs.namenode.http-address.test.nn1</name>
  53.   <value>a2:9870</value>
  54. </property>
  55. <property>
  56.   <name>dfs.namenode.http-address.test.nn2</name>
  57.   <value>a3:9870</value>
  58. </property>
  59. <property>
  60.   <name>dfs.namenode.http-address.test.nn3</name>
  61.   <value>a4:9870</value>
  62. </property>
  63. <property>
  64.   <name>dfs.namenode.http-address.test.nn4</name>
  65.   <value>a5:9870</value>
  66. </property>
  67. <property>
  68.   <name>dfs.namenode.http-address.test.nn5</name>
  69.   <value>a6:9870</value>
  70. </property>
  71. <property>
  72.   <name>dfs.namenode.shared.edits.dir</name>
  73.   <value>qjournal://a2:8485;a3:8485;a4:8485/test</value>
  74. </property>
  75. <!--配置QJM日志的目录-->
  76. <property>
  77.   <name>dfs.journalnode.edits.dir</name>
  78.   <value>/data/hadoop/qjm/edits</value>
  79. </property>
  80. <!--配置为自动切换功能打开，需要在core-site.xml文件中配置ZK地址-->
  81. <property>
  82.   <name>dfs.ha.automatic-failover.enabled</name>
  83.   <value>true</value>
  84. </property>
  85. <property>
  86. <name>dfs.client.failover.proxy.provider.test</name>
  87.   <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
  88. </property>
  89.         <!--配置自动切换的方式-->
  90. <property>
  91.  <name>dfs.ha.fencing.methods</name>
  92.  <value>
  93.   sshfence
  94.   shell(/bin/true)
  95.  </value>
  96. </property>
  97.  
  98. <!--配置SSH key，注意根据不同的用户名修改目录-->
  99.  
  100. <property>
  101.   <name>dfs.ha.fencing.ssh.private-key-files</name>
  102.   <value>/root/.ssh/id_rsa</value>
  103. </property>
  104.  
  105. <property>
  106.   <name>dfs.permissions.enabled</name>
  107.   <value>false</value>
  108. </property>
  109.  
  110. <!-- 配置sshfence隔离机制超时时间 -->
  111. <property>
  112.   <name>dfs.ha.fencing.ssh.connect-timeout</name>
  113.   <value>30000</value>
  114. </property>
  115. <property>
  116.   <name>dfs.namenode.checkpoint.check.period</name>
  117.   <value>3600s</value>
  118. </property>
  119. </configuration>

 4.  配置mapreduce-site.xml文件

  1.     <property>
  2.             <name>mapreduce.framework.name</name>
  3.             <value>yarn</value>
  4.     </property>

 5.  配置yarn-site.xml文件

  1. <!--配置RM高可靠-->
  2.  
  3. <property>
  4.   <name>yarn.resourcemanager.ha.enabled</name>
  5.   <value>true</value>
  6. </property>
  7.  
  8. <property>
  9.   <name>yarn.resourcemanager.cluster-id</name>
  10.   <value>cluster1</value>
  11. </property>
  12.  
  13.     <!--配置Resourcemanager个数，hadoop3以后可以为3到5个-->
  14.  
  15. <property>
  16.   <name>yarn.resourcemanager.ha.rm-ids</name>
  17.   <value>rm1,rm2,rm3</value>
  18. </property>
  19.  
  20.     <!--以下配置每一个RM的地址-->
  21.  
  22. <property>
  23.   <name>yarn.resourcemanager.hostname.rm1</name>
  24.   <value>a2</value>
  25. </property>
  26.  
  27. <property>
  28.   <name>yarn.resourcemanager.hostname.rm2</name>
  29.   <value>a3</value>
  30. </property>
  31.  
  32. <property>
  33.   <name>yarn.resourcemanager.hostname.rm3</name>
  34.   <value>a4</value>
  35. </property>
  36.  
  37.     <!--配置每一个RM的http地址-->
  38.  
  39. <property>
  40.   <name>yarn.resourcemanager.webapp.address.rm1</name>
  41.   <value>a2:8088</value>
  42. </property>
  43.  
  44. <property>
  45.   <name>yarn.resourcemanager.webapp.address.rm2</name>
  46.   <value>a3:8088</value>
  47. </property>
  48.  
  49. <property>
  50.   <name>yarn.resourcemanager.webapp.address.rm3</name>
  51.   <value>a4:8088</value>
  52. </property>
  53.  
  54.     <!--配置zookeeper地址-->
  55.  
  56. <property>
  57.   <name>yarn.resourcemanager.zk-address</name>
  58.   <value>a2:2181,a3:2181,a4:2181</value>
  59. </property>
  60.  
  61. <property>
  62.   <name>yarn.nodemanager.aux-services</name>
  63.   <value>mapreduce_shuffle</value>
  64. </property>
  65.         <property>
  66.                 <name>yarn.application.classpath</name>
  67.                 <value>/apps/hadoop/etc/hadoop:/apps/hadoop/share/hadoop/common/lib/*:/apps/hadoop/share/hadoop/common/*:/apps/hadoop/share/hadoop/hdfs:/apps/hadoop/share/hadoop/hdfs/lib/*:/apps/hadoop/share/hadoop/hdfs/*:/apps/hadoop/share/hadoop/mapreduce/lib/*:/apps/hadoop/share/hadoop/mapreduce/*:/apps/hadoop/share/hadoop/yarn:/apps/hadoop/share/hadoop/yarn/lib/*:/apps/hadoop/share/hadoop/yarn/*</value>
  68.         </property>
  69. <property>
  70.   <name>yarn.nodemanager.resource.memory-mb</name>
  71.   <value>16384</value>
  72. </property>
  73. <property>
  74.   <name>yarn.nodemanager.resource.cpu-vcores</name>
  75.   <value>4</value>
  76. </property>
  77. <property>
  78.   <name>yarn.scheduler.maximum-allocation-vcores</name>
  79.   <value>4</value>
  80. </property>
  81.

 6.  配置workers文件

  1. a2
  2. a3
  3. a4
  4. a5
  5. a6
  6. a7

7.配置start-dfs.sh/stop-dfs.sh

在中间位置，找到一个空白的位置添加：

HDFS_NAMENODE_USER=root

HDFS_SECONDARYNAMENODE_USER=root

HDFS_DATANODE_USER=root

HDFS_DATANODE_SECURE_USER=root

#以下是高可靠配置的

HDFS_JOURNALNODE_USER=root

HDFS_ZKFC_USER=root

8.配置start-yarn.sh/stop-yarn.sh

在两个配置文件中间位置添加以下内容：

YARN_RESOURCEMANAGER_USER=root

YARN_NODEMANAGER_USER=root

9. 现在配置hadoop的环境变量

export HADOOP_HOME=/data/apps/hadoop-3.3.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

 10.拷贝文件其他主机

scp -r /data/apps/hadoop/  a3:/data/apps/

11.启动journalnode 在journalnode机器上执行

hadoop-daemon.sh start journalnode

12.格式化HDFS

hdfs namenode -format

出现successfully formatted为成功

现在将格式化后的hdfs目录，拷贝到a3,a4,a5,a6主机上的相同目录下：

 scp -r /data/hadoop/dfs/  a3:/data/hadoop/

内容是这样 的目录

 13.格式化zkfc

格式化前要停止

hadoop-daemon.sh stop journalnode 不停止会报错，如果不报错就不用在意

hdfs zkfc -formatZK

Successfully created hadoop-ha/cluster in ZK 出现这个就是成功

14.启动dfs yarn

start-dfs.sh  因为配置了环境变量所以可以直接启动

start-yarn.sh

start-all.sh    启动所有

stop-all.sh    停止全部

 验证

web  访问 http://ip:9870

 

 

也可以通过以下命令，检查NameNode和ResourceManager的状态

$ hdfs haadmin -getServiceState nn1

active

$ hdfs haadmin -getServiceState nn2

standby

$ yarn rmadmin -getServiceState rm1

active

$ yarn rmadmin -getServiceState rm2

standby

有可能出现的问题：

 这个问题现在还没有解决 可以手动切换namenode 到前两个namenode 

Currently, the standby NN will log StandbyExceptions at ERROR level any time a client tries to connect to it. So, if the second NN in an HA pair is active, the first NN will spew a lot of these errors in the log, as each client gets redirected to the proper NN. Instead, this should be at INFO level, and should probably be logged in a less "scary" manner (eg "Received READ request from client 1.2.3.4, but in Standby state. Redirecting client to other NameNode.")

有大神知道怎么解决可以留言谢谢

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/木道寻08/article/detail/855748
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号