热门标签
热门文章
- 1Vivado ECO实例教程一 增加LUT（GUI操作)_vivado lut使用
- 2vue实现文字内容无缝向上滚动，鼠标移入停止滚动，鼠标移开继续滚动_js利用transform实现滚动并且鼠标移入停止滚动
- 3IDEA+git+码云托管项目_gitee项目落下来
- 4小爱音箱怎么装app_Redmi小爱触屏音箱：8英寸大屏的智能家居助理
- 5调用百度图像识别api处理网络图片（文字识别）_c#图文识别之百度api调用
- 6什么是shuffle？shuffle的原理及过程
- 7多模态——CLIP:Contrastive Language-Image Pre-training解读
- 8探索高效网络性能测试工具：HTTP Benchmark Netty
- 9AndroidQ高通平台 集成openssh-sshd_android通过ssh访问qnx
- 10【SSA-LSTM】基于麻雀算法优化LSTM 模型预测研究（Matlab代码实现）_优化算法对lstm进行优化matlab代码
当前位置: article > 正文
部署Ambari_ambari部署
作者：小蓝xlanll | 2024-06-13 10:56:08
赞
踩
ambari部署
部署Ambari
1.修改网络配置（所有机器）
进入网卡配置
# vi /etc/sysconfig/network-scripts/ifcfg-ens33
- 1
注意：ifcfg-ens33是自己的网卡名称，使用下面命令可以查看自己的网卡名称，如果不一样，修改为自己的即可
# cd /etc/sysconfig/network-scripts
- 1
BOOTPROTO 设置为 static，ONBOOT 设置为 yes
加入 IPADDR、NETMASK、GATEWAY、DNS1等，（如果存在可以忽略）
注：三台机器操作相同，只修改特定信息即可，所以此处只粘贴了一张图
2.重启网络（所有机器）
# systemctl restart network
- 1
3.修改/etc/hosts文件（所有机器）
# vi etc/hosts
- 1
将 ip 和主机名称对应起来，主要作用是通过名称来查找相应的服务器
4.修改文件打开限制（所有机器）
# vi /etc/security/limits.conf
- 1
打开文件后，在文件末尾添加以下内容：
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
* hard nproc 65536
- 1
- 2
- 3
- 4
- 5
5.关闭防火墙（所有机器）
systemctl stop firewalld（临时关闭）
systemctl disable firewalld（禁用）
- 1
- 2
6.关闭 selinux（所有机器）
vi /etc/selinux/config
- 1
7.设置免密登录(只有swarm01操作）
（1）生成密钥对
# ssh-keygen -t rsa ## 一路回车即可
# cd ~/.ssh ## 进入.ssh 目录，如果目录不存在则创建
# cat id_rsa.pub >> authorized_keys ## 将公钥导入至 authorized_keys
# chmod 700 ~/.ssh ## 修改文件权限
# chmod 600 authorized_keys ## 修改文件权限
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
（2）在swarm01上执行以下命令，使主节点能免密访问其他俩从节点（把swarm01作为主节点，swarm02和swarm03作为从节点）：
# cat ~/.ssh/id_rsa.pub | ssh root@swarm02 'cat - >> ~/.ssh/authorized_keys'
# cat ~/.ssh/id_rsa.pub | ssh root@swarm03 'cat - >> ~/.ssh/authorized_keys'
- 1
- 2
若出现以下错误提示，则在swarm02和swarm03的/root目录下创建.ssh文件夹，再执行上面命令
8.安装jdk（所有机器）
百度网盘链接：https://pan.baidu.com/s/1A7_wBwI_ZmVLVq6WLgv5Pg
提取码：6wv6
里面有 jdk-8u151-linux-x64.tar.gz 和 mysql-connector-java.jar 文件
将下载的压缩包上传到 java 文件夹内
# mkdir /usr/java
- 1
解压压缩包
# tar -zxvf jdk-8u151-linux-x64.tar.gz
- 1
配置 jdk 环境变量，编辑/etc/profile,文末插入以下内容
# vi /etc/profile
- 1
# set java
export JAVA_HOME=/usr/java/jdk1.8.0_151
export PATH=$JAVA_HOME/bin:$PATH
- 1
- 2
- 3
使环境变量生效
# source /etc/profile
- 1
安装验证
# java -version
- 1
9.安装ntp服务
（1）检查是否安装ntp服务（所有机器）
# rpm –qa|grep ntp
- 1
（2）安装ntp服务（所有机器）
# yum install ntp -y
- 1
（3）在集群内所有节点检查是否设置时钟同步开机自启（所有机器）
# systemctl is-enabled ntpd
- 1
（4）若未设置（返回的状态不是 enabled），则设置开机自启（所有机器）
# systemctl enable ntpd
- 1
（5）配置 ntp server（其中一个服务器，这里用swarm01）
# vi /etc/ntp.conf
- 1
重启ntp服务（swarm01）
# systemctl restart ntpd.service
- 1
（6）配置 ntp client（除去上一步所用机器之外的机器，这里是swarm02和swarm03）
# vi /etc/ntp.conf
- 1
注：swarm02和swarm03操作一样，这里只粘贴了一张图
重启ntp服务（swarm02和swarm02）
# systemctl restart ntpd.service
- 1
10.安装 mysql（主节点swarm01）
先安装wget服务
# yum install wget
- 1
下载 mysql5.7 的 rpm 包
# wget https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
- 1
安装上一步下载的 rpm 文件，安装成功后，会在 /etc/yum.repos.d/ 目录下会增加两个文件
# yum -y install mysql57-community-release-el7-11.noarch.rpm
- 1
安装MySQL
# yum install mysql-community-server
- 1
更新密钥（如果出错，多执行几次就好了）
# rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
- 1
再次安装MySQL
# yum install mysql-community-server
- 1
登录MySQL并修改初始密码
# 启动 mysql 服务
service mysqld start
# 查看 root 密码
grep "password" /var/log/mysqld.log
# 登陆 mysql
mysql -u root -p
Enter password:
# 刚安装好的 mysql，如果设置的密码过于简单，会提示报错。
# 可以执行以下命令，设置简单密码
set global validate_password_policy=0;
set global validate_password_length=4;
# 立即修改密码，执行其他操作会报错：
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('root');
# 我们创建密码为 root
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
新增 ambari 用户并增加权限
mysql -uroot -proot
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambari';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari'; # 这个用户，在后续安装 ambari 的时候会
用到
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES; # 刷新权限
- 1
- 2
- 3
- 4
- 5
- 6
- 7
使用 ambari 用户登陆并创建数据库
mysql -uambari -pambari
CREATE DATABASE ambari character set utf8 collate utf8_general_ci;
exit;
- 1
- 2
- 3
11.搭建本地 yum 源（全部机器）
先安装 httpd 服务
# 安装 httpd
yum -y install httpd.x86_64
systemctl enable httpd.service
systemctl start httpd.service
- 1
- 2
- 3
- 4
将以下四个压缩包上传到/var/www/html目录下
（由于这四个包过大，无法上传至百度网盘，所以这里无法提供这四个包）
解压这四个压缩包，解压到/var/www/html路径下
tar -zxvf /var/www/html/HDP-3.1.4.0-centos7-rpm.tar.gz -C /var/www/html
tar -zxvf /var/www/html/HDP-GPL-3.1.4.0-centos7-gpl.tar.gz -C /var/www/html
tar -zvxf /var/www/html/HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html
tar -zxvf /var/www/html/ambari-2.7.4.0-centos7.tar.gz -C /var/www/html
- 1
- 2
- 3
- 4
切换到 cd /etc/yum.repos.d/下，看是否存在 ambari.repo 和 hdp.repo. 若不存在则创建这两个文件，内容如下。
注意：所有机器都需要操作，操作相同，只需要改对应IP既可，下面只演示一台操作过程截图
ambari.repo
#VERSION_NUMBER=2.7.4.0-118
[ambari-2.7.4.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.4.0
baseurl=http://192.168.15.128/ambari/centos7/2.7.4.0-118/
gpgcheck=0
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
主要将 baseurl 换成你压缩包解压后所在的目录，hdp.repo 配置也是这样
hdp.repo
#VERSION_NUMBER=3.1.4.0-315 [HDP-3.1.4.0] name=HDP Version - HDP-3.1.4.0 baseurl=http://192.168.15.128/HDP/centos7/3.1.4.0-315/ gpgcheck=0 gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://192.168.15.128/HDP-UTILS/centos7/1.1.0.22/ gpgcheck=0 gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
12.在主节点上也就是安装 mysql 的机器上安装 ambari-server（这里是swarm01）
安装
yum -y install ambari-server
- 1
将 mysql-connector-java.jar 拷贝到/usr/share/java 目录下（主节点，这里是swarm01）
修改配置文件
echo server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar >> /etc/ambari-server/conf/ambari.properties
- 1
安装 ambari-server
[root@swarm01 /]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/java/jdk1.8.0_151 Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 3 Hostname (localhost): 192.168.15.128 Port (3306): Database name (ambari): Username (ambari): Enter Database Password (bigdata): Configuring ambari database... Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.4.0.118.jar .... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully.
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
将 mysql-connector-java.jar 拷贝到指定目录
将 mysql-connector-java.jar 驱动包拷贝到指定目录，这样的话，在安装 hive 等需要 mysql 数据库的服务的
时候，就有 jdbc 连接驱动了，就不会报错了
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/ja va/mysql-connector-java.jar
- 1
- 2
初始化数据库
mysql -uambari -pambari
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
- 1
- 2
- 3
启动 ambari-server
ambari-server start
- 1
解决办法如下：重新进行ambari-server setup（用root账户登录），然后进入数据库修改权限
[root@swarm01 /]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n input not recognized, please try again: Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... Do you want to change Oracle JDK [y/n] (n)? n Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/g pl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO pa ckages [y/n] (n)? n Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... =========================================================================== === Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB =========================================================================== === Enter choice (3): 3 Hostname (192.168.15.128): Port (3306): Database name (ambari): Username (ambari): root Enter Database Password (bigdata): Configuring ambari database... Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL dire ctly from the database shell to create the schema: /var/lib/ambari-server/r esources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ..... Ambari repo file contains latest json url http://public-repo-1.hortonworks. com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. [root@swarm01 /]# mysql -uroot -proot mysql: [Warning] Using a password on the command line interface can be inse cure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1086 Server version: 5.7.37 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input stateme nt. mysql> set global validate_password_policy=0; Query OK, 0 rows affected (0.03 sec) mysql> set global validate_password_length=1; Query OK, 0 rows affected (0.00 sec) mysql> grant all privileges on *.* to root@'swarm01' identified by 'bigdata '; Query OK, 0 rows affected, 1 warning (0.07 sec) mysql> exit Bye
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
再次启动 ambari-server
ambari-server start
- 1
登陆浏览器访问: http://192.168.15.128:8080/，利用界面部署集群
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小蓝xlanll/article/detail/712199
推荐阅读
相关标签
