【我和openGauss的故事】天龙八部之Centos/RHEL 7 安装部署openGauss 5.0 企业版 一主二备一级联操作指南
前言： 写此文章的目的主要是感觉 openGauss 官网对企业版集群安装部署描述有些冗杂， 对于初次 接触 openGauss 的朋友 可能会有点复杂 ，所以希望能编写一篇 5.0 的安装部署文档，希望对 openGuass 初次接触的朋友也能照着这篇文章一次部署起来，如果在部署中对此文档有异议的地方，请 联系我，我好进一步完善修改。
一、安装需求
集群环境各服务器应具有相同体系架构
-
64bit 和 32bit 不能同一集群
-
ARM 和 x86 两类系统不能同一集群
1.1 硬件环境要求
集群环境openGauss各服务器应满足以下最低硬件需求，生产环境应根据业务需求适时调整硬件配置。
| 项目名称 | 配置描述 | 备注信息 |
|---|---|---|
| 服务器数量 | 4(台) | |
| 内存 | >=32(GB) | 性能及商业部署建议单机不低于128(GB) |
| CPU | >= 1 * 8(核)，2.0(GHz) | 性能及商业部署建议单机不低于1*16(核)，2.0(GHz) 支持超线程和非超线程两种模式，建议选择相同模式 |
| 硬盘 | > 1(GB) 存放openGauss应用程序 > 300(MB) 存放元数据 > 70(%) 存储空间存放数据库数据 | 系统盘建议配置RAID1 数据盘建议配置RAID5(规划4组RAID5数据盘安装openGauss) Disk Cache Policy建议设置Disabled |
| 网络 | >= 300(兆) 以太网 | 建议设置双网卡冗余bond |
1.2 操作系统要求
1.2.1 软件环境要求
| 软件类型 | 配置描述 | 备注信息 |
|---|---|---|
| 操作系统 | x86操作系统 CentOS 7.6及以上 | 生产建议选择CentOS 7系列 |
| inode个数 | 剩余inode个数>15(亿) | |
| 工具 | bzip2 | |
| Python | Python 3.6.X | python需要通过–enable-shared方式编译 |
1.2.2 软件依赖要求
| 所需软件 | 建议版本 | 备注信息 |
|---|---|---|
| libaio-devel | 建议版本：0.3.109-13 | |
| flex | 要求版本：2.5.31 以上 | |
| bison | 建议版本：2.7-4 | |
| ncurses-devel | 建议版本：5.9-13.20130511 | |
| glibc-devel | 建议版本：2.17-111 | |
| patch | 建议版本：2.7.1-10 | |
| redhat-lsb-core | 建议版本：4.1 | |
| readline-devel | 建议版本：7.0-13 | |
二、集群规划
2.1 主机名称规划
| 主机名称 | 描述说明 |
|---|---|
| xsky-node1 | 主节点服务器名称 |
| xsky-node2 | 备节点一服务器名称 |
| xsky-node3 | 备节点二服务器名称 |
| xsky-node4 | 级联节点服务器名称 |
2.2 主机地址规划
| IP地址 | 描述说明 |
|---|---|
| 10.110.7.39 | 主节点IP地址 |
| 10.110.7.40 | 备节点一IP地址 |
| 10.110.7.41 | 备节点二IP地址 |
| 10.110.7.42 | 级联节点IP地址 |
2.3 端口号规划
| 端口号 | 参数名称 | 描述说明 |
|---|---|---|
| 15300 | cmServerPortBase | 主CM Server端口号 |
| 15300 | cmServerPortStandby | 备CM Server端口号 |
| 26000 | dataPortBase | 数据库节点的基础端口号 |
2.4 用户及组规划
| 项目名称 | 名称 | 所属类型 | 规划建议 |
|---|---|---|---|
| 用户名 | omm | 操作系统 | 建议集群各节点密码及ID相同 |
| 组名 | dbgrp | 操作系统 | 建议集群各节点组ID相同 |
2.5 软件目录规划
| 目录名称 | 对应名称 | 目录作用 |
|---|---|---|
| /opt/software/openGauss | software | 安装软件存放目录 |
| /opt/gaussdb/install/app | gaussdbAppPath | 数据库安装目录 |
| /var/log/omm | gaussdbLogPath | 日志目录 |
| /opt/gaussdb/tmp | tmpMppdbPath | 临时文件目录 |
| /opt/gaussdb/install/om | gaussdbToolPath | 数据库工具目录 |
| /opt/gaussdb/corefile | corePath | 数据库core文件目录 |
| /opt/gaussdb/data/cmserver | cmDir | CM数据目录 |
| /opt/gaussdb/install/data/dn | dataNode | 数据库主备节点数据目录 |
三、环境准备
3.1 安装python
本次安装选择Python 3.6.10版本，使用–enable-shared方式编译。
# root用户执行【所有节点】
-- 安装依赖包
[root@xsky-nodexxx ~]# yum install -y gcc-c++ pcre pcre-devel zlib zlib-devel openssl openssl-devel
-- 创建安装目录
[root@xsky-nodexxx ~]# mkdir /usr/local/python3
[root@xsky-nodexxx ~]# cd /home/soft
-- 如服务器可以连接网络，可通过wget获取安装包，如无法联网，可下载安装包并上传至服务器
[root@xsky-nodexxx soft]# wget https://www.python.org/ftp/python/3.6.10/Python-3.6.10.tar.xz
-- 解压源码包
[root@xsky-nodexxx soft]# tar xvJf Python-3.6.10.tar.xz
[root@xsky-nodexxx soft]# cd Python-3.6.10
-- 配置编译参数
[root@xsky-nodexxx Python-3.6.10]# ./configure --prefix=/usr/local/python3 --enable-optimizations --enable-shared CFLAGS=-fPIC --with-ssl
-- 执行安装
[root@xsky-nodexxx Python-3.6.10]# make && make install
-- 创建链接
[root@xsky-nodexxx Python-3.6.10]# ln -s /usr/local/python3/bin/python3 /usr/bin/python3
-- 检验python版本
[root@xsky-nodexxx ~]# python3
Python 3.6.10 (default, Jul 12 2023, 17:08:53)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
3.2 安装软件依赖包
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# yum install -y libaio-devel flex bison ncurses-devel glibc-devel patch redhat-lsb-core readline-devel zlib readline
-- 检查是否已安装
[root@xsky-nodexxx ~]# rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" | grep -E "libaio-devel|flex|bison|ncurses-devel|glibc-devel|patch|redhat-lsb-core|readline-devel|zlib|readline|expect"
3.3 关闭防火墙
# root用户执行【所有节点】
-- 停止 firewalld 服务
[root@xsky-nodexxx ~]# systemctl stop firewalld.service
-- 禁用 firewalld 服务
[root@xsky-nodexxx ~]# systemctl disable firewalld.service
-- 查看 firewalld 服务状态
[root@xsky-nodexxx ~]# systemctl status firewalld
-- 显示如下表示已关闭禁用防火墙
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
3.4 关闭SELinux
# root用户执行【所有节点】
-- 临时关闭SELinux
[root@xsky-nodexxx ~]# setenforce 0
-- 永久关闭SELinux
[root@xsky-nodexxx ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config
-- 检查SELinux状态
[root@xsky-nodexxx ~]# getenforce
-- 状态为Disabled表明SELinux已关闭
Disabled
3.5 修改字符集
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# cat>> /etc/profile<<EOF
export LANG=en_US.UTF-8
EOF
[root@xsky-nodexxx ~]# source /etc/profile
-- 检查字符集
[root@xsky-nodexxx ~]# env |grep -i lang
3.6 设置时区和时间
# root用户执行【所有节点】
# 使用ntp设置时钟同步
-- 安装ntp服务
[root@xsky-nodexxx ~]# yum install -y ntp
-- 开机启动ntp服务
[root@xsky-nodexxx ~]# systemctl enable ntpd
-- 启动ntp服务
[root@xsky-nodexxx ~]# systemctl start ntpd
-- 设置时区Asia/Shanghai
[root@xsky-nodexxx ~]# timedatectl set-timezone Asia/Shanghai
-- 检查时区
[root@xsky-nodexxx ~]# timedatectl |grep -i zone
-- 启用ntp服务
[root@xsky-nodexxx ~]# timedatectl set-ntp yes
-- 编辑定时任务列表
[root@xsky-nodexxx ~]# crontab -e
-- 使用vi/vim对定时任务列表进行编辑，添加如下内容变保存退出
0 12 * * * ntpdate cn.pool.ntp.org
-- 查看时间及时区
[root@xsky-nodexxx ~]# timedatectl status
3.7 修改硬件时钟
# root用户执行【所有节点】
-- 将当前系统时间写入硬件时钟
[root@xsky-nodexxx ~]# hwclock --systohc
-- 查看硬件时钟
[root@xsky-nodexxx ~]# hwclock
3.8 关闭SWAP
# root用户执行【所有节点】
-- 禁用当前的 swap 分区
[root@xsky-nodexxx ~]# swapoff -a
# 永久关闭swap 分区
-- 使用vi/vim编辑 /etc/fstab 文件,注释如下内容
UUID=<swap_partition_uuid> swap swap defaults 0 0
-- 保存退出，下次重启会生效
3.9 设置网卡MTU
# root用户执行【所有节点】
-- 查看网卡列表
[root@xsky-nodexxx ~]# ip addr
-- 假设网卡名称为eth0,设置网卡MTU，MTU值推荐8192，要求不小于1500
[root@xsky-nodexxx ~]# ip link set dev eth0 mtu 8192
-- 查看网卡MTU
[root@xsky-nodexxx ~]# ifconfig -a |grep -i mtu
3.10 关闭透明大页
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# cat >> /etc/rc.d/rc.local<<EOF
if test -f /sys/kernel/mm/transparent_hugepage/enabled;
then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag;
then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
EOF
-- 更新 GRUB 配置以使更改生效
[root@xsky-nodexxx ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
-- 重启操作系统生效
[root@xsky-nodexxx ~]# shutdown -r 0
-- 查看透明大页是否关闭，如输出显示为 never，则透明大页和透明大页的碎片整理已成功关闭
[root@xsky-nodexxx ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[root@xsky-nodexxx ~]# cat /sys/kernel/mm/transparent_hugepage/defrag
3.11 设置root用户远程登陆
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# cat >>/etc/ssh/sshd_config<<EOF
PermitRootLogin yes
EOF
-- 检查设置结果
[root@xsky-nodexxx ~]# cat /etc/ssh/sshd_config |grep PermitRootLogin
3.12 配置SSH
# root用户执行【所有节点】
-- 禁用 SSH 登录时的横幅（Banner）
[root@xsky-nodexxx ~]# sed -i '/Banner/s/^/#/' /etc/ssh/sshd_config
-- 目的是禁用允许以 root 用户登录的配置选项
[root@xsky-nodexxx ~]# sed -i '/PermitRootLogin/s/^/#/' /etc/ssh/sshd_config
[root@xsky-nodexxx ~]# echo -e "\n" >> /etc/ssh/sshd_config
[root@xsky-nodexxx ~]# echo "Banner none " >> /etc/ssh/sshd_config
# 修改Banner配置，去掉连接到系统时，系统提示的欢迎信息。欢迎信息会干扰安装时远程操作的返回结果，影响安装正常执行
[root@xsky-nodexxx ~]# echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
[root@xsky-nodexxx ~]# cat /etc/ssh/sshd_config |grep -v ^#|grep -E 'PermitRootLogin|Banner'
-- 重启生效
[root@xsky-nodexxx ~]# systemctl restart sshd.service
-- 查看SSH状态
[root@xsky-nodexxx ~]# systemctl status sshd.service
3.13 添加hosts解析
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# cat >> /etc/hosts<<EOF
10.110.7.39 xsky-node1
10.110.7.40 xsky-node2
10.110.7.41 xsky-node3
10.110.7.42 xsky-node4
EOF
3.14 调整系统内核参数
如下系统内核参数预安装时脚本未进行自动设置，需通过手工调整。
| 参数名称 | 参数说明 | 推荐取值 |
|---|---|---|
| net.ipv4.tcp_retries1 | 在连接建立过程中TCP协议最大重试次数 | 5 |
| net.ipv4.tcp_syn_retries | TCP协议SYN报文最大重试次数 | 5 |
| net.ipv4.tcp_synack_retries | TCP协议SYN应答报文最大重试次数 | 5 |
| net.ipv4.ip_local_port_range | 物理机可用临时端口范围 | 26000-65535 |
| net.ipv4.tcp_fin_timeout | 系统默认的超时时间 | 60 |
| net.ipv4.tcp_sack | 此参数用于启用或禁用TCP的SACK功能,SACK允许接收端向发送端告知它已经成功接收哪些数据包,从而提高TCP传输效率 0 表示关闭 1 表示开启 | 1 |
| net.ipv4.tcp_timestamps | 此参数用于启用或禁用TCP的时间戳功能,可提高TCP传输精度和效率 0 表示关闭 1 表示开启 | 1 |
| vm.overcommit_ratio | 此参数定义了内存过量分配的比例,当这个比例超过100%时，内核会允许进程分配超过实际可用内存的内存 | 90 |
| vm.extfrag_threshold | 此参数定义了内存碎片的阈值，当内存碎片超过这个阈值时，内核会尝试进行碎片整理以提高内存利用率 | 500 |
# root用户执行【所有节点】
[root@xsky-nodexxx ~]# tee -a /etc/sysctl.conf << EOF
net.ipv4.tcp_retries1=5
net.ipv4.tcp_syn_retries=5
net.ipv4.tcp_synack_retries=5
net.ipv4.tcp_fin_timeout=60
net.ipv4.ip_local_port_range = 26000 65535
net.ipv4.tcp_sack=1
net.ipv4.tcp_timestamps=1
vm.extfrag_threshold=500
vm.overcommit_ratio=90
EOF
-- 然后执行sysctl -p使修改生效
[root@xsky-nodexxx ~]# sysctl -p
3.15 创建目录
# root用户执行【主节点】
[root@xsky-nodexxx ~]# mkdir -p /opt/software/openGauss
[root@xsky-nodexxx ~]# chmod 755 -R /opt/software
3.16 下载软件安装包
3.16.1 下载安装包
使用注册账号登录openGuass官网https://www.opengauss.org/zh/download/下载页面，下载与操作系统匹配的openGauss 5.0.0软件安装包，选择openGauss_5.0.0 企业版下载，并将下载的软件包上传至服务器/opt/software/openGauss目录下。
注：可用鼠标右键点击 
，然后选择“复制链接”，如数据库服务器可连外网，可在服务器上通过wget获取openGauss 5.0.0企业版软件安装包。
[root@xsky-node1 ~]# cd /opt/software/openGauss
[root@xsky-node1 openGauss]# wget https://opengauss.obs.cn-south-1.myhuaweicloud.com/5.0.0/x86/openGauss-5.0.0-CentOS-64bit-all.tar.gz
3.16.2 校验安装包
点击上图 
后 
，将复制的内容粘贴到文本文件，显示内容为：aa9fc724c5030f4cc79dad201675183029c8f36a07667028e681169a2f6482f5，然后将下载的文件通过sha256sum命令进行校验，已确保下载安装包完整性。
# root用户执行【主节点】
[root@xsky-node1 openGauss]# sha256sum openGauss-5.0.0-CentOS-64bit-all.tar.gz
aa9fc724c5030f4cc79dad201675183029c8f36a07667028e681169a2f6482f5 openGauss-5.0.0-CentOS-64bit-all.tar.gz
-- 如校验的值和官网SHA256值相同，表明文件完整
3.16.3 解压安装包
# root用户执行【主节点】
[root@xsky-node1 openGauss]# tar -zxvf openGauss-5.0.0-CentOS-64bit-all.tar.gz
[root@xsky-node1 openGauss]# tar -zxvf openGauss-5.0.0-CentOS-64bit-om.tar.gz
[root@xsky-node1 openGauss]# ll
total 261040
drwxr-xr-x 14 root root 302 Mar 29 03:22 lib
-rw-r--r-- 1 root root 133071038 Mar 29 20:11 openGauss-5.0.0-CentOS-64bit-all.tar.gz
-rw-r--r-- 1 root root 105 Mar 29 03:23 openGauss-5.0.0-CentOS-64bit-cm.sha256
-rw-r--r-- 1 root root 22356000 Mar 29 03:23 openGauss-5.0.0-CentOS-64bit-cm.tar.gz
-rw-r--r-- 1 root root 65 Mar 29 03:22 openGauss-5.0.0-CentOS-64bit-om.sha256
-rw-r--r-- 1 root root 11963876 Mar 29 03:22 openGauss-5.0.0-CentOS-64bit-om.tar.gz
-rw-r--r-- 1 root root 65 Mar 29 03:23 openGauss-5.0.0-CentOS-64bit.sha256
-rw-r--r-- 1 root root 99384569 Mar 29 03:23 openGauss-5.0.0-CentOS-64bit.tar.bz2
drwxr-xr-x 10 root root 4096 Mar 29 03:22 script
-rw------- 1 root root 65 Mar 29 03:21 upgrade_sql.sha256
-rw------- 1 root root 493211 Mar 29 03:21 upgrade_sql.tar.gz
-rw-r--r-- 1 root root 32 Mar 29 03:22 version.cfg
3.17 建立互信
此处使用脚本建立root互信，也可手工建立互信，建立互信前将集群各节点root用户设置为相同口令。
# root用户执行【主节点】
-- 1) 在主节点创建sshtrust.sh互信脚本
[root@xsky-node1 ~]# tee -a /root/sshtrust.sh << EOF
#!/bin/bash
HOSTLIST="
xsky-node1
xsky-node2
xsky-node3
xsky-node4"
# 依赖sshpass包，使用前先rpm -qa|grep sshpass,若无回显则需要安装
rpm -q sshpass &> /dev/null || yum -y install sshpass
[ -f /root/.ssh/id_rsa ] || ssh-keygen -f /root/.ssh/id_rsa -P ''
export SSHPASS=root
for HOST in $HOSTLIST; do
{
HOSTNAME=$(getent hosts $HOST | awk '{ print $2 }')
sshpass -e ssh-copy-id -o StrictHostKeyChecking=no root@$HOSTNAME
# 拷贝主节点/root/.ssh 目录到其他节点
scp -r /root/.ssh root@$HOSTNAME:/root
} &
done
wait
EOF
-- 2) 授权/root/sshtrust.sh
[root@xsky-node1 ~]# chmod +x /root/sshtrust.sh
-- 3) 执行/root/sshtrust.sh
[root@xsky-node1 ~]# sh sshtrust.sh
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:p8d8j8Lyj2iwpf1MgwFX62pBr9mJvxX1HSDbCPEzJRY root@xsky-node1
The key's randomart image is:
+---[RSA 2048]----+
| ooE.o |
| .+.B . |
| . o .* ... |
| + o o. .o|
| S + . o|
| . .# . . |
| *O.O o |
| o.+*o= o |
| ..+B+o . |
+----[SHA256]-----+
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -o 'StrictHostKeyChecking=no' 'root@xsky-node1'"
and check to make sure that only the key(s) you wanted were added.
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -o 'StrictHostKeyChecking=no' 'root@xsky-node4'"
and check to make sure that only the key(s) you wanted were added.
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -o 'StrictHostKeyChecking=no' 'root@xsky-node2'"
and check to make sure that only the key(s) you wanted were added.
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -o 'StrictHostKeyChecking=no' 'root@xsky-node3'"
and check to make sure that only the key(s) you wanted were added.
id_rsa 100% 1679 2.8MB/s 00:00
id_rsa.pub 100% 397 1.1MB/s 00:00
known_hosts 100% 736 2.0MB/s 00:00
authorized_keys 100% 97 263.3KB/s 00:00
id_rsa 100% 1679 3.7MB/s 00:00
id_rsa.pub 100% 397 1.3MB/s 00:00
known_hosts 100% 736 2.0MB/s 00:00
authorized_keys 100% 97 348.7KB/s 00:00
id_rsa 100% 1679 2.8MB/s 00:00
id_rsa.pub 100% 397 783.1KB/s 00:00
known_hosts 100% 736 1.4MB/s 00:00
authorized_keys 100% 97 189.9KB/s 00:00
id_rsa 100% 1679 3.0MB/s 00:00
id_rsa.pub 100% 397 1.0MB/s 00:00
known_hosts 100% 736 1.7MB/s 00:00
authorized_keys 100% 97 281.1KB/s 00:00
-- 4) 测试互信
[root@xsky-node1 ~]# ssh xsky-node2
Last login: Sun Jul 16 00:32:06 2023 from xsky-node1
[root@xsky-node2 ~]# ssh xsky-node3
Last login: Sun Jul 16 00:32:31 2023 from xsky-node1
[root@xsky-node3 ~]# ssh xsky-node4
Last login: Sun Jul 16 00:32:40 2023 from xsky-node3
[root@xsky-node4 ~]# ssh xsky-node1
Last login: Sun Jul 16 00:37:48 2023
[root@xsky-node1 ~]# ssh xsky-node3
Last login: Sun Jul 16 00:38:50 2023 from xsky-node2
3.18 创建XML配置文件
# root用户执行【主节点】
tee -a /opt/software/openGauss/cluster_config.xml << EOF
<?xml version="1.0" encoding="UTF-8"?>
<ROOT>
<!-- openGauss整体信息 -->
<CLUSTER>
<PARAM name="clusterName" value="openGSDB" />
<PARAM name="nodeNames" value="xsky-node1,xsky-node2,xsky-node3,xsky-node4" />
<PARAM name="gaussdbAppPath" value="/opt/gaussdb/install/app" />
<PARAM name="gaussdbLogPath" value="/var/log/omm" />
<PARAM name="tmpMppdbPath" value="/opt/gaussdb/tmp"/>
<PARAM name="gaussdbToolPath" value="/opt/gaussdb/install/om" />
<PARAM name="corePath" value="/opt/gaussdb/corefile"/>
<PARAM name="backIp1s" value="10.110.7.39,10.110.7.40,10.110.7.41,10.110.7.42"/>
</CLUSTER>
<!-- 每台服务器上的节点部署信息 -->
<DEVICELIST>
<!-- xsky-node1节点部署信息 -->
<DEVICE sn="1000001">
<PARAM name="name" value="xsky-node1"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
<PARAM name="backIp1" value="10.110.7.39"/>
<PARAM name="sshIp1" value="10.110.7.39"/>
<!-- cm主 -->
<PARAM name="cmsNum" value="1"/>
<PARAM name="cmDir" value="/opt/gaussdb/install/cm"/>
<PARAM name="cmServerPortBase" value="15300"/>
<PARAM name="cmServerListenIp1" value="10.110.7.39,10.110.7.40,10.110.7.41,10.110.7.42"/>
<PARAM name="cmServerHaIp1" value="10.110.7.39,10.110.7.40,10.110.7.41,10.110.7.42"/>
<!-- cmServerlevel目前只支持1 -->
<PARAM name="cmServerlevel" value="1"/>
<!-- cms主及所有备的hostname -->
<PARAM name="cmServerRelation" value="xsky-node1,xsky-node2,xsky-node3,xsky-node4"/>
<!--dn-->
<PARAM name="dataNum" value="1"/>
<PARAM name="dataPortBase" value="26000"/>
<PARAM name="dataNode1" value="/opt/gaussdb/install/data/dn,xsky-node2,/opt/gaussdb/install/data/dn,xsky-node3,/opt/gaussdb/install/data/dn,xsky-node4,/opt/gaussdb/install/data/dn"/>
<PARAM name="dataNode1_syncNum" value="0"/>
</DEVICE>
<!-- xsky-node2节点部署信息，其中"name"的值配置为主机名称 -->
<DEVICE sn="1000002">
<PARAM name="name" value="xsky-node2"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<PARAM name="backIp1" value="10.110.7.40"/>
<PARAM name="sshIp1" value="10.110.7.40"/>
<!-- cm -->
<PARAM name="cmDir" value="/opt/gaussdb/install/cm"/>
<PARAM name="cmServerPortStandby" value="15300"/>
</DEVICE>
<!-- xsky-node3节点部署信息，其中"name"的值配置为主机名称 -->
<DEVICE sn="1000003">
<PARAM name="name" value="xsky-node3"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<PARAM name="backIp1" value="10.110.7.41"/>
<PARAM name="sshIp1" value="10.110.7.41"/>
<!-- cm -->
<PARAM name="cmServerPortStandby" value="15300"/>
<PARAM name="cmDir" value="/opt/huawei/data/cmserver"/>
</DEVICE>
<!-- xsky-node4节点部署信息，其中"name"的值配置为主机名称 -->
<DEVICE sn="1000004">
<PARAM name="name" value="xsky-node4"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<PARAM name="backIp1" value="10.110.7.42"/>
<PARAM name="sshIp1" value="10.110.7.42"/>
<PARAM name="cascadeRole" value="on"/>
<!-- cm -->
<PARAM name="cmDir" value="/opt/huawei/install/cm"/>
<PARAM name="cmServerPortStandby" value="15300"/>
</DEVICE>
</DEVICELIST>
</ROOT>
EOF
四、执行安装
4.1 初始化安装
采用交互式进行初始化安装，创建omm用户并设置互信。
# root用户执行【主节点】
[root@xsky-node1 ~]# cd /opt/software/openGauss/script
-- 执行初始化安装
[root@xsky-node1 script]# ./gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/cluster_config.xml
-- 命令执行输出结果如下
Parsing the configuration file.
Successfully parsed the configuration file.
Installing the tools on the local node.
Successfully installed the tools on the local node.
Are you sure you want to create trust for root (yes/no)?yes -- 输入yes
Please enter password for root
Password: -- 输入root口令
Successfully created SSH trust for the root permission user.
Setting host ip env
Successfully set host ip env.
Distributing package.
Begin to distribute package to tool path.
Successfully distribute package to tool path.
Begin to distribute package to package path.
Successfully distribute package to package path.
Successfully distributed package.
Are you sure you want to create the user[omm] and create trust for it (yes/no)? yes
Please enter password for cluster user.
Password: -- 设置omm用户口令
Please enter password for cluster user again.
Password: -- 再次输入omm口令
Generate cluster user password files successfully.
Successfully created [omm] user on all nodes.
Preparing SSH service.
Successfully prepared SSH service.
Installing the tools in the cluster.
Successfully installed the tools in the cluster.
Checking hostname mapping.
Successfully checked hostname mapping.
Creating SSH trust for [omm] user.
Please enter password for current user[omm].
Password: -- 输入omm用户口令
Checking network information.
All nodes in the network are Normal.
Successfully checked network information.
Creating SSH trust.
Creating the local key file.
Successfully created the local key files.
Appending local ID to authorized_keys.
Successfully appended local ID to authorized_keys.
Updating the known_hosts file.
Successfully updated the known_hosts file.
Appending authorized_key on the remote node.
Successfully appended authorized_key on all remote node.
Checking common authentication file content.
Successfully checked common authentication content.
Distributing SSH trust file to all node.
Distributing trust keys file to all node successfully.
Successfully distributed SSH trust file to all node.
Verifying SSH trust on all hosts.
Successfully verified SSH trust on all hosts.
Successfully created SSH trust.
Successfully created SSH trust for [omm] user.
Checking OS software.
Successfully check os software.
Checking OS version.
Successfully checked OS version.
Creating cluster's path.
Successfully created cluster's path.
Set and check OS parameter.
Setting OS parameters.
Successfully set OS parameters.
Set and check OS parameter completed.
Preparing CRON service.
Successfully prepared CRON service.
Setting user environmental variables.
Successfully set user environmental variables.
Setting the dynamic link library.
Successfully set the dynamic link library.
Setting Core file
Successfully set core path.
Setting pssh path
Successfully set pssh path.
Setting Cgroup.
Successfully set Cgroup.
Set ARM Optimization.
No need to set ARM Optimization.
Fixing server package owner.
Setting finish flag.
Successfully set finish flag.
Preinstallation succeeded.
-- 如果初始化过程中有warning或者Abnormal，初始化输出结果中会提示执行 /opt/software/openGauss/script/gs_checkos -i A -h xsky-node1,xsky-node2,xsky-node3,xsky-node4 --detail 来检查初始化过程中warning或者Abnormal详细信息，按照信息提示进行修改。
-- 可进入/var/log/omm目录下查看相关初始化日志信息
4.2 执行安装
# omm用户执行【主节点】
-- 切换至omm用户
[root@xsky-node1 ~]# su - omm
-- 执行安装
[omm@xsky-node1 ~]$ gs_install -X /opt/software/openGauss/cluster_config.xml --gsinit-parameter="--locale=en_US.utf8"
-- 上述命令执行输出结果如下
Parsing the configuration file.
Check preinstall on every node.
Successfully checked preinstall on every node.
Creating the backup directory.
Successfully created the backup directory.
begin deploy..
Installing the cluster.
begin prepare Install Cluster..
Checking the installation environment on all nodes.
begin install Cluster..
Installing applications on all nodes.
Successfully installed APP.
begin init Instance..
encrypt cipher and rand files for database.
Please enter password for database: -- 设置数据库密码
Please repeat for database: -- 重复输入相同密码
begin to create CA cert files
The sslcert will be generated in /opt/gaussdb/install/app/share/sslcert/om
Create CA files for cm beginning.
Create CA files on directory [/opt/gaussdb/install/app_a07d57c3/share/sslcert/cm]. file list: ['cacert.pem', 'server.key', 'server.crt', 'client.key', 'client.crt', 'server.key.cipher', 'server.key.rand', 'client.key.cipher', 'client.key.rand']
Non-dss_ssl_enable, no need to create CA for DSS
Cluster installation is completed.
Configuring.
Deleting instances from all nodes.
Successfully deleted instances from all nodes.
Checking node configuration on all nodes.
Initializing instances on all nodes.
Updating instance configuration on all nodes.
Check consistence of memCheck and coresCheck on database nodes.
Successful check consistence of memCheck and coresCheck on all nodes.
Configuring pg_hba on all nodes.
Configuration is completed.
Starting cluster.
======================================================================
Successfully started primary instance. Wait for standby instance.
======================================================================
.
Successfully started cluster.
======================================================================
cluster_state : Normal
redistributing : No
node_count : 4
Datanode State
primary : 1
standby : 2
secondary : 0
cascade_standby : 1
building : 0
abnormal : 0
down : 0
Successfully installed application.
end deploy..
-- 数据库密码设置要符合以下要求
1) 最少包含8个字符，最多包含16个字符。
2) 不能和用户名、当前密码（ALTER）、或当前密码反序相同。
3) 至少包含大写字母（A-Z）、小写字母（a-z）、数字、非字母数字字符（限定为~!@#$%^&*()-_=+\|[{}];:,<.>/?）四类字符中的三类字符
-- 数据库安装执行过程可进入/var/log/omm/omm/om目录查看gs_install和gs_local两个日志文件
4.3 数据库目录及含义
数据库安装完毕后，会生成如下相关目录，各目录及其含义如下：
| 项目目录说明 | 目录名称 | 子目录名称 | 描述说明 |
|---|---|---|---|
| openGauss安装目录 | /opt/gaussdb/install/app | bin | 存放数据库二进制文件目录 |
| | | etc | cgroup工具配置文件 |
| | | include | 存放数据库运行所需要头文件 |
| | | lib | 存放数据库库文件目录 |
| | | share | 存放数据库运行所需要公共文件(如配置文件模板) |
| openGauss数据目录 | /opt/gaussdb/install/data/dn | / | dbnode实例数据目录 |
| openGauss日志目录 | /var/log/omm/omm | bin | 二进制程序的日志目录 |
| | | gs_profile | 数据库内核性能日志目录 |
| | | om | om日志目录 |
| | | pg_audit | 数据库审计日志目录 |
| | | pg_log | 数据库节点实例运行日志目录 |
| openGauss系统工具目录 | /opt/gaussdb/install/om | script | openGauss用户管理脚本文件目录 |
| | | lib | bin目录下二进制文件依赖库文件目录 |
| openGauss临时文件目录 | /opt/gaussdb/tmp | | 存放临时文件目录 |
4.4 安装验证
4.4.1 检查数据库集群状态
# omm用户执行【任意节点】
[root@xsky-nodexxx ~]# su - omm
[omm@xsky-nodexxx ~]$ gs_om -t status --detail
[ CMServer State ]
node node_ip instance state
------------------------------------------------------------------------------
1 xsky-node1 10.110.7.39 1 /opt/gaussdb/install/cm/cm_server Primary
2 xsky-node2 10.110.7.40 2 /opt/gaussdb/install/cm/cm_server Standby
3 xsky-node3 10.110.7.41 3 /opt/huawei/data/cmserver/cm_server Standby
4 xsky-node4 10.110.7.42 4 /opt/huawei/install/cm/cm_server Standby
[ Cluster State ]
cluster_state : Normal
redistributing : No
balanced : Yes
current_az : AZ_ALL
[ Datanode State ]
node node_ip instance state
---------------------------------------------------------------------------------
1 xsky-node1 10.110.7.39 6001 /opt/gaussdb/install/data/dn P Primary Normal
2 xsky-node2 10.110.7.40 6002 /opt/gaussdb/install/data/dn S Standby Normal
3 xsky-node3 10.110.7.41 6003 /opt/gaussdb/install/data/dn S Standby Normal
4 xsky-node4 10.110.7.42 6004 /opt/gaussdb/install/data/dn C Cascade Standby Normal
-- state状态显示为Normal表示数据库可正常使用
-- 也可以使用cm_ctl命令如下方式查看集群状态
[omm@xsky-nodexxx ~]$ cm_ctl query -Cv
[ CMServer State ]
node instance state
------------------------------
1 xsky-node1 1 Primary
2 xsky-node2 2 Standby
3 xsky-node3 3 Standby
4 xsky-node4 4 Standby
[ Cluster State ]
cluster_state : Normal
redistributing : No
balanced : Yes
current_az : AZ_ALL
[ Datanode State ]
node instance state | node instance state | node instance state | node instance state
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 xsky-node1 6001 P Primary Normal | 2 xsky-node2 6002 S Standby Normal | 3 xsky-node3 6003 S Standby Normal | 4 xsky-node4 6004 C Cascade Standby Normal
4.4.2 连接数据库测试
# omm用户执行【任意节点】
[root@xsky-nodexxx ~]# su - omm
-- 26000为数据库主节点端口号，即XML配置文件中dataPortBase值
[omm@xsky-node1 ~]$ gsql -d postgres -p 26000
gsql ((openGauss 5.0.0 build a07d57c3) compiled at 2023-03-29 03:07:56 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
openGauss=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+------------+------------+-------------------
postgres | omm | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
template1 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
(3 rows)
openGauss=#
4.3.3 创建数据库
# omm用户执行【任意节点】
[root@xsky-nodexxx ~]# su - omm
[omm@xsky-node1 ~]$ gsql -d postgres -p 26000
gsql ((openGauss 5.0.0 build a07d57c3) compiled at 2023-03-29 03:07:56 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
-- 创建数据库gaussdb并使用UTF8字符集
openGauss=# CREATE DATABASE gaussdb WITH ENCODING 'UTF8' template = template0;
CREATE DATABASE
openGauss=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+------------+------------+-------------------
gaussdb | omm | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | omm | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
template1 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
(4 rows)
-- \q 命令可退出当前数据库
openGauss=# \q
-- 测试连接gaussdb数据库
[omm@xsky-node1 ~]$ gsql -d gaussdb -p 26000
gsql ((openGauss 5.0.0 build a07d57c3) compiled at 2023-03-29 03:07:56 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
gaussdb=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+------------+------------+-------------------
gaussdb | omm | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | omm | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
template1 | omm | UTF8 | en_US.utf8 | en_US.utf8 | =c/omm +
| | | | | omm=CTc/omm
(4 rows)
gaussdb=#
五、附录
5.1 Failed to check hostname mapping
**问题描述：**数据库初始化安装时有时会遇到如下报错：
[root@xsky-node1 script]# ./gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/cluster_config.xml
Parsing the configuration file.
Successfully parsed the configuration file.
Installing the tools on the local node.
Successfully installed the tools on the local node.
Are you sure you want to create trust for root (yes/no)?yes
Please enter password for root
Password:
Successfully created SSH trust for the root permission user.
Setting host ip env
Successfully set host ip env.
Distributing package.
Begin to distribute package to tool path.
Successfully distribute package to tool path.
Begin to distribute package to package path.
Successfully distribute package to package path.
Successfully distributed package.
Are you sure you want to create the user[omm] and create trust for it (yes/no)? yes
Please enter password for cluster user.
Password:
Please enter password for cluster user again.
Password:
Generate cluster user password files successfully.
Successfully created [omm] user on all nodes.
Preparing SSH service.
Successfully prepared SSH service.
Installing the tools in the cluster.
Successfully installed the tools in the cluster.
Checking hostname mapping.
[FAILURE] xsky-node1:
[GAUSS-51222] : Failed to check hostname mapping. Command: "pssh -s -H xsky-node3 hostname". Error:
Usage: pssh [OPTIONS] command [...]
pssh: error: no such option: -s
[FAILURE] xsky-node2:
[GAUSS-51222] : Failed to check hostname mapping. Command: "pssh -s -H xsky-node4 hostname". Error:
Usage: pssh [OPTIONS] command [...]
pssh: error: no such option: -s
[FAILURE] xsky-node3:
[GAUSS-51222] : Failed to check hostname mapping. Command: "pssh -s -H xsky-node1 hostname". Error:
Usage: pssh [OPTIONS] command [...]
pssh: error: no such option: -s
[FAILURE] xsky-node4:
[GAUSS-51222] : Failed to check hostname mapping. Command: "pssh -s -H xsky-node3 hostname". Error:
Usage: pssh [OPTIONS] command [...]
pssh: error: no such option: -s
**问题原因：**如果在服务器上手工安装了pssh包，安装后的pssh是不提供 -s 参数选项的，如下所示 
而在主节点执行了gs_preinstall后，会重新安装新的pssh，手工安装的pssh和执行初始化安装的pssh文件大小是不同的，版本不同，且数据库初始化安装的pssh提供 -s 参数，而手工安装的pssh是不提供 -s 参数的，由于主备环境pssh不一致，导致pssh校验主机名时报错。
**解决办法：**解决办法就是将其它备节点/usr/bin/pssh或/use/local/bin/pssh改名或删除，使用主节点初始化gs_preinstall默认提供的pssh文件，该操作无需人工干预。
5.2 libpython3.6m.so.1.0文件缺失
**问题描述：**执行数据库初始化或者执行python3命令时，会遇到如下报错：
[root@xsky-node1 script]# ./gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/cluster_config.xml --non-interactive
Parsing the configuration file.
Successfully parsed the configuration file.
Installing the tools on the local node.
Successfully installed the tools on the local node.
Setting host ip env
Successfully set host ip env.
Distributing package.
Begin to distribute package to tool path.
Successfully distribute package to tool path.
Begin to distribute package to package path.
Successfully distribute package to package path.
[GAUSS-51400] : Failed to execute the command: python3 '/opt/software/openGauss/script/local/PreInstallUtility.py' -t check_config -u omm -X /opt/software/openGauss/cluster_config.xml. Result:{'xsky-node4': 'Failure', 'xsky-node3': 'Success', 'xsky-node2': 'Success'}.
Error:
[SUCCESS] xsky-node2:
[SUCCESS] xsky-node3:
[FAILURE] xsky-node4:
Traceback (most recent call last):
File "/opt/software/openGauss/script/local/../gspylib/common/Common.py", line 64, in <module>
import psutil
File "/opt/software/openGauss/script/gspylib/common/../../../lib/psutil/__init__.py", line 102, in <module>
from . import _pslinux as _psplatform
File "/opt/software/openGauss/script/gspylib/common/../../../lib/psutil/_pslinux.py", line 26, in <module>
from . import _psutil_linux as cext
ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/software/openGauss/script/local/PreInstallUtility.py", line 37, in <module>
from gspylib.common.ParameterParsecheck import Parameter
File "/opt/software/openGauss/script/local/../gspylib/common/ParameterParsecheck.py", line 27, in <module>
from gspylib.common.Common import DefaultValue
File "/opt/software/openGauss/script/local/../gspylib/common/Common.py", line 98, in <module>
import psutil
File "/opt/software/openGauss/script/gspylib/common/../../../lib/psutil/__init__.py", line 102, in <module>
from . import _pslinux as _psplatform
File "/opt/software/openGauss/script/gspylib/common/../../../lib/psutil/_pslinux.py", line 26, in <module>
from . import _psutil_linux as cext
ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory
**问题原因：**如果服务器上曾安装过不同python3版本数据库，由于环境设置或被修改，可能导致提示libpython3.6m.so.1.0文件缺失。
解决办法：使用root用户执行find / -name "libpython3.6m.so.1.0"查找文件所在位置，比如/usr/local/python3/lib/libpython3.6m.so.1.0，然后将此文件拷贝到/usr/lib64目录下即可。
5.3 vm.overcommit_memory设置问题
**问题描述：**在执行数据库初始化时遇到如下报Abnormal问题：
[root@xsky-node1 ~]# /opt/software/openGauss/script/gs_checkos -i A -h xsky-node1,xsky-node2,xsky-node3,xsky-node4 --detail
[root@xsky-node1 ~]# /opt/software/openGauss/script/gs_checkos -i A -h xsky-node1,xsky-node2,xsky-node3,xsky-node4 --detail
Checking items:
A1. [ OS version status ] : Normal
[xsky-node1]
centos_7.9.2009_64bit
[xsky-node2]
centos_7.9.2009_64bit
[xsky-node4]
centos_7.9.2009_64bit
[xsky-node3]
centos_7.9.2009_64bit
A2. [ Kernel version status ] : Normal
The names about all kernel versions are same. The value is "3.10.0-1160.83.1.el7.x86_64".
A3. [ Unicode status ] : Normal
The values of all unicode are same. The value is "LANG=en_US.UTF-8".
A4. [ Time zone status ] : Normal
The informations about all timezones are same. The value is "+0800".
A5. [ Swap memory status ] : Warning
[xsky-node3]
SwapMemory 68719472640 TotalMemory 67288281088
A6. [ System control parameters status ] : Abnormal
[xsky-node3]
Abnormal reason: variable 'vm.overcommit_memory' RealValue '2' ExpectedValue '0'.
Check_SysCtl_Parameter failed.
[xsky-node2]
Abnormal reason: variable 'vm.overcommit_memory' RealValue '2' ExpectedValue '0'.
Check_SysCtl_Parameter failed.
[xsky-node4]
Abnormal reason: variable 'vm.overcommit_memory' RealValue '2' ExpectedValue '0'.
Check_SysCtl_Parameter failed.
**问题原因：**初始化安装时建议将vm.overcommit_memory修改为0，而当前值为2，原因是因为官方文档里提示设置vm.overcommit_ratio值时，需当vm.overcommit_memory=2时此参数生效，如果在/etc/sysctl.conf里设置了vm.overcommit_memory=2其实是和数据库初始化建议值存在矛盾。 
解决办法：不要手工在/etc/sysctl.conf里设置vm.overcommit_memory值，利用数据库初始化自动设置该值。
本文分享自微信公众号 - openGauss（openGauss）。
如有侵权，请联系 support@oschina.cn 删除。
本文参与“ OSC源创计划 ”，欢迎正在阅读的你也加入，一起分享。
