盐析白兔

这个屌丝很懒，什么也没留下！

热门标签

Centos7搭建高可用Harbor集群_harbor集群部署

作者：盐析白兔 | 2024-08-15 16:09:21

踩

harbor集群部署

在Centos7搭建高可用Harbor集群

一、主机清单

IP地址	主机名	描述
192.168.0.211	/	负载均衡VIP，5000端口
192.168.0.212	harbor01	Harbor实例1，5000端口
192.168.0.213	harbor02	Harbor实例2，5000端口
192.168.0.214	Docker Server	打包及上传docker镜像

harbor01与harbor02需要安装docker 、docker-compose、harbor、keepalived
Docker Server服务器需要安装docker 、docker-compose
二、系统升级

1、更新软件包列表

yum update -y
1

2、列出可用的更新软件包

# yum list updates
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.ustc.edu.cn
 * updates: mirrors.aliyun.com
1
2
3
4
5
6

3、查看当前系统版本

# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
1
2

4、更新系统

# yum upgrade
# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
1
2
3

三、内核升级

1、查看当前内核版本

# uname -r
3.10.0-1160.71.1.el7.x86_64
1
2

2、查看可升级内核版本

# yum list kernel 
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.ustc.edu.cn
 * updates: mirrors.aliyun.com
Installed Packages
kernel.x86_64                  3.10.0-1160.71.1.el7                                             @anaconda
kernel.x86_64                  3.10.0-1160.119.1.el7                                            @updates
1
2
3
4
5
6
7
8
9

可以看出只有一个内核版本3.10.0，而这次需要升级到内核6.9.7。不能直接使用yum update kernel-*打补丁升级了

3、查看是否安装ELRepo

[root@harbor01 ~]# yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
Loaded plugins: fastestmirror


Error getting repository data for elrepo-kernel, repository not found
1
2
3
4
5

ELRepo源是国外的一个只对Linux操作系统的第三方免费软件资源库，支持Linux和CentOS操作系统的软件安装和升级。

4、导入一个公钥

# rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
1

5、安装一下ELRepo源

# yum install -y https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm
1

6、安装新内核

# yum --enablerepo=elrepo-kernel install kernel-ml -y
1

7、设置新安装的内核成为默认启动选项

将配置文件中GRUB_DEFAULT参数saved改为0

# sed -i s/saved/0/g /etc/default/grub
1

8、更新配置文件

# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.9.7-1.el7.elrepo.x86_64
Found initrd image: /boot/initramfs-6.9.7-1.el7.elrepo.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-1160.119.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-1160.119.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-1160.71.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-1160.71.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-e227a3c248e94736968e30eee994bdb8
Found initrd image: /boot/initramfs-0-rescue-e227a3c248e94736968e30eee994bdb8.img
done
1
2
3
4
5
6
7
8
9
10
11

9、重启：reboot

# reboot
1

10、升级完成

# uname -r
6.9.7-1.el7.elrepo.x86_64
1
2

四、安装docker

1. 卸载旧版本(可选)

sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine
1
2
3
4
5
6
7
8

2、安装需要的[软件包

sudo yum install -y yum-utils device-mapper-persistent-data lvm2
1

3 、设置yum安装源

默认是国外的yum源，基本不可用，我们修改为阿里源。

//中央仓库
# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo 

//国内建议安装阿里仓库
# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
grabbing file http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@harbor01 ~]# yum makecache fast
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * elrepo: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.ustc.edu.cn
 * updates: mirrors.aliyun.com
base                                                                                                           | 3.6 kB  00:00:00     
docker-ce-stable                                                                                               | 3.5 kB  00:00:00     
elrepo                                                                                                         | 3.0 kB  00:00:00     
extras                                                                                                         | 2.9 kB  00:00:00     
updates                                                                                                        | 2.9 kB  00:00:00     
(1/2): docker-ce-stable/7/x86_64/updateinfo                                                                    |   55 B  00:00:00     
(2/2): docker-ce-stable/7/x86_64/primary_db                                                                    | 152 kB  00:00:00     
Metadata Cache Created
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

4、查看docker可用版本

# yum list docker-ce --showduplicates | sort -r
 * updates: mirrors.aliyun.com
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror
 * extras: mirrors.ustc.edu.cn
 * elrepo: mirrors.tuna.tsinghua.edu.cn
docker-ce.x86_64            3:26.1.4-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.1.3-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.1.2-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.1.1-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.1.0-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.0.2-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.0.1-1.el7                      docker-ce-stable
docker-ce.x86_64            3:26.0.0-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.5-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.4-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.3-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.2-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.1-1.el7                      docker-ce-stable
docker-ce.x86_64            3:25.0.0-1.el7                      docker-ce-stable
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

5、安装docker

# 安装最新版
yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y


# 安装指定版本
yum install docker-ce-VERSION_STRING docker-ce-cli-VERSION_STRING containerd.io docker-buildx-plugin docker-compose-plugin -y
1
2
3
4
5
6

6、开启docker服务

# systemctl start docker

# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: active (running) since Sat 2024-06-29 00:23:30 CST; 33s ago
     Docs: https://docs.docker.com
 Main PID: 1692 (dockerd)
    Tasks: 10
   Memory: 32.2M
   CGroup: /system.slice/docker.service
           └─1692 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Jun 29 00:23:28 harbor01 systemd[1]: Starting Docker Application Container Engine...
Jun 29 00:23:28 harbor01 dockerd[1692]: time="2024-06-29T00:23:28.660489449+08:00" level=info msg="Starting up"
Jun 29 00:23:28 harbor01 dockerd[1692]: time="2024-06-29T00:23:28.900546935+08:00" level=info msg="Loading containers: start."
Jun 29 00:23:30 harbor01 dockerd[1692]: time="2024-06-29T00:23:30.264744262+08:00" level=info msg="Firewalld: interface dock...urning"
Jun 29 00:23:30 harbor01 dockerd[1692]: time="2024-06-29T00:23:30.532756602+08:00" level=info msg="Loading containers: done."
Jun 29 00:23:30 harbor01 dockerd[1692]: time="2024-06-29T00:23:30.606393989+08:00" level=info msg="Docker daemon" commit=de5...=26.1.4
Jun 29 00:23:30 harbor01 dockerd[1692]: time="2024-06-29T00:23:30.606732113+08:00" level=info msg="Daemon has completed init...zation"
Jun 29 00:23:30 harbor01 dockerd[1692]: time="2024-06-29T00:23:30.863205537+08:00" level=info msg="API listen on /run/docker.sock"
Jun 29 00:23:30 harbor01 systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

7、安装阿里云镜像加速器

tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://复制自己的加速器地址.mirror.aliyuncs.com"]
}
EOF

systemctl daemon-reload

systemctl restart docker
1
2
3
4
5
6
7
8
9

8、设置docker开机自启

# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
1
2

五、安装docker-compose(通过python的pip安装)

# 安装 epel-release
yum install -y epel-release

# 安装 python-pip
yum install -y python-pip

# 安装 python3-pip
yum install -y python3-pip

# 更新 pip
pip3 install --upgrade pip

# 安装 docker-compose
pip3 install docker-compose

# 查看 docker-compose 版本号
docker-compose -version
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

六、安装harbor

1、下载并解压harbor安装包

cd /usr/local

wget https://github.com/goharbor/harbor/releases/download/v2.11.0/harbor-offline-installer-v2.11.0.tgz

tar -zxvf harbor-offline-installer-v2.11.0.tgz
1
2
3
4
5

2、拷贝并编辑yml文件

cd harbor
cp harbor.yml.tmpl harbor.yml
vim harbor.yml
1
2
3

修改hostname、port，并注释https相关参数。
在这里插入图片描述

3、执行./install.sh

./install
1

在这里插入图片描述

4、执行./prepare.sh

./prepare
docker-compose down -v
docker-compose up -d

#开启防火墙端口
firewall-cmd --zone=public --add-port=5000/tcp --permanent  
firewall-cmd --reload
1
2
3
4
5
6
7

5、登录

在这里插入图片描述

6、设置Harbor开机自启

vim /usr/lib/systemd/system/harbor.service
1

[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor

[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/local/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f /usr/local/harbor/docker-compose.yml down
 
[Install]
WantedBy=multi-user.target
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

7、服务启动

systemctl enable harbor
systemctl start harbor
1
2

8、修改daemon.json

分别在harbor01、harbor02、Docker Server三台主机添加 insecure-registries

# vim /etc/docker/daemon.json

{
"registry-mirrors": ["https://复制自己的加速器地址.aliyuncs.com"],
"insecure-registries":["192.168.0.211:5000","192.168.0.212:5000","192.168.0.213:5000"]
}

# systemctl daemon-reload

# systemctl restart docker

1
2
3
4
5
6
7
8
9
10
11

在harbor01、harbor02主机执行以下命令

docker-compose down -v

docker-compose up -d
1
2
3

9、分别在3台主机验证登录harbor服务器

# docker login 192.168.0.212:5000

# docker login 192.168.0.213:5000
1
2
3

在这里插入图片描述

在这里插入图片描述
显示Login Succeeded代表登录成功

七、实现Harbor仓库双向同步

1、新建目标

在这里插入图片描述

在这里插入图片描述

2、创建复制规则

八、实现高可用

1、安装keepalived

选择指定版本下载的地址

#分别在harbor01、harbor02主机安装
yum -y install make gcc openssl-devel libnfnetlink-devel libnl3-devel net-snmp-devel

yum install -y keepalived

cd /usr/local/

#安装最新版本的keepalived
wget https://keepalived.org/software/keepalived-2.3.1.tar.gz

tar vxf keepalived-2.3.1.tar.gz -C /usr/local/src

cd src/keepalived-2.3.1/

./configure --prefix=/usr/local/keepalived

 make -j 4 && make install
 
 cd /usr/local/keepalived/
 
 cp /usr/local/keepalived/sbin/keepalived /usr/sbin/keepalived
 
 cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

2、编写harbor健康检查脚本

# vim /usr/local/check_harbor.sh
1

脚本的具体内容

#!/bin/sh

# Harbor的健康检查URL  
HARBOR_HEALTH_URL="http://localhost:5000/api/v2.0/health"  
  
# 发送请求并检查状态码  
if curl -s --output /dev/null --write-out "%{http_code}" "$HARBOR_HEALTH_URL" | grep -q '^200$'; then  
    exit 0  # Harbor健康  
else  
    exit 1  # Harbor不健康  
fi
1
2
3
4
5
6
7
8
9
10
11

修改sh文件为可以执行状态

chmod +x /usr/local/check_harbor.sh
1

3、修改keepalived配置

vim /etc/keepalived/keepalived.conf
1

主节点harbor01

! Configuration File for keepalived

global_defs {
   notification_email {
        111111111@qq.com
   }

   router_id harbor_master

}

vrrp_script check_harbor {
    script "/usr/local/check_harbor.sh"
    interval 10 # 每10秒检查一次
    weight -20 
}

vrrp_instance VI_1 {
    state MASTER
    interface ens3
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass harbor1111
    }
    unicast_src_ip 192.168.0.212
    unicast_peer { 
        192.168.0.213
    }
    virtual_ipaddress {
        192.168.0.211
    }
    track_script {
        check_harbor
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

备节点harbor02

! Configuration File for keepalived

global_defs {
   notification_email {
        111111111@qq.com
   }

   router_id harbor_backup

}

vrrp_script check_harbor {
    script "/usr/local/check_harbor.sh"
    interval 10 # 每10秒检查一次
    weight -20 
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens3
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass harbor1111
    }
    unicast_src_ip 192.168.0.213
    unicast_peer {
        192.168.0.212
    }
    virtual_ipaddress {
        192.168.0.211
    }
    track_script {
        check_harbor
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

启动keepalived

systemctl start keepalived

systemctl enable keepalived

systemctl status keepalived
1
2
3
4
5

检测vip是否正常启用
在这里插入图片描述
检查在Docker Server使用vip登录是否正常

[root@dockerserver214 ~]# docker login 192.168.0.211:5000
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
1
2
3
4
5
6
7
8

在这里插入图片描述

4、向Harbor推送镜像

[root@dockerserver214 ~]# docker pull nginx

[root@dockerserver214 ~]# docker tag nginx:latest 192.168.0.211:5000/library/nginx:latest

[root@dockerserver214 ~]# docker push 192.168.0.211:5000/library/nginx
1
2
3
4
5

在这里插入图片描述

5、在harbor拉取镜像

[root@dockerserver214 ~]# docker pull 192.168.0.211:5000/library/nginx
Using default tag: latest
latest: Pulling from library/nginx
a2abf6c4d29d: Pull complete 
a9edb18cadd1: Pull complete 
589b7251471a: Pull complete 
186b1aaa4aa6: Pull complete 
b4df32aa5a72: Pull complete 
a0bcbecc962e: Pull complete 
Digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3
Status: Downloaded newer image for 192.168.0.211:5000/library/nginx:latest
192.168.0.211:5000/library/nginx:latest
1
2
3
4
5
6
7
8
9
10
11
12

九、配置https

1、创建ssl证书

#创建目录
mkdir -p /usr/local/cret
cd /usr/local/cret/

#安装所需工具
yum -y install openssl

#创建ca密钥
openssl genrsa -out ca.key 4096

#创建ca证书，前提是已经创建了ca密钥
openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=192.168.0.211"  -key ca.key  -out ca.crt

#在创建已宿主机IP地址为名称的私钥
openssl genrsa -out 192.168.0.211.key 4096
openssl req -sha512 -new -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=192.168.0.211" -key 192.168.0.211.key -out 192.168.0.211.csr

cat > v3.ext <<EOF
 authorityKeyIdentifier=keyid,issuer
 basicConstraints=CA:FALSE
 keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
 extendedKeyUsage = serverAuth
 subjectAltName = @alt_names

 [alt_names]
 DNS.1=192.168.1.11
 EOF
 
 openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in 192.168.0.211.csr -out 192.168.0.211.crt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

所有ssl正式已经创建完成
在这里插入图片描述
2、修改Harbor文件

3、启动harbor

docker-compose down -v

docker-compose up -d
1
2
3

4、修改harbor健康检查脚本

#!/bin/sh
# Harbor的健康检查URL  
HARBOR_HEALTH_URL="https://localhost/api/v2.0/health"  
  
# 发送请求并检查状态码  
if curl -s --insecure --output /dev/null --write-out "%{http_code}" "$HARBOR_HEALTH_URL" | grep -q '^200$'; then  
    exit 0  # Harbor健康  
else  
    exit 1  # Harbor不健康  
fi
1
2
3
4
5
6
7
8
9
10

5、修改daemon.json

分别在harbor01、harbor02、Docker Server三台主机添加 insecure-registries

# vim /etc/docker/daemon.json

{
"registry-mirrors": ["https://复制自己的加速器地址.aliyuncs.com"],
"insecure-registries":["192.168.0.211","192.168.0.212","192.168.0.213"]
}

# systemctl daemon-reload

# systemctl restart docker
1
2
3
4
5
6
7
8
9
10

6、重新新建目标
在这里插入图片描述

7、创建复制规则

在这里插入图片描述

8、向Harbor推送镜像

[root@dockerserver214 ~]# docker pull nginx

[root@dockerserver214 ~]# docker tag nginx:latest 192.168.0.211/library/nginx:latest

[root@dockerserver214 ~]# docker push 192.168.0.211/library/nginx:latest
1
2
3
4
5

本文内容由网友自发贡献，转载请注明出处：https://www.wpsshop.cn/w/盐析白兔/article/detail/984276

Centos7搭建高可用Harbor集群_harbor集群部署

在Centos7搭建高可用Harbor集群

一、主机清单

二、系统升级

1、更新软件包列表

2、列出可用的 更新软件包

3、查看当前系统版本

4、更新系统

三、内核升级

1、查看当前内核版本

2、查看可升级内核版本

3、查看是否安装ELRepo

4、导入一个公钥

5、安装一下ELRepo源

6、安装新内核

7、设置新安装的内核成为默认启动选项

8、更新配置文件

9、重启：reboot

10、升级完成

四、安装docker

1. 卸载旧版本(可选)

2、安装需要的[软件包

3 、设置yum安装源

4、查看docker可用版本

5、安装docker

6、 开启docker服务

7、安装阿里云镜像加速器

8、设置docker开机自启

五、安装docker-compose(通过python的pip安装)

六、安装harbor

1、下载并解压harbor安装包

2、拷贝并编辑yml文件

3、执行./install.sh

4、执行./prepare.sh

5、登录

6、设置Harbor开机自启

7、服务启动

8、修改daemon.json

9、分别在3台主机验证登录harbor服务器

七、实现Harbor仓库双向同步

1、新建目标

八、实现高可用

1、安装keepalived

2、编写harbor健康检查脚本

3、修改keepalived配置

4、向Harbor推送镜像

5、在harbor拉取镜像

九、配置https

1、创建ssl证书

3、启动harbor

4、修改harbor健康检查脚本

5、修改daemon.json

7、创建复制规则

8、向Harbor推送镜像

2、列出可用的更新软件包

6、开启docker服务