运维架构体系搭建系列-第二篇-devops相关服务搭建_内网搭建devops

一、cicd工具链搭建

1.代码库搭建(gitlab)

最早时候公司有一套gogs，轻量级的代码库，功能较少直接没用了，选择代码库前有考虑过阿里云的codeup，后来为了代码安全还是自建gitlab。

现在已经是0202年了，搭建服务首选肯定是docker拉了镜像直接干。

1.1 下载docker pull镜像启动

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-18.06.1.ce-3.el7
systemctl start docker
systemctl enable docker
1
2
3
4

这里可以修改下镜像下载地址配置阿里云加速镜像站
阿里云镜像服务界面找到加速域名

docker配置文件加入

vim /etc/docker/daemon.json

{
    "registry-mirrors": ["https://xxxxxx.mirror.aliyuncs.com"],
    "live-restore": true
}
#重启
systemctl restart docker
1
2
3
4
5
6
7
8

这里注意如果需要用到ssh免密的需要映射一个2222对应22的端口出来，在配置文件也需要配置2222的ssh

docker pull beginor/gitlab-ce:11.0.1-ce.0
docker run --detach --publish 8443:443 --publish 2222:22--publish 8090:80 --name gitlab --restart always --hostname 10.0.0.0  -v /data/software/gitlab/etc:/etc/gitlab -v /data/software/gitlab/logs:/var/log/gitlab -v /data/software/gitlab/data:/var/opt/gitlab -v /etc/localtime:/etc/localtime:ro --privileged=true beginor/gitlab-ce
1
2

1.2 支持ssh端口配置

vim /data/software/gitlab/etc/gitlab.rb
gitlab_rails['gitlab_shell_ssh_port'] = 2222
gitlab_rails['gitlab_shell_git_timeout'] = 800
1
2
3

1.3 邮件发送配置

vim /data/software/gitlab/etc/gitlab.rb
#腾讯邮箱
nginx['enable'] = true
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "a@qq.com"
gitlab_rails['smtp_password'] = "fkvgpkocjfatbcee"
gitlab_rails['smtp_domain'] = "qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true
user['git_user_email'] = "a@qq.com"
gitlab_rails['gitlab_email_from'] = 'a@qq.com'
1
2
3
4
5
6
7
8
9
10
11
12

#阿里邮箱
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.qiye.aliyun.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "gitlab@.cn"
gitlab_rails['smtp_password'] = "@8888"
gitlab_rails['smtp_domain'] = "qiye.aliyun.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true
user['git_user_email'] = "gitlab@.cn"
gitlab_rails['gitlab_email_from'] = 'gitlab@.cn'
1
2
3
4
5
6
7
8
9
10
11
12

1.4 并发问题配置

在SourceTree升级到2.1.10.0后，默认后台并发查询git仓库更新，项目较多，这个并发查询超了GitLab并发访问的阈值，会封了IP GitLab响应Forbidden

vim /data/software/gitlab/etc/gitlab.rb
gitlab_rails['rack_attack_git_basic_auth'] = {
   'enabled' => true,
   'ip_whitelist' => ["127.0.0.1","0.0.0.0"],
   'maxretry' => 200,
   'findtime' => 60,
   'bantime' => 3600
 }
1
2
3
4
5
6
7
8

1.5 备份到oss配置

vim /data/software/gitlab/etc/gitlab.rb
gitlab_rails['backup_upload_connection'] = {
'provider' => 'aliyun',
'aliyun_accesskey_id' => 'xxxxxxxxxxxxxxxx',
'aliyun_accesskey_secret' => 'xxxxxxxxxxxxmXZz',
'aliyun_oss_endpoint' => 'http://oss-cn-hangzhou-internal.aliyuncs.com',
'aliyun_oss_bucket' => 'gitlab-bk-data',
'aliyun_oss_location' => 'hangzhou',
}
gitlab_rails['backup_upload_remote_directory'] = 'gitlab'
1
2
3
4
5
6
7
8
9
10

配置备份计划及定期删除

59 23 * * *  /usr/bin/docker exec -t 55f86fb06075 gitlab-rake gitlab:backup:create >> /root/gitbk-nohup.out 2>&1
59 23 * * *  /usr/bin/find /data/software/gitlab/data/backups/ -mtime +10 -name '*.tar' -exec rm -rf {} \;
1
2

在阿里云oss上配置定期删除

1.6 支持ssl配置

域名转https

vim /data/software/gitlab/etc/gitlab.rb
nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.com.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key"
external_url "https://gitlab.cn"
nginx['redirect_http_to_https'] = true
1
2
3
4
5

进入容器重新导入配置执行

docker exec -it 55f86fb06075 /bin/bash
gitlab-ctl reconfigure
1
2

1.7 本地nginx配置并转发到容器

本机nginx配置转发到容器中
https的证书可直接从阿里申请

upstream gitlab{
    server localhost:8443;
}

# 转发到容器
server{
    listen 443;
    server_name gitlab.xxx.cn;
    client_max_body_size 1000m;

    ssl_certificate /root/.cert/gitlab.example.com.crt;
    ssl_certificate_key /root/.cert/gitlab.example.com.key;

    location / {
        proxy_pass https://gitlab;
        proxy_http_version 1.1;
        proxy_set_header X_FORWARDED_PROTO https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
    }
}

# 转发到443
server {
    listen 80;
    server_name  gitlab.xxx.cn;
    rewrite ^(.*)$ https://${server_name}$1 permanent;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

gitlab容器nginx配置

upstream gitlab-workhorse {
  server unix:/var/opt/gitlab/gitlab-workhorse/socket;
}

## Redirects all HTTP traffic to the HTTPS host
server {
  listen *:80;

  server_name gitlab.xxx.cn;#域名
  server_tokens off; ## Don't show the nginx version number, a security best practice


  location / {
    return 301 https://gitlab.xxx.cn:443$request_uri;
  }

  access_log  /var/log/gitlab/nginx/gitlab_access.log gitlab_access;
  error_log   /var/log/gitlab/nginx/gitlab_error.log;
}

server {
  listen *:443 ssl http2;


  server_name gitlab.xxx.cn;
  server_tokens off; ## Don't show the nginx version number, a security best practice

  ## Increase this if you want to upload large attachments
  ## Or if you want to accept large git objects over http
  client_max_body_size 0;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

备注：当你映射的时候忘记加入2222到22的端口时，ssh免密钥是无法使用的，这时候可以通过工具做一个转发

下载rinetd以下链接下载不了可能要重新找下载源了，这个工具个人感觉简单好用，需要的也可以留言

wget https://www.boutell.com/rinetd/http/rinetd.tar.gz
mkdir -p /usr/man/man8/
tar xvf rinetd.tar.gz -C /usr/man/man8/
cd /usr/man/man8/rinetd/
make && make install

vim /etc/rinetd.conf
0.0.0.0 222 172.17.0.2 22
allow *.*.*.*
#执行启动
rinetd -c /etc/rinetd.conf
1
2
3
4
5
6
7
8
9
10
11

2.jenkins搭建

由于前面此博客前面有docker的安装方式这里就写yum的安装方式吧

2.1 yum安装

cd /etc/yum.repos.d/
wget http://pkg.jenkins.io/redhat/jenkins.repo 
rpm --import http://pkg.jenkins.io/redhat/jenkins.io.key
yum install -y jenkins #默认安装最新版本。戒者直接安装 jenkins-2.93-1.1.noarch.rpm 包
vim /etc/sysconfig/jenkins
JENKINS_PORT="8081"
JENKINS_HOME="/var/lib/jenkins" #数据目录，建议用固态磁盘来存数据，可以自己定义
/etc/init.d/jenkins start #启劢
chkconfig jenkins on #设置开机启劢
chkconfig --list jenkins
1
2
3
4
5
6
7
8
9
10

2.2 全局插件配置

#查看密码文件
cat /var/lib/jenkins/secrets/initialAdminPassword 
1
2

配置jdk&maven&node

插件下载
NodeJS Plugin == node环境
Maven Integration == maven环境
Localization: Chinese (Simplified) == 中文包

配置全局工具 路径可自定义

2.3 nginx配置

server {
    listen 80;
    server_name jenkins.xxx.cn;
    return 301 https://$server_name$request_uri;
}

upstream jenkins {
        server 127.0.0.1:8090;
}

server {
    listen 443 ssl;
    server_name jenkins.xxx.cn;
    client_max_body_size 100m;
    charset utf-8;
    client_body_buffer_size 10M;
    proxy_redirect     off;
    proxy_set_header   Host             $host:$server_port;
    proxy_set_header   X-Real-IP        $remote_addr;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_next_upstream error timeout invalid_header http_502 http_503 http_504;
    proxy_ignore_client_abort on;
    proxy_read_timeout  180;
    proxy_buffering on;
    proxy_buffer_size 8k;

    proxy_buffers 8 8M;

    gzip                on;
    gzip_min_length     1000;
    gzip_types          text/plain text/css application/json text/xml application/xml application/xml+rss text/javascript;
    ssl_certificate /etc/nginx/conf.d/cert/jenkins.xxx.cn.pem;   
    ssl_certificate_key /etc/nginx/conf.d/cert/jenkins.xxx.cn.key;  
    ssl_session_timeout 5m;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;  #使用此加密套件。
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;   #使用该协议进行配置。
    ssl_prefer_server_ciphers on;

    location / {
       proxy_pass http://jenkins;
       proxy_redirect     default;
       proxy_set_header   Host             $host:$server_port;
       proxy_set_header   X-Real-IP        $remote_addr;
       proxy_set_header X-Forwarded-Host $host:$server_port;
       proxy_set_header X-Forwarded-Server $host;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

完成

3.jdk&nexus&maven&node基础环境搭建

3.1 jdk环境安装

#下载jdk包
jdk-8u231-linux-x64.tar.gz
#建立java目录
mkdir /usr/local/jdk
#解压
tar xvf jdk-8u231-linux-x64.tar.gz -C /usr/local/jdk/
#解析环境变量
vim /etc/profile.d/java.sh 
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_231
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
1
2
3
4
5
6
7
8
9
10
11

3.2 node环境安装

#下载
wget https://nodejs.org/dist/v12.0.0/node-v12.0.0-linux-x64.tar.gz
#创建服务目录
mkdir /usr/local/node
#解压到服务目录
tar xvf node-v12.0.0-linux-x64.tar.gz  -C /usr/local/node
#软链接做全局命令
ln -s /usr/local/node/node-v12.0.0-linux-x64/bin/npm /usr/local/bin/npm
ln -s /usr/local/node/node-v12.0.0-linux-x64/bin/node /usr/local/bin/node
1
2
3
4
5
6
7
8
9

3.3 maven环境安装

#下载地址
https://maven.apache.org/download.cgi
#创建目录
mkdir /usr/local/maven/ 
#解压
tar xvf apache-maven-3.6.3-bin.tar.gz -C /usr/local/maven/
#软链接做全局命令
ln -s /usr/local/maven/apache-maven-3.6.3/bin/mvn /usr/bin/mvn
1
2
3
4
5
6
7
8

maven安装完成后配置setting文件，加入nexus的仓库
vim conf/settings.xml
加入nexus的账号密码及定义releases和snapshots仓库的id

  <servers>
      <server>
        <id>maven-releases</id>
        <username>admin</username>
        <password>nexusnew</password>
    </server>
         <server>
        <id>maven-snapshots</id>
        <username>admin</username>
        <password>nexusnew</password>
    </server>
  </servers>
1
2
3
4
5
6
7
8
9
10
11
12

下载地址

  <mirrors>
    <mirror>
      <id>nexus</id>
      <mirrorOf>*</mirrorOf>
      <name>nexus osc</name>
      <url>https://nexus.xxx.cn/repository/maven-public/</url>
    </mirror>
  </mirrors>
1
2
3
4
5
6
7
8

到此完成配置

开发上传下载在pom.xml中配置

<repositories>
    <!-- 配置nexus远程仓库 -->
    <repository>
        <id>nexus</id>
        <name>Nexus Snapshot Repository</name>
        <url>https://nexus.xxx.cn/repository/maven-public/</url>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

<distributionManagement>
    <repository>
        <id>maven-releases</id>#此id对应maven中配置的id
        <name>Releases</name>
        <url>https://nexus.xxx.cn/repository/maven-releases</url>
    </repository>
    <snapshotRepository>
        <id>maven-snapshots</id>#此id对应maven中配置的id
        <name>Snapshot</name>
        <url>https://nexus.xxx.cn/repository/maven-snapshots</url>
    </snapshotRepository>
</distributionManagement>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

3.4 nexus安装配置

#下载
wget http://download.sonatype.com/nexus/3/nexus-3.14.0-04-unix.tar.gz
#解压
tar -zxvf  nexus-3.14.0-04-unix.tar.gz
#修改启动端口
vim nexus-3.14.0-04/etc/nexus-default.properties   #修改启动端口                             
application-port=8090
#设置打开最大文件数
vim  /etc/security/limits.conf
nexus          soft    nofile  65536
nexus          hard    nofile  65536
#启动
nohup ./nexus run &
1
2
3
4
5
6
7
8
9
10
11
12
13

页面登录
默认密码 admin admin123
注：记住修改密码在maven中的密码账号对应此的

进入maven-releases maven-snapshots 设置允许上传

nginx配置

upstream nexus-server{
    server 127.0.0.1:8085;
}

server {
    listen 80;
    server_name nexus.xxx.cn;
    location / {
        return 301 https://$server_name$request_uri;
    }

    location ~ /.well-known {
        root /tmp;
    }
}

server {
    listen 443 ssl;
    server_name nexus.xxx.cn;
    client_max_body_size 100m;

    ssl_certificate /etc/nginx/conf.d/cert/nexus.xxx.cn.pem;   #将domain name.pem替换成您证书的文件名。
    ssl_certificate_key /etc/nginx/conf.d/cert/nexus.xxx.cn.key;   #将domain name.key替换成您证书的密钥文件名。
    ssl_session_timeout 5m;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;  #使用此加密套件。
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;   #使用该协议进行配置。
    ssl_prefer_server_ciphers on;



    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto "https";
        proxy_pass http://nexus-server;
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

3.5 nexus npm仓库配置使用

登录nexus页面
选择blob stores 来新建

新建一个名为 npm-stores 的存储

npm的仓库有三种
hosted（私有仓库）：用于发布个人开发的npm组件
proxy（代理仓库）：可以代理npm和淘宝镜像
group（组合仓库）：对外公开的仓库，集合了hosted和proxy

创建仓库 hosted（私有仓库）
存储选择 开始建立的 npm-stores

创建仓库 proxy（代理仓库）
存储选择 开始建立的 npm-stores
代理地址可以写淘宝源：https://registry.npm.taobao.org


创建仓库 group（组合仓库）
存储选择 开始建立的 npm-stores

组合仓库中除了输入名称和选择存储空间之外，还要选择要包括的仓库，越靠上优先级越高，如果私有仓库在上，用户下载npm包的时候会优先下载私有仓库中的，如果私有仓库没有再去代理仓库中下载。

如不是内网使用 可建立用户权限来控制上传下载
在user菜单创建用户

授权

完成后可测试
设置npm下载registry
npm config set registry http://xxx/repository/npm-group/
指定下载
npm install --registry=https://nexus.pin-dao.cn/repository/npm-group
登录
npm login –registry=http://xxx/repository/npm-hosted/
发布
npm publish

二、项目管理

1.知识库搭建（conference）

注：我们用的付费版的 （