做这个实验目标是往kubernetes集群里交付dubbo服务,采用jenkins+maven+gitee实现,jenkins使用的是v2.303.1,maven使用3.6.3。jenkins数据存储在nfs目录中。如下图所示:

主机名

提供服务

host11

nginx-proxy、dns

host21

kubernetes node(v.1.15.2)

host22

kubernetes node(v1.15.2)

host200

harbor、httpd、nfs

[kubernetes] 交付dubbo之jenkins联动docker_docker

​​01 jenkins镜像准备​​

1)在host200上下载jenkins镜像

docker pull jenkins/jenkins:2.303.1

2)添加tag,上传到自己的镜像仓库

docker tag 619aabbe0502 harbor.od.com/public/jenkins:2.303.1docker push harbor.od.com/public/jenkins:2.303.1

3)准备dockerfile,以官方镜像为底包,构建能和docker联动的镜像

mkdir -pv /data/dockerfile/jenkinscd /data/dockerfile/jenkins
# 准备sshkeyssh-keygen -t rsa -b 2048 -C "yunweishaonian@yeah.net" -N "" -f /root/.ssh/id_rsa# 将key复制到当前目录下cp -a /root/.ssh/id_rsa .# 获取get-docker.sh,主要是给jenkens安装docker客户端curl -fsSL get.docker.com -o /data/dockerfile/jenkins/get-docker.sh# 复制docker的配置文件cp -a /root/.docker/config.json .# 添加config文件,后卖就可以绵密登录gitee了vi configHost gitee.com    HostName gitee.com    User git    PreferredAuthentications publickey    IdentityFile /root/.ssh/id_rsa# 编辑dockerfilevi /data/dockerfile/jenkins/DockerfileFROM harbor.od.com/public/jenkins:2.303.1USER rootRUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\    echo 'Asia/Shanghai' > /etc/timezoneADD id_rsa /root/.ssh/id_rsaADD config.json /root/.docker/config.jsonADD get-docker.sh /get-docker.shADD config /root/.ssh/configRUN echo "      StrictHostKeyChecking no" >> /etc/ssh/ssh_config &&\        /get-docker.sh
# build成一个新镜像docker build . -t harbor.od.com/infra/jenkins:v2.303.1# 上传至harbordocker push harbor.od.com/infra/jenkins:v2.303.1

4)在gitee配置公钥

[kubernetes] 交付dubbo之jenkins联动docker_json_02

5)测试jenkens镜像是否可用

# 返回成功即可docker -run --rm harbor.od.com/infra/jenkins:v2.303.1 ssh -T git@gitee.com

​​02 kubernetes 准备​​

1)创建namespace

# 创建namespacekubectl create namespace infra# 创建secret,这里的密码为harbor的密码kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=xxxxx -n infra

​03 nfs准备​

1)每个节点都安装nfs-utils

yum install nfs-utils -y

2)在host200上创建共享目录

mkdir -p /data/nfs-volume/jenkins_home

3)host200配置nfs

vi /etc/exports/data/nfs-volume 192.168.122.0/24(rw,no_root_squash)

4)host200启动nfs服务

systemctl enable nfssystemctl start nfs

5)测试

[root@host200 ~]# showmount -eExport list for host200.yunwei.com:/data/nfs-volume 192.168.122.0/24  # node地址段

​04 资源配置清单​

1)创建目录(/data/k8s-yaml/jenkins == http://k8s-yaml.od.com/jenkins/)

mkdir /data/k8s-yaml/jenkins/

2)创建dp.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata:  name: jenkins     namespace: infra  labels:     name: jenkinsspec:  replicas: 1  selector:    matchLabels:       name: jenkins  template:    metadata:      labels:         app: jenkins         name: jenkins    spec:      volumes:      - name: data # 创建一个卷        nfs:  # 卷类型是nfs          server: host200 # nfs服务器地址          path: /data/nfs-volume/jenkins_home      - name: docker # 创建一个卷        hostPath:  # 类型是hostpath,即本节点路径          path: /run/docker.sock  # 将物理节点的docker.sock挂在到jenkins里,这样jenkins就可以通过socket和物理节点的docker通信了           type: ''      containers:      - name: jenkins        image: harbor.od.com/infra/jenkins:v2.303.1        imagePullPolicy: IfNotPresent # 镜像拉去策略 awlays 不管本地有没有,都从远程拉,never只从本地拉,IfNotPresent 优先本地,本地没有去远程拉        ports:        - containerPort: 8080           protocol: TCP        env:        - name: JAVA_OPTS          value: -Xmx512m -Xms512m        volumeMounts:        - name: data          mountPath: /var/jenkins_home # 将host200:/data/nfs-volume/jenkins_home挂在到容器里的/var/jenkins_home        - name: docker          mountPath: /run/docker.sock # 将物理主机的/run/docker.sock挂在到容器的/run/docker.sock      imagePullSecrets:      - name: harbor # 镜像仓库 之前用kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=xxxxx -n infra 定义了      securityContext:         runAsUser: 0  strategy:    type: RollingUpdate # 滚动升级    rollingUpdate:       maxUnavailable: 1      maxSurge: 1  revisionHistoryLimit: 7 # 保留7份滚动版本 ,以便回退  progressDeadlineSeconds: 600 # 启动超时时间

3)创建svc.yaml

kind: ServiceapiVersion: v1metadata:   name: jenkins  namespace: infraspec:  ports:  - protocol: TCP    port: 80 # cluterip的80端口转换到容器的8080    targetPort: 8080  selector:    app: jenkins

4)ingress.yaml

kind: IngressapiVersion: extensions/v1beta1metadata:   name: jenkins  namespace: infraspec:  rules:  - host: jenkins.od.com    http:      paths:      - path: /        backend:           serviceName: jenkins          servicePort: 80

​05 将jenkins交付到kubernetes集群​

1)任意一个node创建资源

kubectl create -f http://k8s-yaml.od.com/jenkins/dp.yamlkubectl create -f http://k8s-yaml.od.com/jenkins/svc.yamlkubectl create -f http://k8s-yaml.od.com/jenkins/ingress.yaml

2)查看资源情况

[root@host21 ~]# kubectl get all -n infraNAME                           READY   STATUS    RESTARTS   AGEpod/jenkins-7df8bf4ff9-qj85m   1/1     Running   0          10hNAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGEservice/jenkins   ClusterIP   10.254.129.207   <none>        80/TCP    13hNAME                      READY   UP-TO-DATE   AVAILABLE   AGEdeployment.apps/jenkins   1/1     1            1           10hNAME                                 DESIRED   CURRENT   READY   AGEreplicaset.apps/jenkins-7df8bf4ff9   1         1         1       10h

​06 验证jenkins​

1)验证jenkins镜像的用户

[root@host21 ~]# kubectl exec -it pod/jenkins-7df8bf4ff9-qj85m -n infra /bin/bashroot@jenkins-7df8bf4ff9-qj85m:/# whoamiroot

2)验证是否可以登录git

root@jenkins-7df8bf4ff9-qj85m:/# ssh -i /root/.ssh/id_rsa -T git@gitee.comHi yunweishaonian! You've successfully authenticated, but GITEE.COM does not provide shell access.

3)验证jenkins是否可以联动host的docker

root@jenkins-7df8bf4ff9-qj85m:/# docker image lsREPOSITORY                                  TAG                  IMAGE ID       CREATED             SIZEharbor.od.com/app/dubbo-demo-service        master_211002_2213   a8f3cec136fc   About an hour ago   377MB

​​07 配置jenkins​​

1)在host11添加解析

# 添加解析vi /var/named/od.com.zonejenkins  A 192.168.122.10# 重启dnssystemctl restart named

2)打开网页,查看提示的文件,将密码输入

[kubernetes] 交付dubbo之jenkins联动docker_json_03

3)选推荐安装的插件,安装完成后创建admin用户即可

[kubernetes] 交付dubbo之jenkins联动docker_git_04

4)进入jenkins

[kubernetes] 交付dubbo之jenkins联动docker_git_05

5)修改安全参数

[kubernetes] 交付dubbo之jenkins联动docker_json_06[kubernetes] 交付dubbo之jenkins联动docker_git_07

6)下载blue ocean插件

[kubernetes] 交付dubbo之jenkins联动docker_git_08

7)在[可选插件]中搜索blue ocean 选择下面的安装后重启

[kubernetes] 交付dubbo之jenkins联动docker_docker_09

8)安装完成后选择restart

[kubernetes] 交付dubbo之jenkins联动docker_json_10

9)一切就绪后,你就可以在首页看到blue ocean了

[kubernetes] 交付dubbo之jenkins联动docker_docker_11