【从零搭建k8s集群并构建一个devops微服务，详细教程】_集群部署微服务

一张图了解DevOps 和 k8s

上面图片来自哔站，学习k8s

你好！

配置软件仓库

deb https://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse

deb https://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse

deb https://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse

# deb https://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
# deb-src https://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse

deb https://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
1
2
3
4
5
6
7
8
9
10
11
12
13
14

搭建一个k8s集群

结合docker使用时，k8s版本最好不要用1.24及以上版本，k8s从1.24版本开始不在直接兼容docker，需要安装cri-docker。

# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

# 关闭selinux，
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时

# 关闭swap
swapoff -a # 临时
sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久

# 关闭完swap后记得重启虚拟机！！！
# 根据规划设置主机名
hostname set-hostname <hostname>

# 在master中添加你的节点
cat >> /etc/hosts/ << EOF 
ip1 k8s-master 
ip2 k8s-node1 
EOF

# 将桥接的IPv4流量传递到iptables的链
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# 生效
sudo sysctl --system
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

初始化 k8s 集群

1、安装cri-docker 因为我的kubectl高于 1.24， 所以需要单独配置这个环境

wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.1/cri-dockerd_0.3.1.3-0.ubuntu-focal_amd64.deb
dpkg -i cri-dockerd_0.3.1.3-0.ubuntu-focal_amd64.deb
systemctl start cri-docker

1
2
3
4

2、安装kubectl kubeadm kube

# 安装依赖软件
apt-get update && sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-https curl

# 配置阿里源的key
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

# 添加k8s软件安装源
tee /etc/apt/sources.list.d/kubernetes.list <<EOF 
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

# 跟新软件源并安装kubelet kubeadm kubectl
apt-get update
apt-get install -y kubelet kubeadm kubectl

# 标记该软件包不被自动更新（ubuntu中需要标记）
apt-mark hold kubelet kubeadm kubectl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

3、初始化

kubeadm init --apiserver-advertise-address 172.16.133.1 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=SystemVerification

# 参数解析
--apiserver-advertise-address       主机地址
--image-repository                  配置国内的k8s镜像源
--pod-network-cidr                  配置pod容器网段
--ignore-preflight-errors           忽略初始化的一些错误

1
2
3
4
5
6
7
8

4、初始化报错， 多个cri

解决方法如下：指定其中一个，再初始化

kubeadm init --apiserver-advertise-address 172.16.133.1 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=SystemVerification --cri-socket unix:///var/run/cri-dockerd.sock
1

初始化ing，放一个图，紧张一下下

初始化又报错

解决方法一下，netstat -nlpt|grep :10250，kill it,再初始化
还是不行
执行

kubeadm reset --cri-socket unix:///var/run/cri-dockerd.sock
1

再初始化ing

还是一样的错，google之后，采用文件初始化的方式，yaml文件内容如下，参考大佬文章：

再执行下面的命令

# 安装官方生成了kubeadm config
$ kubeadm config print init-defaults --component-configs \
KubeProxyConfiguration,KubeletConfiguration > kubeadm-config.yaml

$ cat kubeadm-config.yaml

$ kubeadm init --config=kubeadm-config.yaml
1
2
3
4
5
6
7

好，又报错了

要改一改配置,放一个我的配置

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 101.200.76.174
  bindPort: 6443
nodeRegistration:
  #criSocket: unix:///var/run/containerd/containerd.sock
  criSocket: unix:///var/run/cri-dockerd.sock
  imagePullPolicy: IfNotPresent
  name: master
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.28.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podsubnet: 47.93.224.95/18
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
bindAddressHardFail: false
clientConnection:
  acceptContentTypes: ""
  burst: 0
  contentType: ""
  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
  qps: 0
clusterCIDR: ""
configSyncPeriod: 0s
conntrack:
  maxPerCore: null
  min: null
  tcpCloseWaitTimeout: null
  tcpEstablishedTimeout: null
detectLocal:
  bridgeInterface: ""
  interfaceNamePrefix: ""
detectLocalMode: ""
enableProfiling: false
healthzBindAddress: ""
hostnameOverride: ""
clientConnection:
  acceptContentTypes: ""
  burst: 0
  contentType: ""
  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
  qps: 0
clusterCIDR: ""
configSyncPeriod: 0s
conntrack:
  maxPerCore: null
  min: null
  tcpCloseWaitTimeout: null
  tcpEstablishedTimeout: null    
detectLocal:
  bridgeInterface: ""
  interfaceNamePrefix: ""
detectLocalMode: ""
enableProfiling: false
healthzBindAddress: ""
hostnameOverride: ""    json:
      infoBufferSize: "0"
  verbosity: 0
metricsBindAddress: ""
mode: ""
nodePortAddresses: null
oomScoreAdj: null
portRange: ""
showHiddenMetricsForVersion: ""
winkernel:
  enableDSR: false
  forwardHealthCheckVip: false
  networkName: ""
  rootHnsEndpointName: ""
  sourceVip: ""
---
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
      cacheAuthorizedTTL: 0s
    cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
containerRuntimeEndpoint: ""
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 0s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
logging:
  flushFrequency: 0
  options:
    json:
      infoBufferSize: "0"
  verbosity: 0
memorySwap: {}
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
volumeStatsAggPeriod: 0s
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

没整出来，暂停了，tmd啊啊啊啊
最后问了朋友好像说是cpu核不够，起底要4，然鹅。。。。。

1. CPU：至少2个CPU核心
2. 内存：至少4GB RAM
3. 网络：至少1Gbps网络连接
4. 存储：至少20GB的可用存储空间
   
   拜拜，后面有机会再补全吧，欢迎你们补充～～
   附赠一个成功的链接
   搭建k8s集群，你们可以参考继续操作

安装并配置 gitlab

我们对Markdown编辑器进行了一些功能拓展与语法支持，除了标准的Markdown编辑器功能，我们增加了如下几点新功能，帮助你用它写博客：
下载gitlab gitlab-ce
使用下方的命令，可以自动检测你的ubuntu版本。

curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
1

安装
把下面的EXTERNAL_URL换成你的主机名字就可以了，如果不知道，可以执行 hostname 就可以了

sudo EXTERNAL_URL="http://gitlab.linuxtechi.net" apt install gitlab-ce
1

安装过程如果遇到这个报错

执行以下命令，再执行上一条命令

$ gitlab-ctl reconfigure
$ sudo EXTERNAL_URL="http://gitlab.linuxtechi.net" apt install gitlab-ce
1
2

安装成功会看到以下界面

修改如下配置：

sudo vim /etc/gitlab/gitlab.rb
# 为了服务体积更小一点
external_url 'http://ip:port'
gitlab_rails['time_zone'] = 'Asia/Shanghai'
puma['worker_processes'] = 2
sidekiq['max_concurrency'] = 8
postgresql['shared_buffers'] = "128MB"
postgresql['max_worker_processes'] = 6
prometheus_monitoring['enable'] = false
1
2
3
4
5
6
7
8
9

重启gitlab

$ gitlab-ctl reconfigure
$ gitlab-ctl restart
1
2

上面执行成功后会出现下面这个界面，gitlab安装成功了

以root账号来登陆gitlab, 登陆链接 http://ip:28088/users/sign_in

登陆成功后进入这个页面

gitlab一些配置

1、关闭用户注册

2、开启webhook外部访问
settings-> Network->Outbound requests->Allow requests to the local network from webhooks and integrations-save changes

3、设置当前用户使用中文（可选） 设置两个地方
1 settings->Preferences->Localization->save changes
2 设置当前用户profile的默认语言 localization

创建文件夹 devops （待补充）

安装harbor

1、sh install.sh

2、配置secret

强调文本 强调文本

加粗文本 加粗文本

标记文本

删除文本

引用文本

H₂O is是液体。

2¹⁰ 运算结果是 1024.

安装SonarQube

1、
2、进入/opt/k8s/devops

kubectl apply -f sonarqube/
1

3、登陆到 sonarqube后台，点击头像-> MyAccount -> Security -> Generate Tokens -> generate 生成token 并复制

安装部署jekins