devbox
IT小白
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

K8S中Prometheus+Grafana监控

作者：IT小白 | 2024-07-26 20:29:19

赞

踩

K8S中Prometheus+Grafana监控

1.介绍

        phometheus:当前一套非常流行的开源监控和报警系统。

        运行原理：通过HTTP协议周期性抓取被监控组件的状态。输出被监控组件信息的HTTP接口称为exporter。

        常用组件大部分都有exporter可以直接使用，比如haproxy,nginx，Mysql,Linux系统信息（包括磁盘、内存、CPU、网络等待）。

        prometheus主要特点：

  • 一个多维数据模型（时间序列由metrics指标名字和设置key/value键/值的labels构成）。
  • 非常高效的存储，平均一个采样数据占~3.5字节左右，320万的时间序列，每30秒采样，保持60天，消耗磁盘大概228G。
  • 一种灵活的查询语言（PromQL）。
  • 无依赖存储，支持local和remote不同模型。
  • 采用http协议，使用pull模式，拉取数据。
  • 监控目标，可以采用服务器发现或静态配置的方式。
  • 多种模式的图像和仪表板支持，图形化友好。
  • 通过中间网关支持推送时间。

        Grafana:是一个用于可视化大型测量数据的开源系统，可以对Prometheus 的指标数据进行可视化。

        Prometheus的体系结构图：

        Prometheus直接或通过中间推送网关从已检测的作业中删除指标，以处理短暂的作业。它在本地存储所有报废的样本，并对这些数据运行规则，以汇总和记录现有数据中的新时间序列，或生成警报。Grafana或其他API使用者可以用来可视化收集的数据。

2.部署prometheus

        2.1 使用RBAC进行授权

  1. [root@k8s-node01 k8s-prometheus]# cat prometheus-rbac.yaml 
  2. apiVersion: v1
  3. kind: ServiceAccount
  4. metadata:
  5.   name: prometheus
  6.   namespace: kube-system
  7.   labels:
  8.     kubernetes.io/cluster-service: "true"
  9.     addonmanager.kubernetes.io/mode: Reconcile
  10. ---
  11. apiVersion: rbac.authorization.k8s.io/v1
  12. kind: ClusterRole
  13. metadata:
  14.   name: prometheus
  15.   labels:
  16.     kubernetes.io/cluster-service: "true"
  17.     addonmanager.kubernetes.io/mode: Reconcile 
  18. rules:
  19.   - apiGroups:
  20.       - ""
  21.     resources:
  22.       - nodes
  23.       - nodes/metrics
  24.       - services
  25.       - endpoints
  26.       - pods
  27.     verbs:
  28.       - get
  29.       - list
  30.       - watch
  31.   - apiGroups:
  32.       - ""
  33.     resources:
  34.       - configmaps
  35.     verbs:
  36.       - get
  37.   - nonResourceURLs:
  38.       - "/metrics"
  39.     verbs:
  40.       - get
  41. ---
  42. apiVersion: rbac.authorization.k8s.io/v1
  43. kind: ClusterRoleBinding
  44. metadata:
  45.   name: prometheus
  46.   labels:
  47.     kubernetes.io/cluster-service: "true"
  48.     addonmanager.kubernetes.io/mode: Reconcile
  49. roleRef:
  50.   apiGroup: rbac.authorization.k8s.io
  51.   kind: ClusterRole
  52.   name: prometheus
  53. subjects:
  54. - kind: ServiceAccount
  55.   name: prometheus
  56.   namespace: kube-system
  57.  
  58.  
  59.  
  60. [root@k8s-node01 k8s-prometheus]# kubectl apply -f prometheus-rbac.yaml 
  61. serviceaccount/prometheus created
  62. clusterrole.rbac.authorization.k8s.io/prometheus created
  63. clusterrolebinding.rbac.authorization.k8s.io/prometheus created
  64.

        2.2 配置管理

        使用Configmap保存不需要加密配置信息，yaml中修改对应的NODE IP即可。

  1. [root@k8s-node01 k8s-prometheus]# cat prometheus-configmap.yaml 
  2. # Prometheus configuration format https://prometheus.io/docs/prometheus/latest/configuration/configuration/
  3. apiVersion: v1
  4. kind: ConfigMap
  5. metadata:
  6.   name: prometheus-config
  7.   namespace: kube-system 
  8.   labels:
  9.     kubernetes.io/cluster-service: "true"
  10.     addonmanager.kubernetes.io/mode: EnsureExists
  11. data:
  12.   prometheus.yml: |
  13.     rule_files:
  14.     - /etc/config/rules/*.rules
  15.  
  16.     scrape_configs:
  17.     - job_name: prometheus
  18.       static_configs:
  19.       - targets:
  20.         - localhost:9090
  21.  
  22.     - job_name: kubernetes-nodes
  23.       scrape_interval: 30s
  24.       static_configs:
  25.       - targets:
  26.         - 11.0.1.13:9100
  27.         - 11.0.1.14:9100
  28.  
  29.     - job_name: kubernetes-apiservers
  30.       kubernetes_sd_configs:
  31.       - role: endpoints
  32.       relabel_configs:
  33.       - action: keep
  34.         regex: default;kubernetes;https
  35.         source_labels:
  36.         - __meta_kubernetes_namespace
  37.         - __meta_kubernetes_service_name
  38.         - __meta_kubernetes_endpoint_port_name
  39.       scheme: https
  40.       tls_config:
  41.         ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
  42.         insecure_skip_verify: true
  43.       bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  44.  
  45.     - job_name: kubernetes-nodes-kubelet
  46.       kubernetes_sd_configs:
  47.       - role: node
  48.       relabel_configs:
  49.       - action: labelmap
  50.         regex: __meta_kubernetes_node_label_(.+)
  51.       scheme: https
  52.       tls_config:
  53.         ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
  54.         insecure_skip_verify: true
  55.       bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  56.  
  57.     - job_name: kubernetes-nodes-cadvisor
  58.       kubernetes_sd_configs:
  59.       - role: node
  60.       relabel_configs:
  61.       - action: labelmap
  62.         regex: __meta_kubernetes_node_label_(.+)
  63.       - target_label: __metrics_path__
  64.         replacement: /metrics/cadvisor
  65.       scheme: https
  66.       tls_config:
  67.         ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
  68.         insecure_skip_verify: true
  69.       bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  70.  
  71.     - job_name: kubernetes-service-endpoints
  72.       kubernetes_sd_configs:
  73.       - role: endpoints
  74.       relabel_configs:
  75.       - action: keep
  76.         regex: true
  77.         source_labels:
  78.         - __meta_kubernetes_service_annotation_prometheus_io_scrape
  79.       - action: replace
  80.         regex: (https?)
  81.         source_labels:
  82.         - __meta_kubernetes_service_annotation_prometheus_io_scheme
  83.         target_label: __scheme__
  84.       - action: replace
  85.         regex: (.+)
  86.         source_labels:
  87.         - __meta_kubernetes_service_annotation_prometheus_io_path
  88.         target_label: __metrics_path__
  89.       - action: replace
  90.         regex: ([^:]+)(?::\d+)?;(\d+)
  91.         replacement: $1:$2
  92.         source_labels:
  93.         - __address__
  94.         - __meta_kubernetes_service_annotation_prometheus_io_port
  95.         target_label: __address__
  96.       - action: labelmap
  97.         regex: __meta_kubernetes_service_label_(.+)
  98.       - action: replace
  99.         source_labels:
  100.         - __meta_kubernetes_namespace
  101.         target_label: kubernetes_namespace
  102.       - action: replace
  103.         source_labels:
  104.         - __meta_kubernetes_service_name
  105.         target_label: kubernetes_name
  106.  
  107.     - job_name: kubernetes-services
  108.       kubernetes_sd_configs:
  109.       - role: service
  110.       metrics_path: /probe
  111.       params:
  112.         module:
  113.         - http_2xx
  114.       relabel_configs:
  115.       - action: keep
  116.         regex: true
  117.         source_labels:
  118.         - __meta_kubernetes_service_annotation_prometheus_io_probe
  119.       - source_labels:
  120.         - __address__
  121.         target_label: __param_target
  122.       - replacement: blackbox
  123.         target_label: __address__
  124.       - source_labels:
  125.         - __param_target
  126.         target_label: instance
  127.       - action: labelmap
  128.         regex: __meta_kubernetes_service_label_(.+)
  129.       - source_labels:
  130.         - __meta_kubernetes_namespace
  131.         target_label: kubernetes_namespace
  132.       - source_labels:
  133.         - __meta_kubernetes_service_name
  134.         target_label: kubernetes_name
  135.  
  136.     - job_name: kubernetes-pods
  137.       kubernetes_sd_configs:
  138.       - role: pod
  139.       relabel_configs:
  140.       - action: keep
  141.         regex: true
  142.         source_labels:
  143.         - __meta_kubernetes_pod_annotation_prometheus_io_scrape
  144.       - action: replace
  145.         regex: (.+)
  146.         source_labels:
  147.         - __meta_kubernetes_pod_annotation_prometheus_io_path
  148.         target_label: __metrics_path__
  149.       - action: replace
  150.         regex: ([^:]+)(?::\d+)?;(\d+)
  151.         replacement: $1:$2
  152.         source_labels:
  153.         - __address__
  154.         - __meta_kubernetes_pod_annotation_prometheus_io_port
  155.         target_label: __address__
  156.       - action: labelmap
  157.         regex: __meta_kubernetes_pod_label_(.+)
  158.       - action: replace
  159.         source_labels:
  160.         - __meta_kubernetes_namespace
  161.         target_label: kubernetes_namespace
  162.       - action: replace
  163.         source_labels:
  164.         - __meta_kubernetes_pod_name
  165.         target_label: kubernetes_pod_name
  166.     alerting:
  167.       alertmanagers:
  168.       - static_configs:
  169.           - targets: ["alertmanager:80"]
  170.  
  171. [root@k8s-node01 k8s-prometheus]# kubectl apply -f prometheus-configmap.yaml 
  172. configmap/prometheus-config created
  173.

        2.3 有状态部署prometheus

        这里使用storageclass进行动态供给，给prometheus的数据进行持久化

  1. [root@k8s-node01 k8s-prometheus]# cat prometheus-statefulset.yaml 
  2. apiVersion: apps/v1
  3. kind: StatefulSet
  4. metadata:
  5.   name: prometheus 
  6.   namespace: kube-system
  7.   labels:
  8.     k8s-app: prometheus
  9.     kubernetes.io/cluster-service: "true"
  10.     addonmanager.kubernetes.io/mode: Reconcile
  11.     version: v2.2.1
  12. spec:
  13.   serviceName: "prometheus"
  14.   replicas: 1
  15.   podManagementPolicy: "Parallel"
  16.   updateStrategy:
  17.    type: "RollingUpdate"
  18.   selector:
  19.     matchLabels:
  20.       k8s-app: prometheus
  21.   template:
  22.     metadata:
  23.       labels:
  24.         k8s-app: prometheus
  25.       annotations:
  26.         scheduler.alpha.kubernetes.io/critical-pod: ''
  27.     spec:
  28.       priorityClassName: system-cluster-critical
  29.       serviceAccountName: prometheus
  30.       initContainers:
  31.       - name: "init-chown-data"
  32.         image: "busybox:latest"
  33.         imagePullPolicy: "IfNotPresent"
  34.         command: ["chown", "-R", "65534:65534", "/data"]
  35.         volumeMounts:
  36.         - name: prometheus-data
  37.           mountPath: /data
  38.           subPath: ""
  39.       containers:
  40.         - name: prometheus-server-configmap-reload
  41.           image: "jimmidyson/configmap-reload:v0.1"
  42.           imagePullPolicy: "IfNotPresent"
  43.           args:
  44.             - --volume-dir=/etc/config
  45.             - --webhook-url=http://localhost:9090/-/reload
  46.           volumeMounts:
  47.             - name: config-volume
  48.               mountPath: /etc/config
  49.               readOnly: true
  50.           resources:
  51.             limits:
  52.               cpu: 10m
  53.               memory: 10Mi
  54.             requests:
  55.               cpu: 10m
  56.               memory: 10Mi
  57.  
  58.         - name: prometheus-server
  59.           image: "prom/prometheus:v2.2.1"
  60.           imagePullPolicy: "IfNotPresent"
  61.           args:
  62.             - --config.file=/etc/config/prometheus.yml
  63.             - --storage.tsdb.path=/data
  64.             - --web.console.libraries=/etc/prometheus/console_libraries
  65.             - --web.console.templates=/etc/prometheus/consoles
  66.             - --web.enable-lifecycle
  67.           ports:
  68.             - containerPort: 9090
  69.           readinessProbe:
  70.             httpGet:
  71.               path: /-/ready
  72.               port: 9090
  73.             initialDelaySeconds: 30
  74.             timeoutSeconds: 30
  75.           livenessProbe:
  76.             httpGet:
  77.               path: /-/healthy
  78.               port: 9090
  79.             initialDelaySeconds: 30
  80.             timeoutSeconds: 30
  81.           # based on 10 running nodes with 30 pods each
  82.           resources:
  83.             limits:
  84.               cpu: 200m
  85.               memory: 1000Mi
  86.             requests:
  87.               cpu: 200m
  88.               memory: 1000Mi
  89.             
  90.           volumeMounts:
  91.             - name: config-volume
  92.               mountPath: /etc/config
  93.             - name: prometheus-data
  94.               mountPath: /data
  95.               subPath: ""
  96.             - name: prometheus-rules
  97.               mountPath: /etc/config/rules
  98.  
  99.       terminationGracePeriodSeconds: 300
  100.       volumes:
  101.         - name: config-volume
  102.           configMap:
  103.             name: prometheus-config
  104.         - name: prometheus-rules
  105.           configMap:
  106.             name: prometheus-rules
  107.  
  108.   volumeClaimTemplates:
  109.   - metadata:
  110.       name: prometheus-data
  111.     spec:
  112.       storageClassName: managed-nfs-storage 
  113.       accessModes:
  114.         - ReadWriteOnce
  115.       resources:
  116.         requests:
  117.           storage: "1Gi"
  118.  
  119. [root@k8s-node01 k8s-prometheus]# kubectl apply -f prometheus-statefulset.yaml 
  120. Warning: spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead
  121. statefulset.apps/prometheus created
  122.  
  123. [root@k8s-node01 k8s-prometheus]#kubectl get pod -n kube-system |grep prometheus
  124. NAME                                    READY   STATUS    RESTARTS   AGE
  125. prometheus-0                            2/2     Running   6          1d
  126.

        2.4 创建service暴露访问端口

  1. [root@k8s-node01 k8s-prometheus]# cat prometheus-service.yaml 
  2. kind: Service
  3. apiVersion: v1
  4. metadata: 
  5.   name: prometheus
  6.   namespace: kube-system
  7.   labels: 
  8.     kubernetes.io/name: "Prometheus"
  9.     kubernetes.io/cluster-service: "true"
  10.     addonmanager.kubernetes.io/mode: Reconcile
  11. spec: 
  12.   type: NodePort
  13.   ports: 
  14.     - name: http 
  15.       port: 9090
  16.       protocol: TCP
  17.       targetPort: 9090
  18.       nodePort: 30090
  19.   selector: 
  20.     k8s-app: prometheus
  21.  
  22. [root@k8s-master prometheus-k8s]# kubectl apply -f prometheus-service.yaml

        2.5 web访问

        使用任意一个NodeIP加端口进行访问，访问地址：http://NodeIP:Port

3.部署Grafana

        

  1. [root@k8s-master prometheus-k8s]# cat grafana.yaml
  2. apiVersion: apps/v1
  3. kind: StatefulSet
  4. metadata:
  5.   name: grafana
  6.   namespace: kube-system
  7. spec:
  8.   serviceName: "grafana"
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: grafana
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: grafana
  17.     spec:
  18.       containers:
  19.       - name: grafana
  20.         image: grafana/grafana
  21.         ports:
  22.           - containerPort: 3000
  23.             protocol: TCP
  24.         resources:
  25.           limits:
  26.             cpu: 100m
  27.             memory: 256Mi
  28.           requests:
  29.             cpu: 100m
  30.             memory: 256Mi
  31.         volumeMounts:
  32.           - name: grafana-data
  33.             mountPath: /var/lib/grafana
  34.             subPath: grafana
  35.       securityContext:
  36.         fsGroup: 472
  37.         runAsUser: 472
  38.   volumeClaimTemplates:
  39.   - metadata:
  40.       name: grafana-data
  41.     spec:
  42.       storageClassName: managed-nfs-storage #和prometheus使用同一个存储类
  43.       accessModes:
  44.         - ReadWriteOnce
  45.       resources:
  46.         requests:
  47.           storage: "1Gi"
  48.  
  49. ---
  50.  
  51. apiVersion: v1
  52. kind: Service
  53. metadata:
  54.   name: grafana
  55.   namespace: kube-system
  56. spec:
  57.   type: NodePort
  58.   ports:
  59.   - port : 80
  60.     targetPort: 3000
  61.     nodePort: 30091
  62.   selector:
  63.     app: grafana
  64.  
  65. [root@k8s-master prometheus-k8s]#kubectl apply -f grafana.yaml

访问方式：

使用任意一个NodeIP加端口进行访问，访问地址：http://NodeIP:Port ，默认账号密码为admin

4.监控K8S集群中Pod、Node、资源对象数据的方法

Pod：
kubelet的节点使用cAdvisor提供的metrics接口获取该节点所有Pod和容器相关的性能指标数据，安装kubelet默认就开启了

Node：

需要使用node_exporter收集器采集节点资源利用率。

使用node_exporter.sh脚本分别在所有服务器上部署node_exporter收集器，不需要修改可直接运行脚本

  1. [root@k8s-master prometheus-k8s]# cat node_exporter.sh 
  2. #!/bin/bashwget https://github.com/prometheus/node_exporter/releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz
  3.  
  4. tar zxf node_exporter-0.17.0.linux-amd64.tar.gz
  5. mv node_exporter-0.17.0.linux-amd64 /usr/local/node_exporter
  6.  
  7. cat <<EOF >/usr/lib/systemd/system/node_exporter.service
  8. [Unit]
  9. Description=https://prometheus.io
  10.  
  11. [Service]
  12. Restart=on-failure
  13. ExecStart=/usr/local/node_exporter/node_exporter --collector.systemd --collector.systemd.unit-whitelist=(docker|kubelet|kube-proxy|flanneld).service
  14.  
  15. [Install]
  16. WantedBy=multi-user.target
  17. EOF
  18.  
  19. systemctl daemon-reload
  20. systemctl enable node_exporter
  21. systemctl restart node_exporter
  22. [root@k8s-master prometheus-k8s]# ./node_exporter.sh
  23.  
  24. [root@k8s-master prometheus-k8s]# ps -ef|grep node_exporter
  25. root       6227      1  0 Oct08 ?        00:06:43 /usr/local/node_exporter/node_exporter --collector.systemd --collector.systemd.unit-whitelist=(docker|kubelet|kube-proxy|flanneld).service
  26. root     118269 117584  0 23:27 pts/0    00:00:00 grep --color=auto node_exporter

资源对象：

kube-state-metrics采集了k8s中各种资源对象的状态信息，只需要在master节点部署就行

        1.创建rbac的yaml对metrics进行授权

  1. [root@k8s-master prometheus-k8s]# cat kube-state-metrics-rbac.yaml
  2. apiVersion: v1
  3. kind: ServiceAccount
  4. metadata:
  5.   name: kube-state-metrics
  6.   namespace: kube-system
  7.   labels:
  8.     kubernetes.io/cluster-service: "true"
  9.     addonmanager.kubernetes.io/mode: Reconcile
  10. ---
  11. apiVersion: rbac.authorization.k8s.io/v1
  12. kind: ClusterRole
  13. metadata:
  14.   name: kube-state-metrics
  15.   labels:
  16.     kubernetes.io/cluster-service: "true"
  17.     addonmanager.kubernetes.io/mode: Reconcile
  18. rules:
  19. - apiGroups: [""]
  20.   resources:
  21.   - configmaps
  22.   - secrets
  23.   - nodes
  24.   - pods
  25.   - services
  26.   - resourcequotas
  27.   - replicationcontrollers
  28.   - limitranges
  29.   - persistentvolumeclaims
  30.   - persistentvolumes
  31.   - namespaces
  32.   - endpoints
  33.   verbs: ["list", "watch"]
  34. - apiGroups: ["extensions"]
  35.   resources:
  36.   - daemonsets
  37.   - deployments
  38.   - replicasets
  39.   verbs: ["list", "watch"]
  40. - apiGroups: ["apps"]
  41.   resources:
  42.   - statefulsets
  43.   verbs: ["list", "watch"]
  44. - apiGroups: ["batch"]
  45.   resources:
  46.   - cronjobs
  47.   - jobs
  48.   verbs: ["list", "watch"]
  49. - apiGroups: ["autoscaling"]
  50.   resources:
  51.   - horizontalpodautoscalers
  52.   verbs: ["list", "watch"]
  53. ---
  54. apiVersion: rbac.authorization.k8s.io/v1
  55. kind: Role
  56. metadata:
  57.   name: kube-state-metrics-resizer
  58.   namespace: kube-system
  59.   labels:
  60.     kubernetes.io/cluster-service: "true"
  61.     addonmanager.kubernetes.io/mode: Reconcile
  62. rules:
  63. - apiGroups: [""]
  64.   resources:
  65.   - pods
  66.   verbs: ["get"]
  67. - apiGroups: ["extensions"]
  68.   resources:
  69.   - deployments
  70.   resourceNames: ["kube-state-metrics"]
  71.   verbs: ["get", "update"]
  72. ---
  73. apiVersion: rbac.authorization.k8s.io/v1
  74. kind: ClusterRoleBinding
  75. metadata:
  76.   name: kube-state-metrics
  77.   labels:
  78.     kubernetes.io/cluster-service: "true"
  79.     addonmanager.kubernetes.io/mode: Reconcile
  80. roleRef:
  81.   apiGroup: rbac.authorization.k8s.io
  82.   kind: ClusterRole
  83.   name: kube-state-metrics
  84. subjects:
  85. - kind: ServiceAccount
  86.   name: kube-state-metrics
  87.   namespace: kube-system
  88. ---
  89. apiVersion: rbac.authorization.k8s.io/v1
  90. kind: RoleBinding
  91. metadata:
  92.   name: kube-state-metrics
  93.   namespace: kube-system
  94.   labels:
  95.     kubernetes.io/cluster-service: "true"
  96.     addonmanager.kubernetes.io/mode: Reconcile
  97. roleRef:
  98.   apiGroup: rbac.authorization.k8s.io
  99.   kind: Role
  100.   name: kube-state-metrics-resizer
  101. subjects:
  102. - kind: ServiceAccount
  103.   name: kube-state-metrics
  104.   namespace: kube-system
  105. [root@k8s-master prometheus-k8s]# kubectl apply -f kube-state-metrics-rbac.yaml

        2.编写Deployment和ConfigMap的yaml进行metrics pod部署，不需要进行修改

  1. [root@k8s-master prometheus-k8s]# cat kube-state-metrics-deployment.yaml
  2. apiVersion: apps/v1
  3. kind: Deployment
  4. metadata:
  5.   name: kube-state-metrics
  6.   namespace: kube-system
  7.   labels:
  8.     k8s-app: kube-state-metrics
  9.     kubernetes.io/cluster-service: "true"
  10.     addonmanager.kubernetes.io/mode: Reconcile
  11.     version: v1.3.0
  12. spec:
  13.   selector:
  14.     matchLabels:
  15.       k8s-app: kube-state-metrics
  16.       version: v1.3.0
  17.   replicas: 1
  18.   template:
  19.     metadata:
  20.       labels:
  21.         k8s-app: kube-state-metrics
  22.         version: v1.3.0
  23.       annotations:
  24.         scheduler.alpha.kubernetes.io/critical-pod: ''
  25.     spec:
  26.       priorityClassName: system-cluster-critical
  27.       serviceAccountName: kube-state-metrics
  28.       containers:
  29.       - name: kube-state-metrics
  30.         image: lizhenliang/kube-state-metrics:v1.3.0
  31.         ports:
  32.         - name: http-metrics
  33.           containerPort: 8080
  34.         - name: telemetry
  35.           containerPort: 8081
  36.         readinessProbe:
  37.           httpGet:
  38.             path: /healthz
  39.             port: 8080
  40.           initialDelaySeconds: 5
  41.           timeoutSeconds: 5
  42.       - name: addon-resizer
  43.         image: lizhenliang/addon-resizer:1.8.3
  44.         resources:
  45.           limits:
  46.             cpu: 100m
  47.             memory: 30Mi
  48.           requests:
  49.             cpu: 100m
  50.             memory: 30Mi
  51.         env:
  52.           - name: MY_POD_NAME
  53.             valueFrom:
  54.               fieldRef:
  55.                 fieldPath: metadata.name
  56.           - name: MY_POD_NAMESPACE
  57.             valueFrom:
  58.               fieldRef:
  59.                 fieldPath: metadata.namespace
  60.         volumeMounts:
  61.           - name: config-volume
  62.             mountPath: /etc/config
  63.         command:
  64.           - /pod_nanny
  65.           - --config-dir=/etc/config
  66.           - --container=kube-state-metrics
  67.           - --cpu=100m
  68.           - --extra-cpu=1m
  69.           - --memory=100Mi
  70.           - --extra-memory=2Mi
  71.           - --threshold=5
  72.           - --deployment=kube-state-metrics
  73.       volumes:
  74.         - name: config-volume
  75.           configMap:
  76.             name: kube-state-metrics-config
  77. ---
  78. # Config map for resource configuration.
  79. apiVersion: v1
  80. kind: ConfigMap
  81. metadata:
  82.   name: kube-state-metrics-config
  83.   namespace: kube-system
  84.   labels:
  85.     k8s-app: kube-state-metrics
  86.     kubernetes.io/cluster-service: "true"
  87.     addonmanager.kubernetes.io/mode: Reconcile
  88. data:
  89.   NannyConfiguration: |-
  90.     apiVersion: nannyconfig/v1alpha1
  91.     kind: NannyConfiguration
  92. [root@k8s-master prometheus-k8s]# kubectl apply -f kube-state-metrics-deployment.yaml

        3.编写Service的yaml对metrics进行端口暴露

  1. [root@k8s-master prometheus-k8s]# cat kube-state-metrics-service.yaml
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5.   name: kube-state-metrics
  6.   namespace: kube-system
  7.   labels:
  8.     kubernetes.io/cluster-service: "true"
  9.     addonmanager.kubernetes.io/mode: Reconcile
  10.     kubernetes.io/name: "kube-state-metrics"
  11.   annotations:
  12.     prometheus.io/scrape: 'true'
  13. spec:
  14.   ports:
  15.   - name: http-metrics
  16.     port: 8080
  17.     targetPort: http-metrics
  18.     protocol: TCP
  19.   - name: telemetry
  20.     port: 8081
  21.     targetPort: telemetry
  22.     protocol: TCP
  23.   selector:
  24.     k8s-app: kube-state-metrics
  25. [root@k8s-master prometheus-k8s]# kubectl apply -f kube-state-metrics-service.yaml
  26.  
  27. [root@k8s-master prometheus-k8s]# kubectl get pod,svc -n kube-system
  28. NAME                                        READY   STATUS    RESTARTS   AGE
  29. pod/alertmanager-5d75d5688f-fmlq6           2/2     Running   0          9dpod/coredns-5bd5f9dbd9-wv45t                1/1     Running   1          9dpod/grafana-0                               1/1     Running   2          15dpod/kube-state-metrics-7c76bdbf68-kqqgd     2/2     Running   6          14dpod/kubernetes-dashboard-7d77666777-d5ng4   1/1     Running   5          16dpod/prometheus-0                            2/2     Running   6          15dNAME                           TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
  30. service/alertmanager           ClusterIP   10.0.0.207   <none>        80/TCP              13dservice/grafana                NodePort    10.0.0.74    <none>        80:30091/TCP        15dservice/kube-dns               ClusterIP   10.0.0.2     <none>        53/UDP,53/TCP       14dservice/kube-state-metrics     ClusterIP   10.0.0.194   <none>        8080/TCP,8081/TCP   14dservice/kubernetes-dashboard   NodePort    10.0.0.127   <none>        443:30001/TCP       17dservice/prometheus             NodePort    10.0.0.33    <none>        9090:30090/TCP      14d
报错一：进行2.1步骤时报错：ensure CRDs are installed first

  1. [root@k8s-node01 k8s-prometheus]# kubectl apply -f prometheus-rbac.yaml
  2. serviceaccount/prometheus unchanged
  3. resource mapping not found for name: "prometheus" namespace: "" from "prometheus-rbac.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
  4. ensure CRDs are installed first
  5. resource mapping not found for name: "prometheus" namespace: "" from "prometheus-rbac.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
  6. ensure CRDs are installed first

使用附件的原yaml会报错，原因是因为api过期，需要手动修改 apiVersion: rbac.authorization.k8s.io/v1beta1为apiVersion: rbac.authorization.k8s.io/v1

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/IT小白/article/detail/887084
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号