devbox
我家小花儿
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

k8s部署Elasticsearch集群+Kibana方案--开启X-Pack 安全认证_elasticsearch集群 nfs

作者：我家小花儿 | 2024-05-02 14:01:03

赞

踩

elasticsearch集群 nfs

前言

本文中使用StatefulSet 方式部署 Elasticsearch 集群，并且开启X-Pack 安全认证，存储使用的是NFS，属于一个初学者自己探索的方案，如果有比较好的方案，还请不吝评论赐教。
版本说明：

  • Kubernetes v1.25.6 – v1.26.4
  • Elasticsearch, Kibana 7.13.3
  • NFS Subdir External Provisioner

前置环境
需要安装好Kubernetes 集群，并且配置了存储类(StorageClass)。

一、部署Elasticsearch集群

1. 创建配置文件

elastic-worker-ns.yaml

apiVersion: v1           # 创建命名空间
kind: Namespace
metadata:
  labels:
    app: es7-cluster
    kubernetes.io/name: "Elasticsearch"
  name: elastic-worker

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

es7-cluster.yaml

---
apiVersion: v1             # 创建service 文件用于内部通讯
kind: Service
metadata:
  name: es7-headless
  namespace: elastic-worker
  labels:
    app: es7-cluster
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "Elasticsearch"
spec:
  clusterIP: None
  publishNotReadyAddresses: true
  ports:
  - name: rest-api
    port: 9200
    targetPort: 9200
  - name: inter-node
    port: 9300
    targetPort: 9300
  selector:
    app: es7-cluster
---
apiVersion: v1             
kind: ServiceAccount
metadata:
  name: es7-cluster
  namespace: elastic-worker
  labels:
    app: es7-cluster
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: es7-cluster
  namespace: elastic-worker
  labels:
    app: es7-cluster
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
  - ""
  resources:
  - "services"
  - "namespaces"
  - "endpoints"
  verbs:
  - "get"
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: es7-cluster
  namespace: elastic-worker
  labels:
    app: es7-cluster
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
subjects:
- kind: ServiceAccount
  name: es7-cluster
  namespace: kube-system
  apiGroup: ""
roleRef:
  kind: ClusterRole
  name: es7-cluster
  apiGroup: ""
---
apiVersion: apps/v1             # 创建有状态的服务
kind: StatefulSet
metadata:
  name: es7-cluster
  namespace: elastic-worker
  labels:
    app: es7-cluster
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    srv: srv-elasticsearch
spec:
  serviceName: es7-headless
  replicas: 3
  selector:
    matchLabels:
      app: es7-cluster
      kubernetes.io/cluster-service: "true"
  template:
    metadata:
      labels:
        app: es7-cluster
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccountName: es7-cluster
      containers:         # 主容器
      - name: elasticsearch
        image: registry.cn-hangzhou.aliyuncs.com/greatmap/elasticsearch:7.13.3-p12    # 自定义镜像，下边会详细说明
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9200
          name: rest-api
          protocol: TCP
        - containerPort: 9300
          name: inter-node
          protocol: TCP
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
        - name: localtime
          readOnly: true
          mountPath: /etc/localtime
        env:
        - name: cluster.name
          value: "es7-cluster"
        - name: node.name
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: discovery.zen.minimum_master_nodes
          value: "2"
        - name: discovery.seed_hosts
          value: "es7-headless"
        - name: cluster.initial_master_nodes
          value: "es7-cluster-0,es7-cluster-1,es7-cluster-2"
        - name: ES_JAVA_OPTS
          value: "-Xms1g -Xmx1g"
        - name: xpack.security.enabled    # X-Pack 安全认证
          value: "true"
        - name: xpack.security.transport.ssl.enabled
          value: "true"
        - name: xpack.security.transport.ssl.verification_mode  # 证书校验类型
          value: "certificate"
        - name: xpack.security.transport.ssl.keystore.path    # 证书路径
          value: "elastic-certificates.p12"
        - name: xpack.security.transport.ssl.truststore.path
          value: "elastic-certificates.p12"
        - name: xpack.monitoring.ui.container.elasticsearch.enabled   # 生成并提供与容器相关的监控数据，待验证
          value: "true"
        #- name: reindex.remote.whitelist        # 设置同步白名单，可以用来数据迁移
        #  value: "192.168.10.13:9200"
      initContainers:        # 初始化容器
      - name: fix-permissions
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
        securityContext:
          privileged: true
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
        - name: localtime
          readOnly: true
          mountPath: /etc/localtime
      - name: increase-vm-max-map
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        securityContext:
          privileged: true
      - name: increase-fd-ulimit
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ["sh", "-c", "ulimit -n 65536"]
      volumes:
      - name: localtime
        hostPath:
          path: /etc/localtime
          type: ''
  volumeClaimTemplates:   
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteMany" ]
      # 使用的存储类名称，需要配置一个有效的存储类
      storageClassName: "managed-nfs-storage"
      resources:
        requests:
          storage: 2Gi

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68
  • 69
  • 70
  • 71
  • 72
  • 73
  • 74
  • 75
  • 76
  • 77
  • 78
  • 79
  • 80
  • 81
  • 82
  • 83
  • 84
  • 85
  • 86
  • 87
  • 88
  • 89
  • 90
  • 91
  • 92
  • 93
  • 94
  • 95
  • 96
  • 97
  • 98
  • 99
  • 100
  • 101
  • 102
  • 103
  • 104
  • 105
  • 106
  • 107
  • 108
  • 109
  • 110
  • 111
  • 112
  • 113
  • 114
  • 115
  • 116
  • 117
  • 118
  • 119
  • 120
  • 121
  • 122
  • 123
  • 124
  • 125
  • 126
  • 127
  • 128
  • 129
  • 130
  • 131
  • 132
  • 133
  • 134
  • 135
  • 136
  • 137
  • 138
  • 139
  • 140
  • 141
  • 142
  • 143
  • 144
  • 145
  • 146
  • 147
  • 148
  • 149
  • 150
  • 151
  • 152
  • 153
  • 154
  • 155
  • 156
  • 157
  • 158
  • 159
  • 160
  • 161
  • 162
  • 163
  • 164
  • 165
  • 166
  • 167
  • 168
  • 169
  • 170
  • 171
  • 172
  • 173
  • 174
  • 175
  • 176
  • 177
  • 178
  • 179
  • 180

2. 部署Elasticsearch集群

kubectl create -f es7-cluster.yaml

  • 1

3. 相关说明

  • 单节点模式只开启xpack.security.enabled 既可以，但是集群模式通信就需要用到证书；
  • 只开启xpack.security.enabled 在重新apply 时会失败，提示xpack.security.transport.ssl.enabled 为 false 请设置为 true；
  • 再开启xpack.security.transport.ssl.enabled 需要生成且配置证书路径，但是查找相关资料都是启动后生成证书，再启动容器；在k8s 中 pod 会无限重启，没办法生成证书；
  • 顾此自定义镜像将证书直接打到镜像里，使用Dockerfile进行构建
  • 证书校验类型：full、certificate、none；默认值为“ full ”；

如何生成证书：

# 使用docker 运行elasticsearch容器
docker run -it --rm elasticsearch:7.13.3  bash
# 生成证书
./bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""

  • 1
  • 2
  • 3
  • 4

将证书保存到本地，再通过Dockerfile重新封装镜像

FROM elasticsearch:7.13.3
LABEL maintainer="YZEQIANG <yinzeqiang66@126.com>"
COPY elastic-certificates.p12 /usr/share/elasticsearch/config/
RUN  chown 1000:0 /usr/share/elasticsearch/config/elastic-certificates.p12
EXPOSE 9200 9300
CMD ["eswrapper"]

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

阿里云仓库：(支持x86_64、arm64)

docker pull registry.cn-hangzhou.aliyuncs.com/greatmap/elasticsearch:7.13.3-p12

  • 1

4. 设置密码

进入到任意节点

kubectl exec -it -n elastic-worker pods/es7-cluster-1  -- bash

./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

# interactive 手动设置密码（Stack123），如果是auto，自动随机生成密码
Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana_system]: 
Reenter password for [kibana_system]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20

5. 将es集群对外暴露

es7-external.yaml

apiVersion: v1             # 创建service 文件用于内部通讯
kind: Service
metadata:
  name: es7-external
  namespace: elastic-worker
  labels:
    app: es7-cluster
spec:
  type: NodePort
  ports:
  - name: rest-api
    port: 9200
    protocol: TCP
    targetPort: 9200
    nodePort: 30920
  selector:
    app: es7-cluster

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17

6. 外部连接验证

使用的是edge扩展插件
https://microsoftedge.microsoft.com/addons/detail/elasticvue/geifniocjfnfilcbeloeidajlfmhdlgo?hl=zh-CN
es-节点

二、部署Kibana

1. 创建配置文件

kibana.yaml

apiVersion: v1
kind: Service
metadata:
  name: kibana
  namespace: elastic-worker
  labels:
    k8s-app: kibana
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "Kibana"
    srv: srv-kibana
spec:
  type: NodePort
  ports:
  - port: 5601
    nodePort: 30000
    protocol: TCP
    targetPort: ui
  selector:
    k8s-app: kibana
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana
  namespace: elastic-worker
  labels:
    k8s-app: kibana
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    srv: srv-kibana
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: kibana
  template:
    metadata:
      labels:
        k8s-app: kibana
    spec:
      containers:
      - name: kibana
        image: kibana:7.13.3
        volumeMounts:
        - name: kibana-config
          mountPath: /usr/share/kibana/config/kibana.yml
          readOnly: true
          subPath: kibana.yml
        ports:
        - containerPort: 5601
          name: ui
          protocol: TCP
      volumes:
      - name: kibana-config
        configMap:
          name: kibana
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana
  namespace: elastic-worker
  labels:
    k8s-app: kibana
data:
  kibana.yml: |-
  
    server.name: kibana
    server.host: "0.0.0.0"
    elasticsearch.hosts: [ "http://es7-headless:9200" ]
    elasticsearch.username: "elastic"
    elasticsearch.password: "Stack123"
    monitoring.ui.container.elasticsearch.enabled: true
    i18n.locale: "zh-CN"
    kibana.index: ".kibana"

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68
  • 69
  • 70
  • 71
  • 72
  • 73
  • 74
  • 75
  • 76

2. 部署Kibana

kubectl apply -f kibana.yaml

  • 1

3. 访问验证

http://ip:port
在这里插入图片描述

参考文档：

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/我家小花儿/article/detail/524439
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号