ElasticSearch高可用安装部署（Linux）

ElasticSearch高可用安装部署

一、小型的ElasticSearch集群的节点角色规划

• 对于Ingest节点，如果我们没有格式转换、类型转换等需求，直接设置为false。
• 3-5个节点属于轻量级集群，要保证主节点个数满足((节点数/2)+1)。
• 轻量级集群，节点的多重属性如：Master&Data设置为同一个节点可以理解的。
• 如果进一步优化，5节点可以将Master和Data再分离。

二、大型的ElasticSearch集群的节点架构

ES数据库最好的高可用集群部署架构为：

• 三台服务器做master节点
• N（比如20）台服务器作为data节点（存储资源要大）
• N（比如2）台做ingest节点（用于数据转换，可以提高ES查询效率）

三、高可用ElasticSearch的部署源规划

以适配智能搜索引擎为前提

注：因为智能搜索引擎不涉及到格式转换、类型转换等需求，所以仅部署一个Ingest节点的部署，如后期需要，可横向扩充

四、安装部署ElasticSearch

1.1.添加用户名及下载ElasticSearch 7.13.4安装包（通用）

# root用户执行
passwd elasticsearch
更改用户 elasticsearch 的密码 。
新的 密码：<该用户密码>
重新输入新的 密码：<该用户密码>
su - elasticsearch 
# elasticsearch用户执行
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.13.4-linux-x86_64.tar.gz
1
2
3
4
5
6
7
8

1.2.修改系统环境（通用）

1.2.1.修改系统参数配置

# root用户执行
cat >>/etc/security/limits.conf<<EOF
elasticsearch       soft    nofile  65536
elasticsearch       hard    nofile  65536
elasticsearch       soft    nproc   4096
elasticsearch       hard    nproc   4096
EOF

cat >>/etc/security/limits.d/20-nproc.conf<<EOF
elasticsearch soft nproc 4096
EOF

vim /etc/sysctl.conf
vm.max_map_count=655360
# 立即生效
sysctl -p
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

1.2.2.配置hostname

cat >>/etc/hosts<<EOF
192.168.62.55    es-master-1
192.168.62.56    es-master&data-1
192.168.62.57    es-master&data-2
192.168.62.58    es-data-1
192.168.30.7		es-data&ingest-1
192.168.30.6     es-client
EOF
1
2
3
4
5
6
7
8

12.3.配置elasticsearch用户免密登录

# es-master-1执行
ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/elasticsearch/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/elasticsearch/.ssh/id_rsa.
Your public key has been saved in /home/elasticsearch/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:dYVCatLs1ZoegeTp5IJh4cE9C1IAcfH66YnEXhtCZbg elasticsearch@node1
.....
ssh-copy-id "es-master-1"
ssh-copy-id "es-master&data-1"
ssh-copy-id "es-master&data-2"
ssh-copy-id "es-data-1"
ssh-copy-id "es-data&ingest-1"
ssh-copy-id "es-client"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

1.2.4.创建日志文件和数据文件

# 创建日志文件地址
mkdir  /opt/logs/elasticsearch -p
chown elasticsearch:elasticsearch /opt/logs/elasticsearch -R
# 创建数据存储目录（data节点）
mkdir /data/elasticsearch -p
chown elasticsearch:elasticsearch /data/elasticsearch -R
1
2
3
4
5
6

1.3.解压缩文件（通用）

su - elasticsearch
tar -xvf elasticsearch-7.13.4-linux-x86_64.tar.gz
1
2

1.4.独立安装master-1节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: es-master-1
cluster.name: elasticsearch-cluster
node.name: master-1
path.logs: /opt/logs/elasticsearch
discovery.zen.ping.unicast.hosts: ["es-master-1","es-master&data-1","es-master&data-2"]
discovery.zen.minimum_master_nodes: 2
node.master: true
node.ingest: false
node.data: false
cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
indices.breaker.total.limit: 80%
indices.fielddata.cache.size: 20%
indices.breaker.fielddata.limit: 60%
indices.breaker.request.limit: 60%

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

1.5.安装master&data-1节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: "es-master&data-1"
cluster.name: elasticsearch-cluster
node.name: "master&data-1"
path.logs: /opt/logs/elasticsearch
path.data: /data/elasticsearch
discovery.zen.ping.unicast.hosts: ["es-master-1","es-master&data-1","es-master&data-2"]
discovery.zen.minimum_master_nodes: 2
node.master: true
node.ingest: false
node.data: true
cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
indices.breaker.total.limit: 80%
indices.fielddata.cache.size: 20%
indices.breaker.fielddata.limit: 60%
indices.breaker.request.limit: 60%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

1.6.安装master&data-2节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: "es-master&data-2"
cluster.name: elasticsearch-cluster
node.name: "master&data-2"
path.logs: /opt/logs/elasticsearch
path.data: /data/elasticsearch
discovery.zen.ping.unicast.hosts: ["es-master-1","es-master&data-1","es-master&data-2"]
discovery.zen.minimum_master_nodes: 2
node.master: true
node.ingest: false
node.data: true
cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
indices.breaker.total.limit: 80%
indices.fielddata.cache.size: 20%
indices.breaker.fielddata.limit: 60%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

1.7.安装data-1节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: es-data-1
cluster.name: elasticsearch-cluster
node.name: data-1
path.logs: /opt/logs/elasticsearch
path.data: /data/elasticsearch
node.master: false
node.ingest: false
node.data: true
cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
discovery.seed_hosts: ["es-master-1","es-master&data-1","es-master&data-2"]
indices.breaker.total.limit: 80%
indices.fielddata.cache.size: 20%
indices.breaker.fielddata.limit: 60%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

1.8.安装ingest&data-1节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

  http.cors.enabled: true
  http.cors.allow-origin: "*"
  network.host: es-data&ingest-1
  cluster.name: elasticsearch-cluster
  node.name: data&ingest-1
  path.logs: /opt/logs/elasticsearch
  node.master: false
  node.ingest: true
  node.data: true
  cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
  discovery.seed_hosts: ["es-master-1","es-master&data-1","es-master&data-2"]
  1
  2
  3
  4
  5
  6
  7
  8
  9
  10
  11

1.9.安装client节点

• 修改elasticsearch配置文件elasticsearch-7.13.4/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: es-client
cluster.name: elasticsearch-cluster
node.name: Client
path.logs: /opt/logs/elasticsearch
node.master: false
node.ingest: false
node.data: false
cluster.initial_master_nodes: ["es-master-1","es-master&data-1","es-master&data-2"]
discovery.seed_hosts: ["es-master-1","es-master&data-1","es-master&data-2"]

1
2
3
4
5
6
7
8
9
10
11
12

1.10 启动各节点

cd elasticsearch-7.13.4/bin
nohup ./elasticsearch &
1
2

五、安装部署Kibana

1.1.下载安装包

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.13.4-linux-x86_64.tar.gz
# 下载时需要注意该软件版本必须和elasticsearch版本对应
# 解压该安装包
tar -xvf kibana-7.13.4-linux-x86_64.tar.gz
1
2
3
4

1.2.修改配置文件

cd kibana-7.13.4-linux-x86_64/config
vim config.yml

server.host: "<kibana宿主机IP>"
# client节点的地址
elasticsearch.hosts: ["http://es-client:9200"]
i18n.locale: "zh-CN"
1
2
3
4
5
6
7

1.3.启动kibana

cd kibana-7.13.4-linux-x86_64/
nohup ./kibana --allow-root

#访问以下地址
http://<kibana宿主机IP>:5601
1
2
3
4
5

六、配置ElasticSearch的用户名密码

1.1.生成证书

# 在es-master-1节点上
su - elasticsearch
cd elasticsearch-7.13.4
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
#在elasticsearch-7.13.4/config目录下生成elastic-certificates.p12
1
2
3
4
5

1.2.将证书拷贝到其他节点

scp -r elasticsearch-7.13.4/config/elastic-certificates.p12 "es-master&data-1":~/elasticsearch-7.13.4/config/elastic-certificates.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates.p12 "es-master&data-2":~/elasticsearch-7.13.4/config/elastic-certificates.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates.p12 es-data-1:~/elasticsearch-7.13.4/config/elastic-certificates.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates.p12 "es-data&ingest-1":~/elasticsearch-7.13.4/config/elastic-certificates.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates.p12 es-client:~/elasticsearch-7.13.4/config/elastic-certificates.p12

1
2
3
4
5
6

1.3.修改elasticsearch配置文件(各节点)

• elasticsearch-7.13.4/config/elasticsearch.yml

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
1
2
3
4
5

1.4.重启所有节点

kill -9 <elasticsearch启动进程>
cd elasticsearch-7.13.4/bin
nohup ./elasticsearch &
1
2
3

1.5.设定密码

# 在1.1生成证书的节点
elasticsearch-7.13.4/bin/elasticsearch-setup-passwords interactive
.........
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]: <对应用户的密码>
Reenter password for [elastic]: <对应用户的密码>
Enter password for [apm_system]: <对应用户的密码>
Reenter password for [apm_system]: <对应用户的密码>
Enter password for [kibana_system]: <对应用户的密码>
Reenter password for [kibana_system]: <对应用户的密码>
Enter password for [logstash_system]: <对应用户的密码>
Reenter password for [logstash_system]: <对应用户的密码>
Enter password for [beats_system]: <对应用户的密码>
Reenter password for [beats_system]: <对应用户的密码>
Enter password for [remote_monitoring_user]: <对应用户的密码>
Reenter password for [remote_monitoring_user]: <对应用户的密码>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

1.6.验证密码设定

浏览器访问client节点
http://es-client:9200/
用户：elastic
密码： <对应用户的密码>
1
2
3
4

1.7.Kibana修改elasticsearch的用户密码

• 修改kibana的配置文件kibana-7.13.4-linux-x86_64/config/kibana.yml

  elasticsearch.username: "elasitc"
  elasticsearch.password: "<对应用户的密码>"
  1
  2

1.8.验证Kibana启用用户密码

浏览器访问http://<kibana宿主机IP>:5601/
用户：elasitc
密码：<对应用户的密码>
1
2
3

七、配置ElasticSearch的HTTPS访问

1.1.生成CA证书

# 在es-master-1节点上
su - elasticsearch
cd elasticsearch-7.13.4/
bin/elasticsearch-certutil ca -out config/elastic-stack-ca.p12 -pass ""
# 在elasticsearch-7.13.4/config生成elastic-stack-ca.p12文件
1
2
3
4
5

1.2.生成p12格式的certificate证书

cd elasticsearch-7.13.4
bin/elasticsearch-certutil cert --ca config/elastic-stack-ca.p12 -pass "" -out config/elastic-certificates-https.p12
.........
Enter password for CA (config/elastic-stack-ca.p12) : <直接回车，无密码>
Certificates written to /home/elasticsearch/elasticsearch-7.13.4/config/elastic-certificates-https.p12
.........
1
2
3
4
5
6

1.3.生成pem格式的证书

cd elasticsearch-7.13.4
openssl pkcs12 -in config/elastic-stack-ca.p12 -out config/ca.crt.pem -clcerts -nokeys
Enter Import Password:<直接回车，无密码>
MAC verified OK
1
2
3
4

1.4.将证书拷贝到其他节点

scp -r elasticsearch-7.13.4/config/elastic-certificates-https.p12 "es-master&data-1":~/elasticsearch-7.13.4/config/elastic-certificates-https.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates-https.p12 "es-master&data-2":~/elasticsearch-7.13.4/config/elastic-certificates-https.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates-https.p12 es-data-1:~/elasticsearch-7.13.4/config/elastic-certificates-https.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates-https.p12 "es-data&ingest-1":~/elasticsearch-7.13.4/config/elastic-certificates-https.p12
scp -r elasticsearch-7.13.4/config/elastic-certificates-https.p12 es-client:~/elasticsearch-7.13.4/config/elastic-certificates-https.p12
1
2
3
4
5

1.5.修改elasticsearch配置文件(各节点)

• elasticsearch-7.13.4/config/elasticsearch.yml

xpack.security.enabled: true（如果设定了用户密码则可以忽略此条）
xpack.security.transport.ssl.enabled: true （如果设定了用户密码则可以忽略此条）
xpack.security.http.ssl.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates-https.p12
xpack.security.http.ssl.truststore.path: elastic-certificates-https.p12
1
2
3
4
5
6

1.6.重启所有节点

kill -9 <elasticsearch启动进程>
cd elasticsearch-7.13.4/bin
nohup ./elasticsearch &
1
2
3

1.7.验证https访问

使用浏览器访问https://es-client:9200/
1

1.8.Kibana修改HTTPS链接

# 将1.3生成的ca.crt.pem拷贝到Kibana服务器上
# 修改Kibana配置文件kibana-7.13.4-linux-x86_64/config/kibana.yml

elasticsearch.ssl.certificateAuthorities: ["<1.3生成的pem文件的绝对路径>"]
elasticsearch.ssl.verificationMode: none
elasticsearch.hosts: ["https://<client节点IP>:9200"]
1
2
3
4
5
6

八、安装IK中文分词器

su - elasticsearch
# 获取IK分词器安装包
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.13.4/elasticsearch-analysis-ik-7.13.4.zip
# 解压安装包
unzip elasticsearch-analysis-ik-7.13.4.zip -d elasticsearch-7.13.4/plugins/ik
# 复制到其他节点
scp -r elasticsearch-7.13.4/plugins/ik "es-master&data-1":~/elasticsearch-7.13.4/plugins/ik
scp -r elasticsearch-7.13.4/plugins/ik "es-master&data-2":~/elasticsearch-7.13.4/plugins/ik
scp -r elasticsearch-7.13.4/plugins/ik es-data-1:~/elasticsearch-7.13.4/plugins/ik
scp -r elasticsearch-7.13.4/plugins/ik "es-data&ingest-1":~/elasticsearch-7.13.4/plugins/ik
scp -r elasticsearch-7.13.4/plugins/ik es-client:~/elasticsearch-7.13.4/plugins/ik
1
2
3
4
5
6
7
8
9
10
11

九、配置ElasticSearch读写分离

1.1.读写规划分配

• 针对各Data节点设定

• master&data-1、data-1作为数据写入

• master&data-2、ingest&data-1作为数据读取

1.2.修改数据节点配置

# elasticsearch-7.13.4/config/elasticsearch.yml 
# 修改master&data-1、data-1节点配置文件
node.attr.temperature: hot
# 修改master&data-2、ingest&data-1节点配置文件
node.attr.temperature: cool
# 重启生效
1
2
3
4
5
6

1.3.实现读写分离(代码层)

• 1.创建索引

  PUT index_wr	# 创建一个名为index_wr索引
  {
     "settings": {
       "index.routing.allocation.require.temperature": "hot",# 指定索引分片分布于带hot标签的节点上
       "number_of_replicas": 0,	# 副本数为零
       "number_of_shards": 4		# 主分片数为4，建议与数据节点数量匹配
      }
  }
  
  # 可进行查看，所有新建的分片都分布于带hot标签的数据节点上
  1
  2
  3
  4
  5
  6
  7
  8
  9
  10

• 2.修改索引副本数

  PUT index_wr/_settings
  {
       "index.routing.allocation.require.temperature": null,
       "number_of_replicas": 1 # 副本数为1
  }
  # 可进行查看所有的副本分片全都分布于带有cool标签的数据节点上
  1
  2
  3
  4
  5
  6

• 3.优先读取副本数据

  读取时，参数设定preference=_replica_first
  1

十、集群启停脚本

#!/bin/bash
es_home=/home/elasticsearch/elasticsearch-7.13.4
case $1 in
"start") {
 for i in "es-master-1" "es-master&data-1" "es-master&data-2" "es-data-1" "es-data&ingest-1" "es-client"
 do
   echo "==============$i 上 ElasticSearch 启动=============="
   ssh $i "source /etc/profile;${es_home}/bin/elasticsearch >/dev/null 2>&1 &"
 done
};;
"stop") {
 for i in "es-master-1" "es-master&data-1" "es-master&data-2" "es-data-1" "es-data&ingest-1" "es-client"
 do
   echo "==============$i 上 ElasticSearch 停止=============="
   ssh $i "ps -ef|grep $es_home |grep -v grep|awk '{print \$2}'|xargs kill" >/dev/null 2>&1
 done
 };;
esac
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18