热门标签
热门文章
- 1单链表基本操作(2)_6-1 单链表
- 2Python学习干货，如何用Python进行数据分析？_在熟悉各种典型的数据预处理方法原理的基础上,选择数据分析工具python语言进行数
- 342. 【农产品溯源项目前后端Demo】后端-区块链连接服务_区块链网络 连接前端连还是后端连
- 4Python制作【大麦网】抢票程序，看演唱会再也不怕没票了_python 操控苹果手机抢大麦网演唱会的票
- 5node【引入模块查找机制】_npm模块搜索
- 6（大数据静态处理demo）flume+hadoop+hive+sqoop+mysql
- 7mac上搭建鸿蒙开发环境（2024）_鸿蒙 mac
- 8Vue3项目运行Error: Cannot find module ‘vue-loader-v16/package.json‘问题处理_loader error 3
- 9如何解决“来自计算机网络的异常流量”_我们的系统检测到您的计算机网络中存在异常流量。请稍后重新发送请求。为什么会这
- 10可能是MacOS中最好用的集成开发工具-Xcode初学者(C/C++ 新生)教程_xcode 开发教程
当前位置: article > 正文
【云原生 | Envoy 系列】--Envoy负载均衡策略_envoy 负载均衡
作者：Gausst松鼠会 | 2024-05-30 21:33:21
赞
踩
envoy 负载均衡
1. Envoy负载均衡策略
Envoy提供了几种不同的负载均衡策略,并可大体分为全局负载均衡和分布式负载均衡两大类
- 分布式负载均衡: Envoy自身基于上游主机(区域感知)的位置及健康状态等来确定如何分配负载至相关端点
- 主动健康检查
- 区域感知路由
- 负载均衡算法
- 全局负载均衡: 这是一种通过单个具有全局权限的组件来统一决策负载机制,Envoy的控制平面即是该类组件之一,它能通过指定各种参数来调整应用于各端点的负载.
- 优先级
- 位置权重
- 端点权重
- 端点健康状态
复杂的部署场景可以混合使用两类负载均衡策略,全局负载均衡通过定义高级路由优先级和权重以控制同级别的流量,而分布式负载均衡用于对系统中的微观变动作出反应.
clusters: - name: ... ... load_assignment: {...} cluster_name: ... endpoints: [] # LocalityLbEndpoints列表，每个列表项主要由位置、端点列表、权重和优先级四项组成； - locality: {...} # 位置定义 region: ... zone: ... sub_zone: ... lb_endpoints: [] # 端点列表 - endpoint: {...} # 端点定义 address: {...} # 端点地址 health_check_config: {...} # 当前端点与健康状态检查相关的配置； load_balancing_weight: ... # 当前端点的负载均衡权重，可选； metadata: {...} # 基于匹配的侦听器、过滤器链、路由和端点等为过滤器提供额外信息的元数据，常用用于提供服务配置或辅助负载均衡； health_status: ... # 端点是经EDS发现时，此配置项用于管理式设定端点的健康状态，可用值有UNKOWN、HEALTHY、UNHEALTHY、DRAINING、TIMEOUT和DEGRADED； load_balancing_weight: {...} # 权重 priority: ... # 优先级 policy: {...} # 负载均衡策略设定 drop_overloads: [] # 过载保护机制，丢弃过载流量的机制； overprovisioning_factor: ... # 整数值，定义超配因子（百分比），默认值为140，即1.4； endpoint_stale_after: ... # 过期时长，过期之前未收到任何新流量分配的端点将被视为过时，并标记为不健康；默认值0表示永不过时； lb_subset_config: {...} # 负载均衡子集 ring_hash_lb_config: {...} # 环hash算法配置 original_dst_lb_config: {...} # 原始连接配置 least_request_lb_config: {...} # 最少连接数配置 common_lb_config: {...} # 公共配置 health_panic_threshold: ... # 恐慌阈值,Panic阈值，默认为50%； zone_aware_lb_config: {...} # 区域感知路由的相关配置； locality_weighted_lb_config: {...} # 局部权重负载均衡相关的配置； ignore_new_hosts_until_first_hc: ... # 是否在新加入的主机经历第一次健康状态检查之前不予考虑进负载均衡；
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
2. Envoy负载均衡算法
- 加权轮询(weighted round robin): ROUND_ROBIN
- 加权最少连请求(weighted least request): LEAST_REQUEST
- 环哈希(ring hash): RING_HASH工作方式类似于一致性hash算法
- 磁悬浮(maglev): 类似于环哈希,但大小固定为65537,并需要个主机映射的节点填满整个环,无论配置的主机和位置权重如何,算法都会尝试确保将每个主机至少映射一次;算法名称MAGLEV
- 随机(random): 未配置健康检查策略,则随机负载均衡算法比轮询更好.
- 原始目标负载均衡 ORIGINAL_DST_LB
2.1 加权最少请求
-
所有主机权重相同(对长连接更有意义)
- 这是一种复杂度为O(1)调度算法,它随机选择N个(默认2,可调整)可用主机并从中选取具有最少活动请求的主机
- P2C算法效果不亚于O(1)复杂度的全扫描算法.它确保了集群中具有最大连接数的端点绝不会收到新的请求,直到其连接数小于等于其他主机
-
所有主机权重不同(对短连接更有意义)
- 调度算法使用加权轮询的方式调度,权重将根据主机在请求时的请求负载进行动态调整,方法是权重除以当前活动请求数.
- 该算法在稳态下可提供良好的平衡效果,但可能无法尽快适应不太均衡的负载场景
- 与P2C不同,主机将永远不会真正排空,即便随着时间的推移它将收到更少的请求.
2.1.1 docker-compose
和之前一样,1个front envoy作为流量的入口172.31.22.2
三对后端EP,sidecar+webserver的组合.ip地址分别是172.31.22.11,172.31.22.12,172.31.22.13
version: '3.3' services: envoy: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./front-envoy.yaml:/etc/envoy/envoy.yaml networks: envoymesh: ipv4_address: 172.31.22.2 aliases: - front-proxy depends_on: - webserver01-sidecar - webserver02-sidecar - webserver03-sidecar webserver01-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: red networks: envoymesh: ipv4_address: 172.31.22.11 aliases: - myservice - red webserver01: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver01-sidecar" depends_on: - webserver01-sidecar webserver02-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: blue networks: envoymesh: ipv4_address: 172.31.22.12 aliases: - myservice - blue webserver02: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver02-sidecar" depends_on: - webserver02-sidecar webserver03-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: green networks: envoymesh: ipv4_address: 172.31.22.13 aliases: - myservice - green webserver03: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver03-sidecar" depends_on: - webserver03-sidecar networks: envoymesh: driver: bridge ipam: config: - subnet: 172.31.22.0/24
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
2.1.2 envoy.yaml
定义了三个endpoint的权重1:3:5
admin: profile_path: /tmp/envoy.prof access_log_path: /tmp/admin_access.log address: socket_address: { address: 0.0.0.0, port_value: 9901 } static_resources: listeners: - name: listener_0 address: socket_address: { address: 0.0.0.0, port_value: 80 } filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http codec_type: AUTO route_config: name: local_route virtual_hosts: - name: webservice domains: ["*"] routes: - match: { prefix: "/" } route: { cluster: web_cluster_01 } http_filters: - name: envoy.filters.http.router clusters: - name: web_cluster_01 connect_timeout: 0.25s type: STRICT_DNS lb_policy: LEAST_REQUEST load_assignment: cluster_name: web_cluster_01 endpoints: - lb_endpoints: - endpoint: address: socket_address: address: red port_value: 80 load_balancing_weight: 1 - endpoint: address: socket_address: address: blue port_value: 80 load_balancing_weight: 3 - endpoint: address: socket_address: address: green port_value: 80 load_balancing_weight: 5
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
2.1.3 测试脚本
测试脚本内容:
总共做300次连接,对front envoy/hostname 进行请求,并对返回内容进行过滤统计,最后将统计结果返回
# cat send-request.sh #!/bin/bash declare -i red=0 declare -i blue=0 declare -i green=0 #interval="0.1" counts=300 echo "Send 300 requests, and print the result. This will take a while." echo "" echo "Weight of all endpoints:" echo "Red:Blue:Green = 1:3:5" for ((i=1; i<=${counts}; i++)); do if curl -s http://$1/hostname | grep "red" &> /dev/null; then # $1 is the host address of the front-envoy. red=$[$red+1] elif curl -s http://$1/hostname | grep "blue" &> /dev/null; then blue=$[$blue+1] else green=$[$green+1] fi # sleep $interval done echo "" echo "Response from:" echo "Red:Blue:Green = $red:$blue:$green"
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
2.1.4 部署测试效果
可以看到least_requests的算法不是按绝对比例进行调度,理论上请求数越大越是趋近预设的理想值.
# docker-compose up # ./send-request.sh 172.31.22.2 Send 300 requests, and print the result. This will take a while. Weight of all endpoints: Red:Blue:Green = 1:3:5 Response from: Red:Blue:Green = 29:92:179 # ./send-request.sh 172.31.22.2 Send 3000 requests, and print the result. This will take a while. Weight of all endpoints: Red:Blue:Green = 1:3:5 Response from: Red:Blue:Green = 310:839:1851
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
2.2 环哈希和磁悬浮
环哈希是对2的32次方取模,缺点是计算量较大,节点少时可能存在负载的偏斜
磁悬浮是换哈希的改良,对65537取模,构建权重使得所有节点填满整个环,减少了计算量,计算完成时就确定了使用的是哪个节点,不用去计算下个点.计算消耗小于环哈希,稳定性略逊于换哈希.
2.2.1 docker-compose
version: '3.3' services: envoy: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./front-envoy.yaml:/etc/envoy/envoy.yaml networks: envoymesh: ipv4_address: 172.31.25.2 aliases: - front-proxy depends_on: - webserver01-sidecar - webserver02-sidecar - webserver03-sidecar webserver01-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: red networks: envoymesh: ipv4_address: 172.31.25.11 aliases: - myservice - red webserver01: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver01-sidecar" depends_on: - webserver01-sidecar webserver02-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: blue networks: envoymesh: ipv4_address: 172.31.25.12 aliases: - myservice - blue webserver02: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver02-sidecar" depends_on: - webserver02-sidecar webserver03-sidecar: image: envoyproxy/envoy-alpine:v1.21.5 volumes: - ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml hostname: green networks: envoymesh: ipv4_address: 172.31.25.13 aliases: - myservice - green webserver03: image: ikubernetes/demoapp:v1.0 environment: - PORT=8080 - HOST=127.0.0.1 network_mode: "service:webserver03-sidecar" depends_on: - webserver03-sidecar networks: envoymesh: driver: bridge ipam: config: - subnet: 172.31.25.0/24
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
2.2.2 envoy.yaml
定义了算法为RING_HASH,环最大为2的20次方,最小为2的9次方.
对于什么进行hash,是基于route_config中hash_policy的参数进行设定,常见的会根据源地址hash或uri,浏览器等进行hash
健康检查基于/livez返回值200-399进行判断
admin: profile_path: /tmp/envoy.prof access_log_path: /tmp/admin_access.log address: socket_address: { address: 0.0.0.0, port_value: 9901 } static_resources: listeners: - name: listener_0 address: socket_address: { address: 0.0.0.0, port_value: 80 } filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http codec_type: AUTO route_config: name: local_route virtual_hosts: - name: webservice domains: ["*"] routes: - match: { prefix: "/" } route: cluster: web_cluster_01 hash_policy: # - connection_properties: # source_ip: true - header: header_name: User-Agent http_filters: - name: envoy.filters.http.router clusters: - name: web_cluster_01 connect_timeout: 0.5s type: STRICT_DNS lb_policy: RING_HASH ring_hash_lb_config: maximum_ring_size: 1048576 minimum_ring_size: 512 load_assignment: cluster_name: web_cluster_01 endpoints: - lb_endpoints: - endpoint: address: socket_address: address: myservice port_value: 80 health_checks: - timeout: 5s interval: 10s unhealthy_threshold: 2 healthy_threshold: 2 http_health_check: path: /livez expected_statuses: start: 200 end: 399
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
2.2.3 部署测试
启动容器后,当User-Agent不发生改变,始终会被调度到通一台服务器上.
一旦User-Agent发生改变,根据计算后的hash值,请求会被分别调度到三台服务器之一进行调度,并保持不变.即使间隔很久再次访问.
# docker-compose up # while true;do curl 172.31.25.2;sleep 2;done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! root@k8s-node-1:~# while true;do curl -H "User-Agent: Chrome" 172.31.25.2;sleep 2;done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! root@k8s-node-1:~# while true;do curl -H "User-Agent: IE6.0" 172.31.25.2;sleep 2;done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12! ^C root@k8s-node-1:~# while true;do curl -H "User-Agent: IE4.0" 172.31.25.2;sleep 2;done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.25.11! ^C root@k8s-node-1:~# while true;do curl -H "User-Agent: IE7.0" 172.31.25.2;sleep 2;done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.25.13! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.25.13! iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.25.13! # curl 172.31.25.2 iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.25.12!
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
3. 负载均衡算法总结
-
负载均衡算法
- ROUND_ROBIN(轮询): 默认负载均衡算法
- LEAST_CONN(最小连接数): 随机选取两个健康的主机,再从所选取的两个主机中选择一个链接数较少的主机
- RANDOM(随机): 从所有健康的主机中,随机选取一个主机,在负载均衡池的端点上均匀分配负载.在没有健康检查策略的情况下,随机通常会比轮询调度策略更高效,但不会有任何顺序
-
会话保持
- 根据HTTP header中的内容进行HASH
- 根据Cookie中的内容进行HASH
- 根据源IP进行HASH
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Gausst松鼠会/article/detail/648695
推荐阅读
相关标签
