赞
踩
背景:kafka的jmx指标数据要展示到监控需要prometheus的java agent来采集,jmx指标直接展示不太友好,还需要通过python或java api展示出来,在通过prometheus或其他手段上报到监控,比较麻烦,好在有开源的jmx_exporter可以直接使用,有延时等详细的指标,但是yml模板比较粗糙需要了解jmx的指标含义并知道具体格式然后了解yml中规则配置然后进行改写,如下是我的改写可以采集一些详细的需要信息,jmx的指标含义可以参考kafka官网
github:https://github.com/prometheus/jmx_exporter
配置文件:https://github.com/prometheus/jmx_exporter/blob/master/example_configs/kafka-2_0_0.yml ## kafka服务侧的配置 在kafka-server-start.sh中最后一行之前添加如下配置
export KAFKA_OPTS="-javaagent:$base_dir/../jmx_prometheus_javaagent-0.16.1.jar=9011:$base_dir/../kafka-xxt-2_0_0.yml"
注释:这里我对原有yml文件进行了改写,命名为kafka-xxt-2_0_0.yml,可共使用,路径即为agent和yml文件存放的路径,建议放到kafka的安装目录下
lowercaseOutputName: true rules: # Special cases and very specific rules - pattern : kafka.server<type=KafkaRequestHandlerPool, name=RequestHandlerAvgIdlePercent><>OneMinuteRate name: kafka_server_kafkarequesthandlerpool_requesthandleravgidlepercent type: GAUGE #- pattern : kafka.server<type=(.+), name=(.+),(.*)=(.*)><>Value # name: kafka_server_$1_$2 # type: GAUGE # labels: # "$3": "$4" - pattern : kafka.server<type=BrokerTopicMetrics, name=(.+), topic=(.*)><>(\w+)Rate name: kafka_server_brokertopicmetrics_$1 type: GAUGE labels: "topic": "$2" "rate": "$3" - pattern : kafka.server<type=BrokerTopicMetrics, name=(.+)><>(\w+)Rate name: kafka_server_brokertopicmetrics_$1 type: GAUGE labels: "rate": "$2" - pattern : kafka.server<type=BrokerTopicMetrics, name=(.+)><>Count name: kafka_server_brokertopicmetrics_$1_count type: GAUGE - pattern : kafka.server<type=ReplicaManager, name=(\w+)><>(\w+)Rate name: kafka_server_replicamanager_$1 type: GAUGE labels: "rate": "$2" - pattern : kafka.server<type=ReplicaManager, name=(\w+)><>Value name: kafka_server_replicamanager_$1_value type: GAUGE - pattern : kafka.network<type=RequestMetrics, name=(.+)Ms, request=(.*)><>(\d+)thPercentile name: kafka_network_requestmetrics_$1 type: GAUGE labels: "reqeust": "$2" "quantile": "0.$3" - pattern : kafka.server<type=SessionExpireListener, name=(\w+)><>(\w+)Rate name: kafka_server_SessionExpireListener_$1 type: GAUGE labels: "rate": "$2" - pattern : kafka.server<type=SessionExpireListener, name=(\w+)><>Count name: kafka_server_SessionExpireListener_$1_count type: GAUGE - pattern : kafka.server<type=socket-server-metrics, listener=(\w+), networkProcessor=(\d+)><>connection-([\w-]+) name: kafka_socket-server-metrics_$1_networkprocessor_$2 type: GAUGE labels: "connection": "$3" #- pattern : kafka.server<type=socket-server-metrics, listener=(\w+), networkProcessor=(\d+)><>connection-count # name: kafka_socketservermetric_$1_networkprocessor_$2_connectioncount # type: GAUGE - pattern : kafka.network<type=RequestChannel, name=(\w+)><>Value name: requestchannel_$1_value type: GAUGE - pattern : kafka.log<type=LogFlushStats, name=LogFlushRateAndTimeMs><>(\w+)Rate name: kafka_log_logflushstats_logflushrateandtimems type: GAUGE labels: "rate": "$1" - pattern : kafka.log<type=LogFlushStats, name=LogFlushRateAndTimeMs><>(\d+)thPercentile name: kafka_log_logflushstats_logflushrateandtimems type: GAUGE labels: "quantile": "0.$1"
具体的指标含义和样式可以参数kafka官网和通过python 查询jmx指标获取,然后和模板规则对对照,明白具体样式在做修改
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。