微服务 分布式监控链路追踪服务架构演进 SkyWalking Jaeger Zipkin_zipkin jaeger

案例

一年实现降本60%，货拉拉全链路监控演进史

架构

实现一个全链路监控平台很难吗？Pinpoint、SkyWalking、Zipkin，哪个实现比较好？

日志排查问题困难？分布式日志链路跟踪来帮你

使用opentelemetry 搭建新一代可视化分布式链路追踪系统

SkyWalking

docker部署skywalking

Skywalking安装部署，Springboot集成Skywalking实现分布式链路追踪

Spring Boot 链路追踪 SkyWalking 入门

docker部署 skywalking 8.6.0 es6

docker run --name skywalking-oap \
--restart always \
-p 11800:11800 -p 12800:12800 -d \
-e TZ=Asia/Shanghai \
-e SW_ES_USER= \
-e SW_ES_PASSWORD= \
-e SW_STORAGE=elasticsearch \
-e SW_STORAGE_ES_CLUSTER_NODES=10.13.177.0:9201 \
-v /etc/localtime:/etc/localtime:ro \
apache/skywalking-oap-server:8.6.0-es6

docker run -d \
--name skywalking-ui \
--restart always \
-p 9898:8080 \
--link skywalking-oap:skywalking-oap \
-e TZ=Asia/Shanghai \
-e SW_OAP_ADDRESS=skywalking-oap:12800 \
-v /etc/localtime:/etc/localtime:ro \
apache/skywalking-ui:8.6.0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

docker 服务 agent配置

jenkins 启动配置：

docker run -d \
         --name ${NAME} \
         --net=host \
         -e JAVA_OPTS='-javaagent:/data/logs/agent/skywalking-agent.jar -Dskywalking.agent.service_name=bop-fms-query-info -server -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/data/logs/heapdump' \
         -e JAVA_ARGS="${JAVA_ARGS}" \
         -m 2048M \
         --memory-swap 2304M \
         -v /data0/${NAME}/logs:/data/logs \
         -v /data0/skywalking/agent:/data/logs/agent \
         -v /etc/localtime:/etc/localtime:ro \
         --ulimit nofile=200000 \
         registry.api.bo.com/${BUILD_TARGET}/${NAME}:${BUILD_TAG}

dockerfile配置：

FROM registry.api.bo.com/fms/base_image:jdk1.8.0_202_jdk

EXPOSE 8857

RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN mkdir -p /data/logs/

ADD bop-fms-query-info.jar /bop-fms-query-info.jar

ENV JAVA_OPTS=""

ENTRYPOINT java ${JAVA_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /bop-fms-query-info.jar ${JAVA_ARGS}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

Skywalking agent 同一服务器部署多个项目指定不同的 agent.config

java -javaagent:/data/logs/agent/skywalking-agent.jar -Dskywalking_config=/data/logs/agent/config/agent-copy.config -jar my-project.jar
1

Jaeger

Springboot整合OpenTracing分布式链路追踪系统Jaeger

zipkin

docker 部署zipkin 依赖 kafka、mysql

docker run -d --no-healthcheck \
-v /etc/localtime:/etc/localtime:ro \
-e STORAGE_TYPE=mysql \
-e MYSQL_USER=super_s \
-e MYSQL_PASS=xxxx \
-e MYSQL_HOST=m5985i.hebe.grid.ge.com.cn \
-e MYSQL_TCP_PORT=5985 \
-e MYSQL_DB=zipkin \
-e KAFKA_BOOTSTRAP_SERVERS=10.185.17.5:9092,10.182.16.8:9092,10.182.16.10:9092,10.182.16.15:9092 \
-e KAFKA_TOPIC=bop.proc.zipkin \
-e JAVA_OPTS='-Dzipkin.collector.kafka.overrides.auto.offset.reset=latest -Djava.security.egd=file:/dev/./urandom -Xmx2048m' \
-e ZIPKIN_UI_LOGS_URL='http://elk.bop.ge.com/app/kibana#/discover?_g=()&_a=(columns:!(servicename,serviceuri,serviceurl,traceid,operatorUid),filters:!(),index:'5e647230-013f-11ea-8157-fb72e9934870',interval:auto,query:(language:lucene,query:'{traceId}'),sort:!('@timestamp',desc))' \
-e ZIPKIN_UI_SUGGEST_LENS=false \
-e SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
--net host \
--name zipkin \
openzipkin/zipkin
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

docker 部署zipkin 依赖 kafka、es

docker run -d --no-healthcheck \
-v /etc/localtime:/etc/localtime:ro \
-e STORAGE_TYPE=elasticsearch \
-e ES_HOSTS=http://es.data.ge.com.cn:9206 \
-e ES_INDEX=bop-zipkin \
-e ES_USERNAME=ge_es \
-e ES_PASSWORD=bop_eg \
-e KAFKA_BOOTSTRAP_SERVERS=10.185.17.5:9092,10.182.16.8:9092,10.182.16.10:9092 \
-e KAFKA_TOPIC=bop.proc.zipkin \
-e JAVA_OPTS='-Dzipkin.collector.kafka.overrides.auto.offset.reset=latest -Djava.security.egd=file:/dev/./urandom -Xmx2048m' \
-e ZIPKIN_UI_LOGS_URL='http://elk.bop.ge.com/app/kibana#/discover?_g=()&_a=(columns:!(servicename,serviceuri,serviceurl,traceid,operatorUid),filters:!(),index:'5e647230-013f-11ea-8157-fb72e9934870',interval:auto,query:(language:lucene,query:'{traceId}'),sort:!('@timestamp',desc))' \
-e ZIPKIN_UI_SUGGEST_LENS=false \
-e SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
--net host \
--name zipkin \
openzipkin/zipkin
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

单机监控

调研字节码插桩技术，用于系统监控设计和实现