devbox
木道寻08
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

iLogtail专题六：实际场景采集模拟_ilogtail收集info日志里的堆栈信息

作者：木道寻08 | 2024-07-27 20:13:09

赞

踩

ilogtail收集info日志里的堆栈信息

 公司一直使用的Filebeat进行日志采集
 由于Filebeat采集组件一些问题，现需要使用iLogtail进行代替
 现记录下iLogtail介绍和实际使用过程
 这是iLogtail系列的第六篇文章

背景：

由于平时日志采集存在堆栈信息日志情况，需要对日志进行正确分割。

前期准备：

准备日志打印程序并启动。输出日志信息如下：

2022-03-02 14:01:30.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值1

2022-03-02 14:01:30.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值2

2022-03-02 14:01:30.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值3

2022-03-02 14:01:30.006 ERROR 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : ERROR

java.lang.ArithmeticException: / by zero

        at com.ai.toptea.sysm.invilid.InvilidResource.invilidJob2(InvilidResource.java:35) ~[sysm-data-invilid-2.0.0.jar:2.0.0]

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]

        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]

        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]

        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]

        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_152]

        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_152]

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_152]

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_152]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_152]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_152]

        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]

2022-03-02 14:01:30.006 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值4

2022-03-02 14:01:30.006 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值5

采集配置：

多行日志在提交给插件部分时也需要使用 processor_split_log_regex 进行基于行首正则的切分，日期开头配置如下：

{

    "detail":{

        "SplitKey":"content",

        "SplitRegex":"\\d+-\\d+-\\d+ \\d+:\\d+:\\d+.*"

    },

    "type":"processor_split_log_regex"

}

 

采集到kafka中的数据，已将堆栈信息和error日志正常合并，可以满足日常采集需求。

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值1\n"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"},{"Key":"type","Value":"LOGTAIL_LOG"}]}

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值2\n"},{"Key":"type","Value":"LOGTAIL_LOG"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"}]}

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.000 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值3\n"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"},{"Key":"type","Value":"LOGTAIL_LOG"}]}

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.001 ERROR 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : ERROR\n\njava.lang.ArithmeticException: / by zero\n\tat com.ai.toptea.sysm.invilid.InvilidResource.invilidJob2(InvilidResource.java:35) ~[sysm-data-invilid-2.0.0.jar:2.0.0]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]\n\tat org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]\n\tat org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]\n\tat org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_152]\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_152]\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_152]\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_152]\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_152]\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_152]\n\tat java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]\n\n"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"},{"Key":"type","Value":"LOGTAIL_LOG"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"}]}

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.001 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值4\n"},{"Key":"type","Value":"LOGTAIL_LOG"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"}]}

{"Time":1646200910,"Contents":[{"Key":"content","Value":"2022-03-02 14:01:50.001 DEBUG 3049 --- [pool-5-thread-1] c.a.toptea.sysm.invilid.InvilidResource  : 输出正常值5"},{"Key":"type","Value":"LOGTAIL_LOG"},{"Key":"@timestamp","Value":"@timestamp"},{"Key":"beat","Value":"beat"},{"Key":"fields","Value":"fields"},{"Key":"input_type","Value":"log"},{"Key":"offset","Value":"offset"}]}

备注：

处理数据插件介绍可参考使用Logtail插件处理数据 - 日志服务 - 阿里云

尽量不要通过修改基础配置的方式实现，测试中修改基础设置未生效，查看源码后发现可能有所限制。

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/木道寻08/article/detail/891569
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号