喵喵爱编程

这个屌丝很懒，什么也没留下！

热门标签

安全运维 -- splunk 操作手册_splunk使用教程

作者：喵喵爱编程 | 2024-07-20 11:31:00

踩

splunk使用教程

0x00 背景

splunk 日常运维操作笔记。按照场景编号。

0x01 agent安装

下载链接 Splunk Universal Forwarder Previous Releases | Splunk

linux：

tar -zxvf splunkforwarder-8.0.3-a6754d8441bf-Linux-x86_64.tgz -C /opt
cp -r config /opt/splunkforwarder/etc/apps
vi /opt/splunkforwarder/etc/apps/prefix_app_inputs/local/inputs.conf
[monitor:///home/var/log/*log] #修改监控路径

/opt/splunkforwarder/bin/splunk start --accept-license # 输入app账号密码
/opt/splunkforwarder/bin/splunk show deploy-poll # 检查配置文件是否配置好DS指向
/opt/splunkforwarder/bin/splunk enable boot-start # 开启自启动

windows:

Splunk 静默安装
msiexec -i splunkforwarder-9.0.3-a6754d8441bf-x64-release.msi SPLUNKUSERNAME=admin SPLUNKPASSWORD=pass AGREETOLICENSE=Yes /quiet

下面命令需要管理员权限：
echo f |xcopy /s /y /f deploymentclient.conf "%programfiles%\SplunkUniversalForwarder\etc\apps\yourcompany_all_deploymentclient\local\deploymentclient.conf"

(yourcompany_all_deploymentclient 文件夹名字随便取）

deploymentclient.conf

[deployment-client]

#phoneHomeIntervalInSecs = 600

clientName = your_indexname

[target-broker:deploymentServer]

targetUri = siemds.yourcompany.com:8089

yourcompany_all_deploymentclient 目录结构（app.conf 可有可无）

0x02 splunk接入log（读取日志文件方式）

1.登录DS 后台

vim /opt/splunk/etc/development-apps/prefix_all_indexes/local/indexes.conf

新增索引 [prefix_mailcasph]

[prefix_mailcasph]

homePath = volume:hotwarmdb/prefix_mailcasph/db

coldPath = volume:colddb/prefix_mailcasph/colddb

thawedPath = $SPLUNK_DB/prefix_mailcasph/thaweddb

inputs 负责数据采集，props负责数据解析

目录结构参考：

配置prefix_mailcasph_inputs文件夹：

vim prefix_mailcasph_inputs/local/app.conf (这个一般参考default/app.conf 不做修改)

[install]
state = enabled

[package]
check_for_updates = false

[ui]
is_visible = false
is_manageable = false

vim prefix_mailcasph_inputs/local/inputs.conf （inputs.conf - Splunk Documentation）

[monitor://D:\Exchange Server\Logging\HttpProxy\Eas\HttpProxy*.log]
index = prefix_mailcasph
sourcetype = ms:exchange:http_proxy
crcSalt = <SOURCE>
disabled = false

metadata 文件夹

local.meta 也是默认配置：

[]
access = read : [ * ], write : [ admin ]
export = system

配置prefix_mailcasph_props文件夹

vim prefix_mailcasph_props/local/app.conf

[install]
state = enabled

[package]
check_for_updates = false

[ui]
is_visible = false
is_manageable = false

props.conf (props.conf - Splunk Documentation) (来自splunk论坛某篇文章 https://www.splunk.com/en_us/blog/security/detecting-microsoft-exchange-vulnerabilities-0-8-days-later.html)

[ms:exchange:http_proxy]

CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
FIELD_DELIMITER=,
KV_MODE=none
SHOULD_LINEMERGE=false
disabled=false
TIMESTAMP_FIELDS=DateTime
TRANSFORMS-killheader1 = kh1
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)\d{4}\-\d{2}\-\d{2}T
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
MAX_TIMESTAMP_LOOKAHEAD = 25
NO_BINARY_CHECK=true
REPORT-extractfields = extractfields

transforms.conf (来自splunk论坛某篇文章 https://www.splunk.com/en_us/blog/security/detecting-microsoft-exchange-vulnerabilities-0-8-days-later.html)

[kh1]
REGEX = ^DateTime
DEST_KEY = queue
FORMAT = nullQueue
[extractfields]
DELIMS=","
FIELDS=DateTime,RequestId,MajorVersion,MinorVersion,BuildVersion,RevisionVersion,ClientRequestId,Protocol,UrlHo
st,UrlStem,ProtocolAction,AuthenticationType,IsAuthenticated,AuthenticatedUser,Organization,AnchorMailbox,UserA
gent,ClientIpAddress,ServerHostName,HttpStatus,BackEndStatus,ErrorCode,Method,ProxyAction,TargetServer,TargetSe
rverVersion,RoutingType,RoutingHint,BackEndCookie,ServerLocatorHost,ServerLocatorLatency,RequestBytes,ResponseB
ytes,TargetOutstandingRequests,AuthModulePerfContext,HttpPipelineLatency,CalculateTargetBackEndLatency,GlsLaten
cyBreakup,TotalGlsLatency,AccountForestLatencyBreakup,TotalAccountForestLatency,ResourceForestLatencyBreakup,To
talResourceForestLatency,ADLatency,SharedCacheLatencyBreakup,TotalSharedCacheLatency,ActivityContextLifeTime,Mo
duleToHandlerSwitchingLatency,ClientReqStreamLatency,BackendReqInitLatency,BackendReqStreamLatency,BackendProce
ssingLatency,BackendRespInitLatency,BackendRespStreamLatency,ClientRespStreamLatency,KerberosAuthHeaderLatency,
HandlerCompletionLatency,RequestHandlerLatency,HandlerToModuleSwitchingLatency,ProxyTime,CoreLatency,RoutingLat
ency,HttpProxyOverhead,TotalRequestTime,RouteRefresherLatency,UrlQuery,BackEndGenericInfo,GenericInfo,GenericEr
rors,EdgeTraceId,DatabaseGuid,UserADObjectGuid,PartitionEndpointLookupLatency,RoutingStatus

metadata 文件夹

local.meta 也是默认配置：

[]
access = read : [ * ], write : [ admin ]
export = system

2.下发配置

2.1 DS 下发配置到CM （需要admin密码）

sudo -u splunk /opt/splunk/bin/splunk reload deploy-server -class yourcompany_indexer_apps

这一步如果不熟练可以登录CM检查：

eg: 检查新增的index推送情况：

cat /opt/splunk/etc/master-apps/yourcompany_all_indexes/local/indexes.conf

登录CM 查看index status: checking the status of the applied bundle

sudo -u splunk /opt/splunk/bin/splunk show cluster-bundle-status

2.2 CM下发配置到indexer（需要admin密码）

sudo -u splunk /opt/splunk/bin/splunk apply cluster-bundle

这一步如果不熟练可以登录任意一台indexer检查：

eg: 检查新增的index推送情况：

cat /opt/splunk/etc/slave-apps/yourcompany_all_indexes/local/indexes.conf

如果没问题，在SH web应该可以搜索到数据啦。

0x03 splunk 安装app

splunk应用商店：

https://splunkbase.splunk.com/apps

阅读应用介绍，看应用是装在SH 还是 index 和 SH 都装。

1.在DS安装应用

因为是集群模式，所以登录DS web，选择 “应用 -- 管理应用 -- 从文件安装应用“。点击浏览，选择需要安装的应用，上载安装。

2.DS 下发到Deployer 和 CM

DS 后台，

推送到DP

sudo -u splunk /opt/splunk/bin/splunk reload deploy-server -class yourcompany_searchhead_apps

推送到CM
sudo -u splunk /opt/splunk/bin/splunk reload deploy-server -class yourcompany_indexer_apps

yourcompany_indexer_apps 这个app 是在 DS -- 转发器 -- 服务器管理里定义的。

3.DP和CM分别推送到SH和Indexer

推送到SH

sudo -u splunk /opt/splunk/bin/splunk apply shcluster-bundle -target https://10.111.xx.xxx:8089 # 集群模式任意选择一台SH

推送到Indexer

sudo -u splunk /opt/splunk/bin/splunk apply cluster-bundle

安装好app后可以通过syslog/文件等接入方式配置app的sourcetype来解析日志。

0x04 splunk 安装app解析日志

以radius为例：

1.自建索引和自建app解析：

新建索引参考上文此处略过。

DS 后台 /opt/splunk/etc/deployment-apps 新建 2个文件夹。

yourcompany_windows_radius_inputs

yourcompany_windows_radius_outputs

目录结构如下：（文件内容可以先阅读0x02章节）

yourcompany_windows_radius_inputs/metadata/local.meta

yourcompany_windows_radius_inputs/local/app.conf

yourcompany_windows_radius_inputs/local/inputs.conf

[monitor://C:\Windows\System32\LogFiles\WLAN\*]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = radius

[monitor://C:\Windows\System32\LogFiles\NPS\*]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = radius

[monitor://C:\Windows\System32\LogFiles\IN*.log]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = radius

yourcompany_windows_radius_outputs

yourcompany_windows_radius_outputs/local/props.conf

[radius]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\"\w+\",\"\w+\"\,\d{2}
CHARSET=GB2312
TIME_FORMAT=%m/%d/%Y,%H:%M:%S,%N
TIME_PREFIX=\"\w+\",\"\w+\"\,

这种 prop.conf 只能实现最简单的日志解析。

2.部署agent文本方式接入日志

客户端服务器新增文件夹：

apps/config/radius/yourcompany_all_deploymentclient/local/deploymentclient.conf

[deployment-client]

#phoneHomeIntervalInSecs = 600

clientName = yourcompany_radius

[target-broker:deploymentServer]

targetUri = siemds.yourcompany.com:8089

这时候发现可以接收到日志啦：(日志被简单分割)

"W10NPS0888","NAPS",06/12/2024,12:12:12,3,,"W10NPS0888\b7bbcccd777d",,,,,,,,0,"172.10.10.10","UA-77C7--UA-07C0707C-777",,,,,,,1,,16,"311 1 10.111.10.10 05/20/2024 12:12:12 11981996",,,,,,,,,"UA-01xxxxxxxxxxxxxxxx157",,,,,,,,,,,,,,,,,,,,,,,,,"Wired_BOX_Macbypass",1,,,,

3.去应用商店查找官方app

运气很好，找到一个：

https://splunkbase.splunk.com/app/6989

看介绍说只需要安装到SH即可。(一般情况下Add-on 需要放SH和Index，而App只需要放SH)

下载传到DP后台，在DP推送到SH

sudo -u splunk /opt/splunk/bin/splunk apply shcluster-bundle -target https://10.111.xx.xxx:8089 # 集群模式任意选择一台SH

修改DS配置，sourcetype改成app定义的。

/opt/splunk/etc/deployment-apps/yourcompany_windows_radius_inputs/local/inputs.conf

[monitor://C:\Windows\System32\LogFiles\WLAN\*]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = microsoft:nps:radius

[monitor://C:\Windows\System32\LogFiles\NPS\*]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = microsoft:nps:radius

[monitor://C:\Windows\System32\LogFiles\IN*.log]
index = yourcompany_radius
crcSalt = <SOURCE>
alwaysOpenFile = 1
disabled = false
sourcetype = microsoft:nps:radius

DS 配置推送到app：

sudo -u splunk /opt/splunk/bin/splunk reload deploy-server -class yourcompany_windows_radius_apps # DS转发器 -- 服务器管理里定义的

yourcompany_windows_radius_apps 服务器类下面含有2个应用：

yourcompany_windows_ad_outputs、yourcompany_windows_radius_inputs。

yourcompany_windows_ad_outputs 这个应用的作用定义输出：

yourcompany_windows_ad_outputs/local/outputs.conf

[tcpout]
defaultGroup = myIndexers
forceTimebasedAutoLB = true

[tcpout:myIndexers]
server=10.111.1.101:9998,10.111.1.102:9998,10.111.1.103:9998,10.111.1.104:9998 # 集群模式所有indexer

tcpSendBufSz = 512000

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/喵喵爱编程/article/detail/856655