ranger审计日志对接CDH solr

作者：耀灵

一、准备条件

1、已安装完毕ranger-admin
2、已在CDH上部署solr（注意在安装solr时更改下solr在zk上的节点信息）

二、更改相关配置

1、修改ranger-2.1.0-admin/contrib/solr_for_audit_setup/install.properties

SOLR_USER=solr
SOLR_GROUP=solr
MAX_AUDIT_RETENTION_DAYS=90

SOLR_INSTALL=false



SOLR_DOWNLOAD_URL=


SOLR_INSTALL_FOLDER=/opt/cloudera/parcels/CDH/lib/solr

SOLR_RANGER_HOME=/opt/cloudera/parcels/CDH/lib/solr/ranger_audit_server

SOLR_RANGER_PORT=8983

SOLR_DEPLOYMENT=solrcloud

SOLR_RANGER_DATA_FOLDER=/opt/cloudera/parcels/CDH/lib/solr/ranger_audit_server/data


SOLR_ZK=100.116.3.226:2181,100.116.3.227,100.116.3.228/solr/configs/ranger_audits
SOLR_HOST_URL=http://`hostname -f`:8983
SOLR_SHARDS=3
SOLR_REPLICATION=2

SOLR_LOG_FOLDER=/opt/logs/solr/ranger_audits

SOLR_RANGER_COLLECTION=ranger_audits

SOLR_MAX_MEM=2g
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

2、修改ranger-2.1.0-admin/contrib/solr_for_audit_setup/conf/solrconfig.xml

此处修改为

EEE MMM dd HH:mm:ss [z ]yyyy
1

此处直接改为cdh上solr jar对应的目录，这个配置文件之前是基于非集成的solr配置的，不做修改的话会报配置文件找不到的错误。

3、修改ranger-admin的install.propertities

PYTHON_COMMAND_INVOKER=python

DB_FLAVOR=MYSQL

SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar


db_root_user=root
db_root_password=VYi2OfrsO0Mz
db_host=100.116.3.249
db_ssl_enabled=false
db_ssl_required=false
db_ssl_verifyServerCertificate=false
db_ssl_auth_type=2-way
javax_net_ssl_keyStore=
javax_net_ssl_keyStorePassword=
javax_net_ssl_trustStore=
javax_net_ssl_trustStorePassword=
db_name=ranger
db_user=ranger
db_password=VYi2OfrsO0Mz

rangerAdmin_password=
rangerTagsync_password=
rangerUsersync_password=
keyadmin_password=


audit_store=solr

audit_elasticsearch_urls=
audit_elasticsearch_port=
audit_elasticsearch_protocol=
audit_elasticsearch_user=
audit_elasticsearch_password=
audit_elasticsearch_index=
audit_elasticsearch_bootstrap_enabled=false


audit_solr_urls=http://100.116.3.227:8983/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=100.116.3.226:2181,100.116.3.227:2181,100.116.3.228:2181/ranger_audits

audit_solr_collection_name=ranger_audits
audit_solr_config_name=ranger_audits
audit_solr_no_shards=
audit_solr_no_replica=
audit_solr_max_shards_per_node=
audit_solr_acl_user_list_sasl=solr,infra-solr
audit_solr_bootstrap_enabled=true



policymgr_external_url=http://localhost:6080
policymgr_http_enabled=true
policymgr_https_keystore_file=
policymgr_https_keystore_keyalias=rangeradmin
policymgr_https_keystore_password=

policymgr_supportedcomponents=



unix_user=ranger
unix_user_pwd=ranger
unix_group=ranger


authentication_method=NONE
remoteLoginEnabled=true
authServiceHostName=localhost
authServicePort=5151
ranger_unixauth_keystore=keystore.jks
ranger_unixauth_keystore_password=password
ranger_unixauth_truststore=cacerts
ranger_unixauth_truststore_password=changeit


xa_ldap_url=
xa_ldap_userDNpattern=
xa_ldap_groupSearchBase=
xa_ldap_groupSearchFilter=
xa_ldap_groupRoleAttribute=
xa_ldap_base_dn=
xa_ldap_bind_dn=
xa_ldap_bind_password=
xa_ldap_referral=
xa_ldap_userSearchFilter=

xa_ldap_ad_domain=
xa_ldap_ad_url=
xa_ldap_ad_base_dn=
xa_ldap_ad_bind_dn=
xa_ldap_ad_bind_password=
xa_ldap_ad_referral=
xa_ldap_ad_userSearchFilter=

spnego_principal=
spnego_keytab=
token_valid=30
cookie_domain=
cookie_path=/
admin_principal=
admin_keytab=
lookup_principal=
lookup_keytab=
hadoop_conf=/etc/hadoop/conf
sso_enabled=false
sso_providerurl=https://127.0.0.1:8443/gateway/knoxsso/api/v1/websso
sso_publickey=


RANGER_ADMIN_LOG_DIR=$PWD

RANGER_PID_DIR_PATH=/var/run/ranger

XAPOLICYMGR_DIR=$PWD
app_home=$PWD/ews/webapp
TMPFILE=$PWD/.fi_tmp
LOGFILE=$PWD/logfile
LOGFILES="$LOGFILE"

JAVA_BIN='java'
JAVA_VERSION_REQUIRED='1.8'
JAVA_ORACLE='Java(TM) SE Runtime Environment'

ranger_admin_max_heap_size=1g
PATCH_RETRY_INTERVAL=120
STALE_PATCH_ENTRY_HOLD_TIME=10

mysql_core_file=db/mysql/optimized/current/ranger_core_db_mysql.sql
mysql_audit_file=db/mysql/xa_audit_db.sql

oracle_core_file=db/oracle/optimized/current/ranger_core_db_oracle.sql
oracle_audit_file=db/oracle/xa_audit_db_oracle.sql
postgres_core_file=db/postgres/optimized/current/ranger_core_db_postgres.sql
postgres_audit_file=db/postgres/xa_audit_db_postgres.sql
sqlserver_core_file=db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
sqlserver_audit_file=db/sqlserver/xa_audit_db_sqlserver.sql
sqlanywhere_core_file=db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
sqlanywhere_audit_file=db/sqlanywhere/xa_audit_db_sqlanywhere.sql
cred_keystore_filename=$app_home/WEB-INF/classes/conf/.jceks/rangeradmin.jceks
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

三、初始化

1、修改完成后，初始化solr、ranger-admin
ranger-2.1.0-admin/contrib/solr_for_audit_setup

./setup.sh
1

./setup.sh
1

四、上传相关配置到zookeeper并创建对应collection

1、相关指令如下

查看创建的solrhome的实例 
solrctl instancedir --list 


创建 collection1 实例并将配置文件上传到 zookeeper: 
#conf必须是一个目录，这里的目录是 ranger-2.1.0-admin/contrib/solr_for_audit_setup/conf
solrctl instancedir --create ranger_audits conf 


可以通过下面命令查看上传的实体： 
solrctl instancedir --list 
创建 collection 
1）创建ranger_audits
solrctl collection --create ranger_audits -s 3 -c ranger_audits -r 2 -m 3
1
2
3
4
5
6
7
8
9
10
11
12
13
14

2、验证：登陆zookeeper查看

查看solr的上collection 是否创建成

更多技术信息请查看云掣官网https://yunche.pro/?t=yrgw