nginx-kafka插件安装_nginx 安装kafka插件

      插件的作用，在统计系统的pv、uv等指标时，可以使用nginx直接对接kafka，实现效率提升。
安装nginx-kafka插件步骤

1.安装git

yum install -y git
1

2.切换到/usr/local/src目录，然后将kafka的c客户端源码clone到本地

cd /usr/local/src
git clone https://github.com/edenhill/librdkafka
1
2

3.进入到librdkafka，然后进行编译

cd librdkafka
yum install -y gcc gcc-c++ pcre-devel zlib-devel
./configure
make && make install
1
2
3
4

4.安装nginx整合kafka的插件，进入到/usr/local/src，clone nginx整合kafka的源码

cd /usr/local/src
git clone https://github.com/brg-liuwei/ngx_kafka_module
1
2

5.进入到nginx的源码包目录下 （编译nginx，然后将将插件同时编译）

cd /usr/local/src/nginx-1.12.2
./configure --add-module=/usr/local/src/ngx_kafka_module/
make
make install
1
2
3
4

6.修改nginx的配置文件，详情请查看当前目录的nginx.conf

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    #gzip  on;
    
    kafka;
    kafka_broker_list  (kafka集群地址)例如:node1:9092,node2:9092,node3:9092; 	
    
    server {
        listen       80;
        server_name  localhost;
        #charset koi8-r;
        #access_log  logs/host.access.log  main;

    	location = /kafka/user {
                kafka_topic user;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

7.启动zk和kafka集群(创建topic)

bin/kafka-topics.sh --create --zookeeper (zookeeper集群) --replication-factor (副本数) --partitions (分区数) --topic user
1

8.启动nginx，报错，找不到kafka.so.1的文件

error while loading shared libraries: librdkafka.so.1: cannot open shared object file: No such file or directory
1

原因是没有加载库编译
9.加载so库

echo "/usr/local/lib" >> /etc/ld.so.conf
ldconfig
1
2

10 再次启动nginx,测试时需要访问location配置的地址，开启kafka消费者测试