EMQ安装及集群搭建(简易版)_emqx 集群

简介：

EMQ X (Erlang/Enterprise/Elastic MQTT Broker) 是基于 Erlang/OTP 平台开发的开源物联网 MQTT 消息服务器。Erlang/OTP 最初是爱立信为开发电信设备系统设计的编程语言平台，电信设备(路由器、接入网关等)典型设计是通过背板连接主控板卡与多块业务板卡的分布式系统。

Erlang/OTP是出色的**软实时 (Soft-Realtime)、低延时 (Low-Latency)、分布式 (Distributed)**的语言平台。

MQTT 是轻量的 (Lightweight)、发布订阅模式 (PubSub) 的物联网消息协议。

EMQ X 设计目标是实现高可靠，并支持承载海量物联网终端的MQTT连接，支持在海量物联网设备间低延时消息路由:

1. 稳定承载大规模的 MQTT 客户端连接，单服务器节点支持50万到100万连接。
2. 分布式节点集群，快速低延时的消息路由，单集群支持1000万规模的路由。
3. 消息服务器内扩展，支持定制多种认证方式、高效存储消息到后端数据库。
4. 完整物联网协议支持，MQTT、MQTT-SN、CoAP、LwM2M、WebSocket 或私有协议支持。

拉取镜像

docker pull registry.cn-hangzhou.aliyuncs.com/synbop/emqttd:2.3.6
1

运行镜像
–name 名字
-p 18083 服务器启动端口
-p 1882 TCP端口
-p 8083 WS端口
-p 8084 WSS端口
-p 8883 SSL端口
-d 指定容器

docker run --name emq -p 18083:18083 -p 1883:1883 -p 8084:8084 -p 8883:8883 -p 8083:8083 -d registry.cn-hangzhou.aliyuncs.com/synbop/emqttd:2.3.6


[root@livetech mnt]# docker cp emq:/opt/emqttd/etc/emq.conf /mnt/emq/confing/emq.conf
invalid output path: directory "/mnt/emq/confing" does not exist
[root@livetech mnt]# mkdir /mnt/emq/confing
mkdir: cannot create directory ‘/mnt/emq/confing’: No such file or directory
[root@livetech mnt]# mkdir /mnt/emq/
[root@livetech mnt]# mkdir /mnt/emq/confing
[root@livetech mnt]# docker cp emq:/opt/emqttd/etc/emq.conf /mnt/emq/confing/emq.conf
[root@livetech mnt]# mkdir /mnt/emq/log
[root@livetech mnt]# docker cp emq:/opt/emqttd/log /mnt/emq/log/




docker run --name emq -p 18083:18083 -p 1883:1883 -p 8084:8084 -p 8883:8883 -p 8083:8083
-v /mnt/emq/confing/emq.conf:/opt/emqttd/etc/emq.conf -v /mnt/emq/log/:/opt/emqttd/log  -d registry.cn-hangzhou.aliyuncs.com/synbop/emqttd:2.3.6

docker run -p 18083:18083 \
	-p 1883:1883 \
	-p 8084:8084 \
	-p 8883:8883 \
	-p 8183:8083 \
	--name emq \
	--restart unless-stopped \
	-v /mnt/emq/confing/emq.conf:/opt/emqttd/etc/emq.conf \
	-v /mnt/emq/log/:/opt/emqttd/log \
    -d registry.cn-hangzhou.aliyuncs.com/synbop/emqttd:2.3.6 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

进入emq服务页面
  在浏览器输入机器IP:18083 就可以进入emqtt页面

初始的账户 admin, 密码 public

运行状态

docker stats
1

# 进入容器, 不能用 /bin/bash 进入
docker exec -it emq /bin/sh
1
2

关闭匿名认证(默认是开启的谁都能够登录)

# 编辑配置文件
vi /opt/emqttd/etc/emq.conf 
# 更改允许匿名 True -> false
allow_anonymous = false
1
2
3
4

/opt/emqttd/bin $ ./emqttd_ctl plugins load emq_auth_username
Start apps: [emq_auth_username]
Plugin emq_auth_username loaded successfully.
/opt/emqttd/bin $ emqttd_ctl users add admin Root123...
ok
1
2
3
4
5

添加用户
/bin/emqx_ctl users 
users list                                      # List users
users add <Username> <Password>                 # Add User
users update <Username> <NewPassword>           # Update User
users del <Username>                            # Delete User
1
2
3
4
5
6

查询用户列表
emqttd_ctl users list
1
2

EMQ集群 https://askemq.com/t/topic/2504

https://mcxiaoke.gitbooks.io/mqtt-cn/content/mqtt/05-Security.html

docker run -p 18083:18083
-p 1883:1883
-p 8084:8084
-p 8883:8883
-p 8083:8083
–name emq
–restart unless-stopped
-v /mnt/emq/confing/emq.conf:/opt/emqttd/etc/emq.conf
-v /mnt/emq/log/:/opt/emqttd/log
-d registry.cn-hangzhou.aliyuncs.com/synbop/emqttd:2.3.6

——————————————————————————————————————————————

最新版emqx/emqx

[root@localhost ~]# docker pull emqx/emqx

[root@localhost ~]# docker load -i emqx-patulous.tar 
1ad27bdd166b: Loading layer [==================================================>]  5.865MB/5.865MB
db159a8af725: Loading layer [==================================================>]   7.68kB/7.68kB
88c2cb501968: Loading layer [==================================================>]  69.28MB/69.28MB
f6c9ab062868: Loading layer [==================================================>]  10.24kB/10.24kB
da29e80a3968: Loading layer [==================================================>]  7.749MB/7.749MB
5f70bf18a086: Loading layer [==================================================>]  1.024kB/1.024kB
97f008ad2c1d: Loading layer [==================================================>]   12.8kB/12.8kB
b978a35504a1: Loading layer [==================================================>]  69.28MB/69.28MB
Loaded image: emqx/emqx:latest
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
mysql        latest    40b83de8fb1a   4 weeks ago    535MB
emqx/emqx    latest    eb0d480e054f   3 months ago   148MB
[root@localhost ~]# 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

启动

docker run -d --name emqx \
-p 1883:1883 \
-p 8081:8081 \
-p 8183:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
emqx/emqx
1
2
3
4
5
6
7
8

创建emqx挂载目录

mkdir -p /mnt/emqx/{bin,etc,lib,data,log}
1

将临时emqx容器文件目录拷贝至创建的挂载目录中

docker cp emqx:/opt/emqx/bin /mnt/emqx
docker cp emqx:/opt/emqx/etc /mnt/emqx
docker cp emqx:/opt/emqx/lib /mnt/emqx
docker cp emqx:/opt/emqx/data /mnt/emqx
docker cp emqx:/opt/emqx/log /mnt/emqx

1
2
3
4
5
6

修改挂载目录权限

chown -R 1000:1000 /mnt/emqx/
chmod -R 755 /mnt/emqx/

1
2
3

删除临时emqx容器

docker rm -f emqx
1

重新启动新的emqx挂载配置目录

docker run -d --restart=always  --privileged=true  --name emqx \
-p 1883:1883 \
-p 8081:8081 \
-p 8083:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
-v /mnt/emqx/bin:/opt/emqx/bin \
-v /mnt/emqx/etc:/opt/emqx/etc \
-v /mnt/emqx/lib:/opt/emqx/lib \
-v /mnt/emqx/data:/opt/emqx/data \
-v /mnt/emqx/log:/opt/emqx/log \
emqx/emqx

1
2
3
4
5
6
7
8
9
10
11
12
13
14

目录结构说明

EMQX 默认开启的 MQTT 服务 TCP 端口

关闭匿名认证(默认是开启的谁都能够登录)

[root@livetech etc]# vim emqx.conf 
[root@livetech etc]# pwd
/mnt/emqx/etc
[root@livetech etc]# 


## Value: true | false | false_quick_deny
allow_anonymous = false

## Allow or deny if no ACL rules matched.

1
2
3
4
5
6
7
8
9
10
11

预设认证用户

编辑emqx_auth_mnesia.conf配置文件

## Examples
auth.client.1.clientid = admin
auth.client.1.password = Root123...

## Examples:
auth.user.1.username = admin
auth.user.1.password = Root123...

提示

预设认证数据在配置文件中使用了明文密码，出于安全性与可维护性考虑应当避免使用该功能

1
2
3
4
5
6
7
8
9
10
11
12
13
14

使用-f选项强制删除，即docker rmi -f image-id.

如果网速不好可以上传镜像

命令:

docker save 镜像名/镜像ID -o 镜像保存在哪个位置与名字
1
exmaple:

查看启动日志

[root@livetech ~]# docker logs emqx
Starting emqx on node f0f8786b2450@172.17.0.5
Start mqtt:tcp:internal listener on 127.0.0.1:11883 successfully.
Start mqtt:tcp:external listener on 0.0.0.0:1883 successfully.
Start mqtt:ws:external listener on 0.0.0.0:8083 successfully.
Start mqtt:ssl:external listener on 0.0.0.0:8883 successfully.
Start mqtt:wss:external listener on 0.0.0.0:8084 successfully.
Start http:management listener on 8081 successfully.
2022-11-17T01:31:20.122728+00:00 [warning] [Dashboard] Using default password for dashboard 'admin' user. Please use './bin/emqx_ctl admins' command to change it. NOTE: the default password in config file is only used to initialise the database record, changing the config file after database is initialised has no effect.
Start http:dashboard listener on 18083 successfully.
EMQ X Broker 4.3.18 is running now!
[root@livetech ~]# 

1
2
3
4
5
6
7
8
9
10
11
12
13

emqx开启MySQL权限认证

建立用户和权限的 mysql 表

CREATE DATABASE emqx charset utf8;

use eqmx;

CREATE TABLE mqtt_user ( 
id int(11) unsigned NOT NULL AUTO_INCREMENT, 
username varchar(100) DEFAULT NULL, 
password varchar(100) DEFAULT NULL, 
salt varchar(20) DEFAULT NULL, 
is_superuser tinyint(1) DEFAULT 0, 
created datetime DEFAULT NULL, 
PRIMARY KEY (id), 
UNIQUE KEY mqtt_username (username) 
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

CREATE TABLE mqtt_acl ( 
id int(11) unsigned NOT NULL AUTO_INCREMENT, 
allow int(1) DEFAULT NULL COMMENT '0: deny, 1: allow', 
ipaddr varchar(60) DEFAULT NULL COMMENT 'IpAddress', 
username varchar(100) DEFAULT NULL COMMENT 'Username', 
clientid varchar(100) DEFAULT NULL COMMENT 'ClientId', 
access int(2) NOT NULL COMMENT '1: subscribe, 2: publish, 3: pubsub', 
topic varchar(100) NOT NULL DEFAULT '' COMMENT 'Topic Filter', 
PRIMARY KEY (id) 
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

修改emq的mysql配置文件

[root@localhost plugins]# pwd
/mnt/emqx/etc/plugins

[root@localhost plugins]# vim emqx_auth_mysql.conf 

## Examples: 3306, 127.0.0.1:3306, localhost:3306
auth.mysql.server = 你的mysql-IP:3306

## MySQL pool size.
##
## Value: Number
auth.mysql.pool = 8

## MySQL username.
##
## Value: String
auth.mysql.username = root

## MySQL password.
##
## Value: String
auth.mysql.password = Root123...

## MySQL database.
##
## Value: String
auth.mysql.database = emqx

## MySQL query timeout

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

重启emq

docker restart emqx

1
2

登录web页面开启emqx_auth_mysql 插件or容器启动插件

## 如果在控制台开启插件失败可以尝试在容器内部重启emqx
## 然后再次尝试在控制台开启插件

容器启动

docker exec -it emqx bash

bash-5.0$ pwd
/opt/emqx/bin
bash-5.0$ 



bash-5.0$ emqx_ctl plugins reload emqx_auth_mysql
Plugin emqx_auth_mysql reloaded successfully.
bash-5.0$ 

官方文档说明
当配置文件发生更改，如果需要配置立即生效，你可以执行 emqx_ctl reload <Plugin\> 命令，即使插件在配置修改时并未处于运行状态，你也应当使用此命令而不是emqx_ctl load <Plugin>，因为 `emqx_ctl load <Plugin> 不会编译新的配置文件。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

docker配置emqx 集群部署

190服务

docker run -d --network host --restart=always  --privileged=true  --name emqx \
-p 1883:1883 \
-p 8081:8081 \
-p 8183:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
-e EMQX_NAME="node01"  \
-e EMQX_HOST=192.168.50.190 \
-e EMQX_LISTENER__TCP_EXTERNAL=1883 \
-e EMQX_JOIN_CLUSTER="node01@192.168.50.190" \
-e EMQX_CLUSTER__DISCOVERY="static" \
-e EMQX_CLUSTER__STATIC__SEEDS="node01@192.168.50.190,node02@192.168.50.191" \
-v /mnt/emqx/bin:/opt/emqx/bin \
-v /mnt/emqx/etc:/opt/emqx/etc \
-v /mnt/emqx/lib:/opt/emqx/lib \
-v /mnt/emqx/data:/opt/emqx/data \
-v /mnt/emqx/log:/opt/emqx/log \
emqx/emqx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

docker run -d --restart=always  --privileged=true  --name emqx \
-p 1884:1883 \
-p 8081:8081 \
-p 8183:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
-v /mnt/emqx/bin:/opt/emqx/bin \
-v /mnt/emqx/etc:/opt/emqx/etc \
-v /mnt/emqx/lib:/opt/emqx/lib \
-v /mnt/emqx/data:/opt/emqx/data \
-v /mnt/emqx/log:/opt/emqx/log \
emqx/emqx
1
2
3
4
5
6
7
8
9
10
11
12
13

修改配置文件

#集群名称必须一致
cluster.name = emqxcl
# 静态
cluster.discovery = static

cluster.static.seeds = node01@192.168.50.190,node02@192.168.50.191

node.name = node01@192.168.50.190

1
2
3
4
5
6
7
8
9

重启

[root@localhost ~]# docker restart emqx
emqx
[root@localhost ~]# 

1
2
3
4

191服务

docker run -d --network host --restart=always  --privileged=true  --name emqx \
-p 1883:1883 \
-p 8081:8081 \
-p 8183:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
-e EMQX_NAME="node02"  \
-e EMQX_HOST=192.168.50.191 \
-e EMQX_LISTENER__TCP_EXTERNAL=1883 \
-e EMQX_JOIN_CLUSTER="node02@192.168.50.191" \
-e EMQX_CLUSTER__DISCOVERY="static" \
-e EMQX_CLUSTER__STATIC__SEEDS="node01@192.168.50.190,node02@192.168.50.191" \
-v /mnt/emqx/bin:/opt/emqx/bin \
-v /mnt/emqx/etc:/opt/emqx/etc \
-v /mnt/emqx/lib:/opt/emqx/lib \
-v /mnt/emqx/data:/opt/emqx/data \
-v /mnt/emqx/log:/opt/emqx/log \
emqx/emqx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

docker run -d --restart=always  --privileged=true  --name emqx \
-p 1883:1883 \
-p 8081:8081 \
-p 8183:8083 \
-p 8084:8084 \
-p 8883:8883 \
-p 18083:18083 \
-v /mnt/emqx/bin:/opt/emqx/bin \
-v /mnt/emqx/etc:/opt/emqx/etc \
-v /mnt/emqx/lib:/opt/emqx/lib \
-v /mnt/emqx/data:/opt/emqx/data \
-v /mnt/emqx/log:/opt/emqx/log \
emqx/emqx
1
2
3
4
5
6
7
8
9
10
11
12
13

修改配置文件

[root@localhost ~]# vim /mnt/emqx/etc/emqx.conf 


#集群名称必须一致
cluster.name = emqxcl
# 静态
cluster.discovery = static

cluster.static.seeds = node01@192.168.50.190,node02@192.168.50.191

node.name = node01@192.168.50.191
1
2
3
4
5
6
7
8
9
10
11

重启

[root@localhost ~]# docker restart emqx
emqx
[root@localhost ~]# 

1
2
3
4

查看集群状态

./emqx_ctl cluster  status
1

EMQXzip动态手动安装部署集群

192服务器(198.199按照192配置进行修改，注意修改相应主机名和IP)

unzip emqx-centos7-4.3.18-amd64.zip 

# 修改配置文件
vim etc/emqx.conf

#集群名称必须一致
cluster.name = emqxcl
# 修改为本机主机名和IP
node.name = node-01@192.168.50.192 
# 数据存储目录
node.data_dir = data  

1
2
3
4
5
6
7
8
9
10
11
12

192执行集群加入命令

./emqx_ctl cluster join node-02@192.168.50.198
# 查看节点状态
[root@localhost bin]# ./emqx_ctl cluster  status
Erlang/OTP 23 [erts-11.2.2.8] [emqx] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

Eshell V11.2.2.8  (abort with ^G)
1> There seem to be missing dynamic libs from the OS.
Using libs from /root/tools/emqx/dynlibs instead.
NOTE: EMQX's rpm or deb package installation is recommended!
Cluster status: #{running_nodes => ['node-01@192.168.50.192'],
                  stopped_nodes => []}
# 集群加入
[root@localhost bin]# ./emqx_ctl cluster join node-02@192.168.50.198
Erlang/OTP 23 [erts-11.2.2.8] [emqx] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

There seem to be missing dynamic libs from the OS.
Using libs from /root/tools/emqx/dynlibs instead.
NOTE: EMQX's rpm or deb package installation is recommended!
=CRITICAL REPORT==== 23-Nov-2022::11:03:28.058073 ===
[EMQ X] emqx shutdown for join
Join the cluster successfully.
Cluster status: #{running_nodes =>
                      ['node-01@192.168.50.192','node-02@192.168.50.198',
                       'node-03@192.168.50.199'],    
                 stopped_nodes => []}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

198执行集群加入命令

./emqx_ctl cluster join node-03@192.168.50.199
# 集群加入
[root@localhost bin]# ./emqx_ctl cluster join node-03@192.168.50.199
Erlang/OTP 23 [erts-11.2.2.8] [emqx] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

There seem to be missing dynamic libs from the OS.
Using libs from /root/tools/emqx/dynlibs instead.
NOTE: EMQX's rpm or deb package installation is recommended!
=CRITICAL REPORT==== 23-Nov-2022::11:03:28.058073 ===
[EMQ X] emqx shutdown for join
Join the cluster successfully.
Cluster status: #{running_nodes =>
                      ['node-01@192.168.50.192','node-02@192.168.50.198',
                       'node-03@192.168.50.199'],
                 stopped_nodes => []}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

199执行集群加入命令

./emqx_ctl cluster join node-01@192.168.50.192
# 集群加入
[root@localhost bin]# ./emqx_ctl cluster join node-01@192.168.50.192
Erlang/OTP 23 [erts-11.2.2.8] [emqx] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

There seem to be missing dynamic libs from the OS.
Using libs from /root/tools/emqx/dynlibs instead.
NOTE: EMQX's rpm or deb package installation is recommended!
=CRITICAL REPORT==== 23-Nov-2022::11:03:28.058073 ===
[EMQ X] emqx shutdown for join
Join the cluster successfully.
Cluster status: #{running_nodes =>
                      ['node-01@192.168.50.192','node-02@192.168.50.198',
                       'node-03@192.168.50.199'],
                 stopped_nodes => []}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

nginx配置负载

#################################################################################

stream{

  #   emqx tcp
  #  轮询负载均衡配置
      upstream emqx_tcp {
           #hash $remote_addr consistent;
           server 192.168.50.192:1883 weight=1  max_fails=3 fail_timeout=30s;
           server 192.168.50.199:1883 weight=1  max_fails=3 fail_timeout=30s;
           server 192.168.50.198:1883 weight=1  max_fails=3 fail_timeout=30s;
    }

 #      emqx tcp server
        server {
          # 监听 8884 端口
          listen 8884;
          # 反向代理到 emqx_tcp
          proxy_pass emqx_tcp;
            tcp_nodelay on;
    }

}

#################################################################################


#测试nginx配置是否正确
./sbin/nginx -t
#重新加载配置
./sbin/nginx -s reload
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

可能遇到问题总结：

1.端口问题：1883,18083，直接把虚拟机防火墙关了，不用费事在开放端口了
2.没有负载均衡成功，因为开放的8884端口被限制了。执行命令：

getenforce
setenforce 0
1
2