devbox
空白诗007
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

k8s部署rocketmq单节点（server+broker+console）—— 筑梦之路_kubesphere rocketmq单节点

作者：空白诗007 | 2024-08-22 04:57:54

赞

踩

kubesphere rocketmq单节点

之前写过几篇关于rocketmq构建双架构镜像、docker-compose部署的文章，接上篇docker-compose部署rocketmq，这里记录下k8s下如何部署。

多架构环境下docker-compose部署rocketmq单机模式—— 筑梦之路-CSDN博客

nameserver节点

  1. apiVersion: apps/v1
  2. kind: StatefulSet
  3. metadata:
  4.   name: rocketmq-nameserv
  5.   labels: {app: rocketmq}
  6.   namespace: merry
  7. spec:
  8.   serviceName: rocketmq
  9.   replicas: 1
  10.   selector:
  11.     matchLabels: {app: rocketmq}
  12.   template:
  13.     metadata:
  14.       labels: {app: rocketmq}
  15.     spec:
  16.       affinity:
  17.         nodeAffinity:
  18.            preferredDuringSchedulingIgnoredDuringExecution:
  19.            - weight: 1
  20.              preference:
  21.                matchExpressions:
  22.                - key: appnode
  23.                  operator: In
  24.                  values:
  25.                  - true
  26.       containers:
  27.       - name: rocketmq
  28.         image: harbor.codemiracle.com.cn/baseapp/rocketmq:4.7.1
  29.         imagePullPolicy: IfNotPresent
  30.         command: ["/home/rocketmq/rocketmq-4.7.1/bin/mqnamesrv"]
  31.         resources:
  32.           limits: {cpu: 5000m, memory: 2000Mi}
  33.           requests: {cpu: 1000m, memory: 2000Mi}
  34.         env:
  35.         - {name: TZ, value: Asia/Shanghai}
  36.         #- {name: JAVA_OPT, value: -Duser.home=/opt}
  37.         #- {name: JAVA_OPT_EXT, value: -server -Xms1900m -Xmx1900m -Xmn1024m}
  38.         ports:
  39.         - {containerPort: 9876}
  40.         volumeMounts:
  41.         - {name: vol-init-d, mountPath: /home/rocketmq/logs}
  42.   volumeClaimTemplates:
  43.   - metadata: {name: vol-init-d}
  44.     spec:
  45.       accessModes: [ReadWriteOnce]
  46.       resources:
  47.         requests: {storage: 1000Gi}
  48.       storageClassName: custom-hostpath
  49. ---
  50. apiVersion: v1
  51. kind: Service
  52. metadata:
  53.   name: rocketmq-svc
  54.   labels:
  55.     app: rocketmq
  56.   namespace: merry
  57. spec:
  58.   type: NodePort
  59.   ports:
  60.     - port: 9876
  61.       targetPort: 9876
  62.       nodePort: 9876
  63.   selector:
  64.     app: rocketmq

broker节点

  1. ---
  2. apiVersion: v1
  3. kind: ConfigMap
  4. metadata:
  5.   name: broker-config
  6.   namespace: merry
  7. data:
  8.   broker.conf: |
  9.         #所属集群名字
  10.         brokerClusterName=DefaultCluster
  11.         #broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
  12.         #在broker-b.properties使用:broker-b
  13.         brokerName=broker-a
  14.         #0 表示Master，>0 表示Slave
  15.         brokerId=0
  16.         #nameServer地址，分号分割
  17.         #namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
  18.         namesrvAddr=rocketmq-svc:9876
  19.         #启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException:         connect to <192.168.0.120:10909> failed
  20.         # 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，        不要使用docker 内部IP
  21.         #brokerIP1=192.168.0.254
  22.         brokerIP1=192.168.100.100
  23.         #在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
  24.         defaultTopicQueueNums=4
  25.         #是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
  26.         #原因下篇博客见~ 哈哈哈哈
  27.         autoCreateTopicEnable=true
  28.         #是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
  29.         autoCreateSubscriptionGroup=true
  30.         #Broker 对外服务的监听端口
  31.         listenPort=10911
  32.         #haService中使用  默认值为：listenPort + 1
  33.         haListenPort=10912
  34.         #主要用于slave同步master  listenPort - 2
  35.         fastListenPort=10909
  36.         #删除文件时间点，默认凌晨4点
  37.         deleteWhen=04
  38.         #文件保留时间，默认48小时
  39.         fileReservedTime=48
  40.         #commitLog每个文件的大小默认1G
  41.         mapedFileSizeCommitLog=1073741824
  42.         #ConsumeQueue每个文件默认存30W条，根据业务情况调整
  43.         mapedFileSizeConsumeQueue=300000
  44.         #destroyMapedFileIntervalForcibly=120000
  45.         #redeleteHangedFileInterval=120000
  46.         #检测物理文件磁盘空间
  47.         diskMaxUsedSpaceRatio=88
  48.         #存储路径
  49.         storePathRootDir=/home/rocketmq/store
  50.         #commitLog 存储路径
  51.         #storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
  52.         #消费队列存储
  53.         #storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
  54.         #消息索引存储路径
  55.         #storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
  56.         #checkpoint 文件存储路径
  57.         #storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
  58.         #abort 文件存储路径
  59.         #abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
  60.         #限制的消息大小
  61.         maxMessageSize=4194304
  62.         #flushCommitLogLeastPages=4
  63.         #flushConsumeQueueLeastPages=2
  64.         #flushCommitLogThoroughInterval=10000
  65.         #flushConsumeQueueThoroughInterval=60000
  66.         #Broker 的角色
  67.         #- ASYNC_MASTER 异步复制Master
  68.         #- SYNC_MASTER 同步双写Master
  69.         #- SLAVE
  70.         brokerRole=ASYNC_MASTER
  71.         #刷盘方式
  72.         #- ASYNC_FLUSH 异步刷盘
  73.         #- SYNC_FLUSH 同步刷盘
  74.         flushDiskType=ASYNC_FLUSH
  75.         #发消息线程池数量
  76.         sendMessageThreadPoolNums=64
  77.         #拉消息线程池数量
  78.         pullMessageThreadPoolNums=64
  79.         #消息发送队列等待时间，默认200
  80.         waitTimeMillsInSendQueue=400
  81.         #发送消息是否使用可重入锁
  82.         useReentrantLockWhenPutMessage=true
  83. ---
  84. apiVersion: v1
  85. kind: Service
  86. metadata:
  87.   name: rmqbroker
  88.   labels:
  89.     app: rmqbroker
  90.   namespace: merry
  91. spec:
  92.   type: NodePort
  93.   ports:
  94.     - port: 10909
  95.       targetPort: 10909
  96.       name: slave-master
  97.       nodePort: 10909
  98.     - port: 10911
  99.       targetPort: 10911
  100.       name: listen
  101.       nodePort: 10911
  102.     - port: 10912
  103.       targetPort: 10912
  104.       name: ha-service
  105.       nodePort: 10912
  106.   selector:
  107.     app: rmqbroker
  108. ---
  109. apiVersion: apps/v1
  110. kind: StatefulSet
  111. metadata:
  112.   name: rmqbroker
  113.   labels: {app: rmqbroker}
  114.   namespace: merry
  115. spec:
  116.   serviceName: rmqbroker
  117.   replicas: 1
  118.   selector:
  119.     matchLabels: {app: rmqbroker}
  120.   template:
  121.     metadata:
  122.       labels: {app: rmqbroker}
  123.     spec:
  124.       affinity:
  125.         nodeAffinity:
  126.            preferredDuringSchedulingIgnoredDuringExecution:
  127.            - weight: 1
  128.              preference:
  129.                matchExpressions:
  130.                - key: rmbroker0node
  131.                  operator: In
  132.                  values:
  133.                  - true    
  134.       containers:
  135.       - name: rmqbroker
  136.         image: harbor.codemiracle.com.cn/baseapp/rocketmq:4.7.1
  137.         imagePullPolicy: IfNotPresent
  138.         command: ["/bin/sh","-c","/home/rocketmq/rocketmq-4.7.1/bin/mqbroker -c /etc/rocketmq/broker.conf"]
  139.         resources:
  140.           limits: {cpu: 4000m, memory: 8Gi}
  141.           requests: {cpu: 1000m, memory: 2Gi}
  142.         env:
  143.         - {name: TZ, value: Asia/Shanghai}
  144.         - {name: JAVA_OPT_EXT, value: -Duser.home=/opt -server -Xms1536m -Xmx4096m}
  145.         #command: [/bin/bash, mqbroker, -c, /etc/rocketmq/broker.conf, -n, 'rocketmq-svc:9876',
  146.         #  autoCreateTopicEnable=true]
  147.         ports:
  148.         - {containerPort: 10909}
  149.         - {containerPort: 10911}
  150.         - {containerPort: 10912}
  151.         volumeMounts:
  152.         - {name: vol-store, mountPath: /home/rocketmq/store}
  153.         - {name: vol-broker-config, mountPath: /etc/rocketmq}
  154.       volumes:
  155.       - name: vol-broker-config
  156.         configMap: {name: broker-config}
  157.   volumeClaimTemplates:
  158.   - metadata: {name: vol-store}
  159.     spec:
  160.       accessModes: [ReadWriteOnce]
  161.       resources:
  162.         requests: {storage: 1000Gi}
  163.       storageClassName: custom-hostpath

console客户端

  1. kind: ConfigMap
  2. apiVersion: v1
  3. metadata:
  4.   name: rocketmq-console-cm
  5.   namespace: merry
  6. data:
  7.   users.properties: |+
  8.     # 该文件支持热修改，即添加和修改用户时，不需要重新启动console
  9.     # 格式， 每行定义一个用户， username=password[,N]  #N是可选项，可以为0 (普通用户)； 1 （管理员）  
  10.     
  11.     #定义管理员 
  12.     admin=admin123,1
  13.     
  14.     #定义普通用户
  15.     #user1=user1
  16.     #user2=user2
  17.   
  18.   role-permission.yml: |+
  19.     # 该文件支持热修改，即添加和修改用户时，不需要重新启动console
  20.     # 格式，如果增加和删除接口权限，直接在列表中增加和删除接口地址即可。
  21.     # 接口路径配置支持通配符
  22.     # * 表示匹配0或多个不是/的字符
  23.     # ** 表示匹配0或多个任意字符
  24.     # ? 表示匹配1个任意字符
  25.     rolePerms:
  26.       # 普通用户
  27.       ordinary:
  28.         - /rocketmq/nsaddr
  29.         - /ops/*
  30.         - /dashboard/**
  31.         - /topic/*.query
  32.         - /topic/sendTopicMessage.do
  33.         - /producer/*.query
  34.         - /message/*
  35.         - /messageTrace/*
  36.         - /monitor/*
  37. ---
  38. apiVersion: apps/v1
  39. kind: Deployment
  40. metadata:
  41.   labels:
  42.     run: rocketmq-console
  43.   name: rocketmq-console-deploy
  44.   namespace: merry
  45. spec:
  46.   replicas: 1
  47.   selector:
  48.     matchLabels:
  49.       run: rocketmq-console
  50.   template:
  51.     metadata:
  52.       labels:
  53.         run: rocketmq-console
  54.     spec:
  55.       containers:
  56.         - name: rocketmq-console
  57.           image: harbor.codemiracle.com.cn/baseapp/rocketmq-console:2.0.0
  58.           env:
  59.             - name: TZ
  60.               value: Aisa/Shanghai
  61.             - name: JAVA_OPTS
  62.               value: "-Drocketmq.namesrv.addr=rocketmq-svc:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Drocketmq.config.loginRequired=true -Drocketmq.config.dataPath=/tmp/rocketmq-console/data"
  63.           volumeMounts:
  64.             - mountPath: /tmp/rocketmq-console/data/users.properties
  65.               name: rocketmq-console-cm
  66.               subPath: users.properties
  67.             - mountPath: /tmp/rocketmq-console/data/role-permission.yml
  68.               name: rocketmq-console-cm
  69.               subPath: role-permission.yml
  70.       volumes:
  71.         - name: rocketmq-console-cm
  72.           configMap:
  73.             name: rocketmq-console-cm
  74. ---
  75. apiVersion: v1
  76. kind: Service
  77. metadata:
  78.   labels:
  79.     run: rocketmq-console
  80.   name: rocketmq-console
  81.   namespace: merry
  82. spec:
  83.   type: NodePort
  84.   ports:
  85.     - name: http
  86.       protocol: TCP
  87.       targetPort: 8080
  88.       port: 8080
  89.       nodePort: 18080
  90.   selector:
  91.     run: rocketmq-console

这里主要以rocketmq 4.7.1为例，其他版本类似。

-----------------------------------------------------------------------

FAQ

2023年12月4日 经测试，以上部署会存在无法写入和消费的问题，使用docker-compose部署正常，测试过程如下：

  1. # 进入broker节点容器内
  2.  
  3. # 发送消息
  4.  
  5. export NAMESRV_ADDR=rocketmq-svc:9876 
  6.  
  7. sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
  8.  
  9. # 接收消息
  10.  
  11. sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer 
  12.  
  13.  
  14. # 如果能正常生产 消费则正常，反之则反。

 如何解决该问题？

更换镜像为king019/rocketmq:4.8.0

docker hub地址：https://hub.docker.com/r/king019/rocketmq

修改deploy yaml 中的启动命令

  1. # nameserver
  2.  
  3. command: ["/opt/soft/rocketmq/bin/mqnamesrv"]
  4.  
  5. # broker
  6.  
  7. command: ["/bin/sh","-c","/opt/soft/rocketmq/bin/mqbroker -c /etc/rocketmq/broker.conf"]

添加环境变量

- {name: JAVA_HOME, value: /usr/lib/jvm/java-1.8-openjdk}

 修改挂载日志路径

/root/logs/rocketmqlogs/

然后再次测试，均能正常使用。 

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/空白诗007/article/detail/1014780
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号