devbox
不正经
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

doris部署记录

作者：不正经 | 2024-05-26 04:52:46

赞

踩

doris部署记录

文章目录

一、下载

  • 地址：https://doris.apache.org/download

二、服务器
服务器192.168.226.20192.168.226.21192.168.226.22
master✅❎❎
follower✅✅✅
observer❎❎❎
四、部署前置工作

  • 验证java版本>=1.8

    java -version

  • 创建目录/home/soft上传apache-doris-2.0.6-bin-x64.tar.gz到20/21/22服务器并解压。

    mkdir -r /home/soft

    rz上传doris文件tar包

    tar -zxvf apache-doris-2.0.6-bin-x64.tar.gz解压缩

    mv apache-doris-2.0.6-bin-x64 doris-2.0.6重命名

  • 修改fe配置文件

    192.168.226.20:cd doris-2.0.6 | vim conf/fe.conf配置priority_networks = 192.168.226.20/24

    192.168.226.21:cd doris-2.0.6 | vim conf/fe.conf配置priority_networks = 192.168.226.21/24

    192.168.226.22:cd doris-2.0.6 | vim conf/fe.conf配置priority_networks = 192.168.226.22/24

  • 修改fe默认端口避免冲突

    ##默认端口：
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
##修改后端口：
http_port = 18030
rpc_port = 19020
query_port = 19030
edit_log_port = 19010

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

  • 修改be配置文件

    192.168.226.20:cd doris-2.0.6 | vim conf/be.conf配置priority_networks = 192.168.226.20/24

    192.168.226.21:cd doris-2.0.6 | vim conf/be.conf配置priority_networks = 192.168.226.21/24

    192.168.226.22:cd doris-2.0.6 | vim conf/be.conf配置priority_networks = 192.168.226.22/24

  • 默认fe元数据存储位置（选配）

    meta_dir = ${DORIS_HOME}/doris-meta

  • 修改be默认端口避免冲突

    ##默认端口
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
##修改后端口
be_port = 19060
webserver_port = 18040
heartbeat_service_port = 19050
brpc_port = 18060

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

  • 默认数据存储位置（选配）

    storage_root_path = ${DORIS_HOME}/storage

五、FE启动

  • 启动master

    ./bin/start_fe.sh --daemon启动脚本

    curl http://127.0.0.1:18030/api/bootstrap{“msg”:“success”,“code”:0,“data”:{“replayedJournalId”:0,“queryPort”:0,“rpcPort”:0,“version”:“”},“count”:0}则启动成功

  • 增加两个从节点follower

    #一、在主节点或者客户端登陆FE mysql -h192.168.226.20 -P19030
ALTER SYSTEM ADD FOLLOWER "192.168.226.21:19010";
ALTER SYSTEM ADD FOLLOWER "192.168.226.22:19010";

#在从节点启动完成后查看前端状态,显示为如下状态则为启动成功
SHOW FRONTENDS;
--------------------------------
|Role	    |IsMaster |Join |Alive|
|FOLLOWER	|false	  |true |true |
|FOLLOWER	|false	  |true |true |
|FOLLOWER	|true		  |true |true |
---------------------------------

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12

  • 启动从节点

    • 第一次启动从节点为了和主节点保持联系需要增加–helper 192.168.226.20:19010

      192.168.226.21:./bin/start_fe.sh --helper 192.168.226.20:19010 --daemon
      192.168.226.22:./bin/start_fe.sh --helper 192.168.226.20:19010 --daemon

六、BE启动

  • 启动be

    ./bin/start_be.sh --daemon

  • 添加后端到FE中

    mysql -h192.168.226.20 -P19030
alter system add backend "192.168.226.20:19050";
alter system add backend "192.168.226.21:19050";
alter system add backend "192.168.226.22:19050";

##Alive : true表示节点运行正常
SHOW  backends;
--------------------------------------------
｜Host	         ｜HLastHeartbeat	    ｜Alive｜
｜192.168.226.20｜2024-03-26 10:02:45	｜true ｜
｜192.168.226.21｜2024-03-26 10:02:45	｜true ｜
｜192.168.226.22｜2024-03-26 10:02:45	｜true ｜
----------------------------------------------


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14

  • 启动21/22BE

    ./bin/start_be.sh --daemon

七、简单测试

  • 创建数据表

    create database demo;
use demo;


CREATE TABLE IF NOT EXISTS demo.example_tbl
(
    `user_id` LARGEINT NOT NULL COMMENT "用户id",
    `date` DATE NOT NULL COMMENT "数据灌入日期时间",
    `city` VARCHAR(20) COMMENT "用户所在城市",
    `age` SMALLINT COMMENT "用户年龄",
    `sex` TINYINT COMMENT "用户性别",
    `last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
    `cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
    `max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
    `min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
PROPERTIES (
    "replication_allocation" = "tag.location.default: 1"
);

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21

  • 上传文件

    cd /home/soft/doris-2.0.6
rz
##示例数据test.csv文件
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
##文件所在目录执行：
curl  --location-trusted -u root: -T test.csv -H "column_separator:," http://127.0.0.1:18030/api/demo/example_tbl/_stream_load
#执行成功返回如下信息
{
    "TxnId": 30303,
    "Label": "8690a5c7-a493-48fc-b274-1bb7cd656f25",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 7,
    "NumberLoadedRows": 7,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 399,
    "LoadTimeMs": 381,
    "BeginTxnTimeMs": 3,
    "StreamLoadPutTimeMs": 5,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 191,
    "CommitAndPublishTimeMs": 175
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31

  • 我们上面完成了建表，输数据导入，下面我们就可以体验 Doris 的数据快速查询分析能力。

    mysql> select * from example_tbl;
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10000   | 2017-10-01 | 北京   |   20 |    0 | 2017-10-01 07:00:00 |   35 |             10 |              2 |
| 10001   | 2017-10-01 | 北京   |   30 |    1 | 2017-10-01 17:05:45 |    2 |             22 |             22 |
| 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
| 10003   | 2017-10-02 | 广州   |   32 |    0 | 2017-10-02 11:20:00 |   30 |             11 |             11 |
| 10004   | 2017-10-01 | 深圳   |   35 |    0 | 2017-10-01 10:00:15 |  100 |              3 |              3 |
| 10004   | 2017-10-03 | 深圳   |   35 |    0 | 2017-10-03 10:20:22 |   11 |              6 |              6 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
6 rows in set (0.02 sec)

mysql> select * from example_tbl where city='上海';
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
1 row in set (0.05 sec)

mysql> select city, sum(cost) as total_cost from example_tbl group by city;
+--------+------------+
| city   | total_cost |
+--------+------------+
| 广州   |         30 |
| 上海   |        200 |
| 北京   |         37 |
| 深圳   |        111 |
+--------+------------+
4 rows in set (0.05 sec)

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
八、运维

  • jps检查各个服务器服务状态

    (base) [root@centos3 doris]# jps

    11819 DorisFE

    19034 DorisBE

  • 服务启动与关闭

    cd /home/soft/doris-2.0.6

    FE:

    ./bin/stop_fe.sh

    ./bin/start_fe.sh

    BE:

    ./bin/stop_be.sh

    ./bin/start_be.sh

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/不正经/article/detail/625234
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号