Doris的查询计划

作者：小桥流水78 | 2024-07-28 01:15:29

踩

Doris的查询计划

本文章向大家介绍Doris的查询计划，主要内容包括1、查询计划、2、MySQL查询计划、2、Doris的查询计划、（2）聚合、（3）关联查询、（4）查询 Profile、基本概念、基础应用、原理机制和需要注意的事项等，并结合实例形式分析了其使用技巧，希望通过本文能帮助到大家理解应用这部分内容。

1、查询计划

SQL语句只告诉机器干什么，没有告诉具体怎么干。DBMS内嵌了查询优化器，对用户透明。但是有时候我们写的SQL语言查询很慢，就需要通过查询计划看看机器具体是怎么执行这个SQL的，确定查询慢的瓶颈问题，然后修改SQL进行优化。

2、MySQL查询计划

（1）MySQL使用explain + sql语法查看执行计划


MariaDB [(none)]> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
MariaDB [test]> EXPLAIN SELECT * FROM test.users;
+------+-------------+-------+------+---------------+------+---------+------+------+-------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+-------+
|    1 | SIMPLE      | users | ALL  | NULL          | NULL | NULL    | NULL |    3 |       |
+------+-------------+-------+------+---------------+------+---------+------+------+-------+
1 row in set (0.00 sec)
 
MariaDB [test]>

（2）该查询类型select_type是简单查询SIMPLE,此外还有其他查询类型

SIMPLE ：简单查询
PRIMARY：最外层查询
SUBQUERY：映射为子查询
DERIVED：子查询
UNION：联合
UNION RESULT：使用联合的结果

（3）table字段表示MySQL在表中找到所需行的方式，又称访问类型。

ALL：全数据表扫描
INDEX：全索引表扫描
RANGE：对索引列进行范围查找，常见于between、<、>等的查询
INDEX_MERGE：合并索引，使用多个单列索引搜索
REF：根据索引查找一个或多个值
EQ_REF：扫描时使用primary key 或 unique类型
CONST：const表很快,只读取一次
SYSTEM：system是const类型的特例，表仅有一行。

（4）查询性能

ALL < index < range < index_merge < ref_or_null < ref < eq_ref < system/const

2、Doris的查询计划

分布式查询计划是由多个 Fragment 组成的，每个 Fragment 负责查询计划的一部分，各个 Fragment 直接会通过 ExchangeNode 算子进行数据的传输。

（1）简单查询


MySQL [tpa]> explain graph SELECT * FROM t1;
+-----------------------------------------------------------------+
| Explain String                                                  |
+-----------------------------------------------------------------+
|                                                                 |
|   ┌───────────────┐                                             |
|   │[1: ResultSink]│                                             |
|   │[Fragment: 1]  │                                             |
|   │RESULT SINK    │                                             |
|   └───────────────┘                                             |
|           │                                                     |
|           │                                                     |
|    ┌─────────────┐                                              |
|    │[1: EXCHANGE]│                                              |
|    │[Fragment: 1]│                                              |
|    └─────────────┘                                              |
|           │                                                     |
|           │                                                     |
| ┌───────────────────┐                                           |
| │[1: DataStreamSink]│                                           |
| │[Fragment: 0]      │                                           |
| │STREAM DATA SINK   │                                           |
| │  EXCHANGE ID: 01  │                                           |
| │  UNPARTITIONED    │                                           |
| └───────────────────┘                                           |
|           │                                                     |
|           │                                                     |
|  ┌─────────────────┐                                            |
|  │[0: OlapScanNode]│                                            |
|  │[Fragment: 0]    │                                            |
|  │TABLE: t1        │                                            |
|  └─────────────────┘                                            |
+-----------------------------------------------------------------+
28 rows in set (0.00 sec)
 
MySQL [tpa]> explain SELECT * FROM t1;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String                                                                                                                                              |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                                                                                             |
|  OUTPUT EXPRS:`default_cluster:tpa.t1`.`siteid` | `default_cluster:tpa.t1`.`citycode` | `default_cluster:tpa.t1`.`username` | `default_cluster:tpa.t1`.`pv` |
|   PARTITION: UNPARTITIONED                                                                                                                                  |
|                                                                                                                                                             |
|   RESULT SINK                                                                                                                                               |
|                                                                                                                                                             |
|   1:EXCHANGE                                                                                                                                                |
|                                                                                                                                                             |
| PLAN FRAGMENT 1                                                                                                                                             |
|  OUTPUT EXPRS:                                                                                                                                              |
|   PARTITION: RANDOM                                                                                                                                         |
|                                                                                                                                                             |
|   STREAM DATA SINK                                                                                                                                          |
|     EXCHANGE ID: 01                                                                                                                                         |
|     UNPARTITIONED                                                                                                                                           |
|                                                                                                                                                             |
|   0:OlapScanNode                                                                                                                                            |
|      TABLE: t1                                                                                                                                              |
|      PREAGGREGATION: OFF. Reason: No AggregateInfo                                                                                                          |
|      partitions=1/1                                                                                                                                         |
|      rollup: t1                                                                                                                                             |
|      tabletRatio=10/10                                                                                                                                      |
|      tabletList=439810,439813,439816,439819,439822,439825,439828,439831,439834,439837                                                                       |
|      cardinality=5                                                                                                                                          |
|      avgRowSize=30.0                                                                                                                                        |
|      numNodes=3                                                                                                                                             |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
26 rows in set (0.00 sec)
 
MySQL [tpa]>

（2）聚合


MySQL [tpa]> explain graph SELECT SUM(t1.pv) FROM t1;
+-----------------------------------------------------------------------------------------------------------+
| Explain String                                                                                            |
+-----------------------------------------------------------------------------------------------------------+
|                                                                                                           |
|          ┌───────────────┐                                                                                |
|          │[3: ResultSink]│                                                                                |
|          │[Fragment: 1]  │                                                                                |
|          │RESULT SINK    │                                                                                |
|          └───────────────┘                                                                                |
|                  │                                                                                        |
|                  │                                                                                        |
|  ┌───────────────────────────────┐                                                                        |
|  │[3: AGGREGATE (merge finalize)]│                                                                        |
|  │[Fragment: 1]                  │                                                                        |
|  └───────────────────────────────┘                                                                        |
|                  │                                                                                        |
|                  │                                                                                        |
|           ┌─────────────┐                                                                                 |
|           │[2: EXCHANGE]│                                                                                 |
|           │[Fragment: 1]│                                                                                 |
|           └─────────────┘                                                                                 |
|                  │                                                                                        |
|                  │                                                                                        |
|        ┌───────────────────┐                                                                              |
|        │[2: DataStreamSink]│                                                                              |
|        │[Fragment: 0]      │                                                                              |
|        │STREAM DATA SINK   │                                                                              |
|        │  EXCHANGE ID: 02  │                                                                              |
|        │  UNPARTITIONED    │                                                                              |
|        └───────────────────┘                                                                              |
|                  │                                                                                        |
|                  │                                                                                        |
| ┌─────────────────────────────────┐                                                                       |
| │[1: AGGREGATE (update serialize)]│                                                                       |
| │[Fragment: 0]                    │                                                                       |
| └─────────────────────────────────┘                                                                       |
|                  │                                                                                        |
|                  │                                                                                        |
|         ┌─────────────────┐                                                                               |
|         │[0: OlapScanNode]│                                                                               |
|         │[Fragment: 0]    │                                                                               |
|         │TABLE: t1        │                                                                               |
|         └─────────────────┘                                                                               |
+-----------------------------------------------------------------------------------------------------------+
40 rows in set (0.01 sec)
 
MySQL [tpa]> explain SELECT SUM(t1.pv) FROM t1;
+---------------------------------------------------------------------------------------+
| Explain String                                                                        |
+---------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                       |
|  OUTPUT EXPRS: sum(`t1`.`pv`)                                                         |
|   PARTITION: UNPARTITIONED                                                            |
|                                                                                       |
|   RESULT SINK                                                                         |
|                                                                                       |
|   3:AGGREGATE (merge finalize)                                                        |
|   |  output: sum( sum(`t1`.`pv`))                                                     |
|   |  group by:                                                                        |
|   |  cardinality=-1                                                                   |
|   |                                                                                   |
|   2:EXCHANGE                                                                          |
|                                                                                       |
| PLAN FRAGMENT 1                                                                       |
|  OUTPUT EXPRS:                                                                        |
|   PARTITION: RANDOM                                                                   |
|                                                                                       |
|   STREAM DATA SINK                                                                    |
|     EXCHANGE ID: 02                                                                   |
|     UNPARTITIONED                                                                     |
|                                                                                       |
|   1:AGGREGATE (update serialize)                                                      |
|   |  output: sum(`t1`.`pv`)                                                           |
|   |  group by:                                                                        |
|   |  cardinality=1                                                                    |
|   |                                                                                   |
|   0:OlapScanNode                                                                      |
|      TABLE: t1                                                                        |
|      PREAGGREGATION: ON                                                               |
|      partitions=1/1                                                                   |
|      rollup: t1                                                                       |
|      tabletRatio=10/10                                                                |
|      tabletList=439810,439813,439816,439819,439822,439825,439828,439831,439834,439837 |
|      cardinality=4                                                                    |
|      avgRowSize=8.0                                                                   |
|      numNodes=3                                                                       |
+---------------------------------------------------------------------------------------+
36 rows in set (0.01 sec)
 
MySQL [tpa]>

（3）关联查询


MySQL [tpa]> SELECT SUM(t1.pv) FROM t1 JOIN t2 WHERE t1.siteid = t2.siteid;
+----------------+
| sum(`t1`.`pv`) |
+----------------+
|             12 |
+----------------+
1 row in set (0.29 sec)
 
MySQL [tpa]> SELECT SUM(pv) FROM t2 WHERE siteid IN (SELECT siteid FROM t1 WHERE siteid > 2);
+-----------+
| sum(`pv`) |
+-----------+
|         8 |
+-----------+
1 row in set (0.36 sec)
 
MySQL [tpa]> explain SELECT SUM(t1.pv) FROM t1 JOIN t2 WHERE t1.siteid = t2.siteid;
+---------------------------------------------------------------------------------------+
| Explain String                                                                        |
+---------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                       |
|  OUTPUT EXPRS:<slot 3> sum(`t1`.`pv`)                                                 |
|   PARTITION: UNPARTITIONED                                                            |
|                                                                                       |
|   RESULT SINK                                                                         |
|                                                                                       |
|   6:AGGREGATE (merge finalize)                                                        |
|   |  output: sum(<slot 3> sum(`t1`.`pv`))                                             |
|   |  group by:                                                                        |
|   |  cardinality=-1                                                                   |
|   |                                                                                   |
|   5:EXCHANGE                                                                          |
|                                                                                       |
| PLAN FRAGMENT 1                                                                       |
|  OUTPUT EXPRS:                                                                        |
|   PARTITION: RANDOM                                                                   |
|                                                                                       |
|   STREAM DATA SINK                                                                    |
|     EXCHANGE ID: 05                                                                   |
|     UNPARTITIONED                                                                     |
|                                                                                       |
|   3:AGGREGATE (update serialize)                                                      |
|   |  output: sum(`t1`.`pv`)                                                           |
|   |  group by:                                                                        |
|   |  cardinality=1                                                                    |
|   |                                                                                   |
|   2:HASH JOIN                                                                         |
|   |  join op: INNER JOIN (BUCKET_SHUFFLE)                                             |
|   |  runtime filter: true                                                             |
|   |  hash predicates:                                                                 |
|   |  colocate: false, reason: table not in the same group                             |
|   |  equal join conjunct: `t1`.`siteid` = `t2`.`siteid`                               |
|   |  cardinality=4                                                                    |
|   |                                                                                   |
|   |----4:EXCHANGE                                                                     |
|   |                                                                                   |
|   0:OlapScanNode                                                                      |
|      TABLE: t1                                                                        |
|      PREAGGREGATION: ON                                                               |
|      partitions=1/1                                                                   |
|      rollup: t1                                                                       |
|      tabletRatio=10/10                                                                |
|      tabletList=439810,439813,439816,439819,439822,439825,439828,439831,439834,439837 |
|      cardinality=4                                                                    |
|      avgRowSize=12.0                                                                  |
|      numNodes=3                                                                       |
|                                                                                       |
| PLAN FRAGMENT 2                                                                       |
|  OUTPUT EXPRS:                                                                        |
|   PARTITION: RANDOM                                                                   |
|                                                                                       |
|   STREAM DATA SINK                                                                    |
|     EXCHANGE ID: 04                                                                   |
|     BUCKET_SHFFULE_HASH_PARTITIONED: `t2`.`siteid`                                    |
|                                                                                       |
|   1:OlapScanNode                                                                      |
|      TABLE: t2                                                                        |
|      PREAGGREGATION: OFF. Reason: null                                                |
|      partitions=1/3                                                                   |
|      rollup: t2                                                                       |
|      tabletRatio=10/10                                                                |
|      tabletList=440143,440146,440149,440152,440155,440158,440161,440164,440167,440170 |
|      cardinality=0                                                                    |
|      avgRowSize=4.0                                                                   |
|      numNodes=1                                                                       |
+---------------------------------------------------------------------------------------+
65 rows in set (0.73 sec)
 
MySQL [tpa]> explain graph SELECT SUM(t1.pv) FROM t1 JOIN t2 WHERE t1.siteid = t2.siteid;
+-------------------------------------------------------------------------------------------------------------------------------+
| Explain String                                                                                                                |
+-------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                               |
|                    ┌───────────────┐                                                                                          |
|                    │[6: ResultSink]│                                                                                          |
|                    │[Fragment: 2]  │                                                                                          |
|                    │RESULT SINK    │                                                                                          |
|                    └───────────────┘                                                                                          |
|                            │                                                                                                  |
|                            │                                                                                                  |
|            ┌───────────────────────────────┐                                                                                  |
|            │[6: AGGREGATE (merge finalize)]│                                                                                  |
|            │[Fragment: 2]                  │                                                                                  |
|            └───────────────────────────────┘                                                                                  |
|                            │                                                                                                  |
|                            │                                                                                                  |
|                     ┌─────────────┐                                                                                           |
|                     │[5: EXCHANGE]│                                                                                           |
|                     │[Fragment: 2]│                                                                                           |
|                     └─────────────┘                                                                                           |
|                            │                                                                                                  |
|                            │                                                                                                  |
|                  ┌───────────────────┐                                                                                        |
|                  │[5: DataStreamSink]│                                                                                        |
|                  │[Fragment: 0]      │                                                                                        |
|                  │STREAM DATA SINK   │                                                                                        |
|                  │  EXCHANGE ID: 05  │                                                                                        |
|                  │  UNPARTITIONED    │                                                                                        |
|                  └───────────────────┘                                                                                        |
|                            │                                                                                                  |
|                            │                                                                                                  |
|           ┌─────────────────────────────────┐                                                                                 |
|           │[3: AGGREGATE (update serialize)]│                                                                                 |
|           │[Fragment: 0]                    │                                                                                 |
|           └─────────────────────────────────┘                                                                                 |
|                            │                                                                                                  |
|                            │                                                                                                  |
|         ┌────────────────────────────────────┐                                                                                |
|         │[2: HASH JOIN]                      │                                                                                |
|         │[Fragment: 0]                       │                                                                                |
|         │join op: INNER JOIN (BUCKET_SHUFFLE)│                                                                                |
|         └────────────────────────────────────┘                                                                                |
|          ┌─────────────────┴─────────┐                                                                                        |
|          │                           │                                                                                        |
| ┌─────────────────┐           ┌─────────────┐                                                                                 |
| │[0: OlapScanNode]│           │[4: EXCHANGE]│                                                                                 |
| │[Fragment: 0]    │           │[Fragment: 0]│                                                                                 |
| │TABLE: t1        │           └─────────────┘                                                                                 |
| └─────────────────┘                  │                                                                                        |
|                                      │                                                                                        |
|                     ┌─────────────────────────────────┐                                                                       |
|                     │[4: DataStreamSink]              │                                                                       |
|                     │[Fragment: 1]                    │                                                                       |
|                     │STREAM DATA SINK                 │                                                                       |
|                     │  EXCHANGE ID: 04                │                                                                       |
|                     │  BUCKET_SHFFULE_HASH_PARTITIONED│                                                                       |
|                     └─────────────────────────────────┘                                                                       |
|                                      │                                                                                        |
|                                      │                                                                                        |
|                             ┌─────────────────┐                                                                               |
|                             │[1: OlapScanNode]│                                                                               |
|                             │[Fragment: 1]    │                                                                               |
|                             │TABLE: t2        │                                                                               |
|                             └─────────────────┘                                                                               |
+-------------------------------------------------------------------------------------------------------------------------------+
62 rows in set (0.21 sec)

（4）查询 Profile

Profile 包含查询语句各个节点的具体执行情况，可以帮助我们分析查询瓶颈。

先开启is_report_success


MySQL [tpa]> show query profile "/"G
Empty set (0.01 sec)
 
MySQL [tpa]> SET is_report_success=true;
Query OK, 0 rows affected (0.00 sec)


MySQL [tpa]> show query profile "/"G
Empty set (0.00 sec)
 
MySQL [tpa]> SELECT SUM(t1.pv) FROM t1 JOIN t2 WHERE t1.siteid = t2.siteid;
+----------------+
| sum(`t1`.`pv`) |
+----------------+
|             12 |
+----------------+
1 row in set (2.45 sec)
 
MySQL [tpa]> show query profile "/"G
*************************** 1. row ***************************
   QueryId: 15400462342f4d04-b294791b760d77e7
      User: default_cluster:test
 DefaultDb: default_cluster:tpa
       SQL: SELECT SUM(t1.pv) FROM t1 JOIN t2 WHERE t1.siteid = t2.siteid
 QueryType: Query
 StartTime: 2021-08-26 09:54:04
   EndTime: 2021-08-26 09:54:04
 TotalTime: 118ms
QueryState: EOF
1 row in set (0.00 sec)
 
MySQL [tpa]> show query profile "/15400462342f4d04-b294791b760d77e7"G
*************************** 1. row ***************************
Fragments: 
          ┌────────────────────────┐
          │[-1: DataBufferSender]  │
          │Fragment: 0             │
          │MaxActiveTime: 106.143ms│
          └────────────────────────┘
                       │
                       │
            ┌─────────────────────┐
            │[6: AGGREGATION_NODE]│
            │Fragment: 0          │
            └─────────────────────┘
                       │
                       │
             ┌──────────────────┐
             │[5: EXCHANGE_NODE]│
             │Fragment: 0       │
             └──────────────────┘
                       │
                       │
          ┌────────────────────────┐
          │[5: DataStreamSender]   │
          │Fragment: 1             │
          │MaxActiveTime: 101.428ms│
          └────────────────────────┘
                       │
                       │
            ┌─────────────────────┐
            │[3: AGGREGATION_NODE]│
            │Fragment: 1          │
            └─────────────────────┘
                       │
                       │
             ┌───────────────────┐
             │[2: HASH_JOIN_NODE]│
             │Fragment: 1        │
             └───────────────────┘
          ┌────────────┴──────────┐
          │                       │
┌───────────────────┐   ┌──────────────────┐
│[0: OLAP_SCAN_NODE]│   │[4: EXCHANGE_NODE]│
│Fragment: 1        │   │Fragment: 1       │
└───────────────────┘   └──────────────────┘
          │                       │
          │                       │
   ┌─────────────┐    ┌───────────────────────┐
   │[OlapScanner]│    │[4: DataStreamSender]  │
   │Fragment: 1  │    │Fragment: 2            │
   └─────────────┘    │MaxActiveTime: 79.987ms│
          │           └───────────────────────┘
          │                       │
 ┌─────────────────┐              │
 │[SegmentIterator]│    ┌───────────────────┐
 │Fragment: 1      │    │[1: OLAP_SCAN_NODE]│
 └─────────────────┘    │Fragment: 2        │
                        └───────────────────┘
                                  │
                                  │
                           ┌─────────────┐
                           │[OlapScanner]│
                           │Fragment: 2  │
                           └─────────────┘
                                  │
                                  │
                         ┌─────────────────┐
                         │[SegmentIterator]│
                         │Fragment: 2      │
                         └─────────────────┘
 
1 row in set (0.00 sec)
 
MySQL [tpa]>

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小桥流水78/article/detail/892300