OceanBase OLAP collation utf8mb4_bin 优先

在大数据系统中，如无特别需要，建议 collation 指定为 utf8mb4_bin。

utf8mb4_bin是一种二进制的排序规则，比较字符串时直接比较字符串的二进制值，不需要进行复杂的字符比较和排序运算，这样可以有效减少CPU的使用，提高查询效率，特别是在涉及到大量数据操作时，性能优势更为明显。

在 AP 场景使用 OceanBase MySQL 租户模式时你可以在租户级别设置这两个值作为默认配置，以获得最佳的默认性能：

set global collation_connection = utf8mb4_bin;
set global collation_server = utf8mb4_bin;
1
2

utf8mb4 是编码格式，设置 collation 为 bin 不会影响内存的存储格式，不会造成字符集不兼容等问题，它仅仅影响排序过程中的排序规则。

使用 bin 排序和不使用 bin 排序的详细对比如下，可以看到，使用 bin 排序，性能可以提升 15%。按照我的经验，在实际应用场景中，性能提升比例可能更大！

OceanBase(admin@test)>show variables like 'collation_server';
+------------------+--------------------+
| Variable_name    | Value              |
+------------------+--------------------+
| collation_server | utf8mb4_general_ci |
+------------------+--------------------+
1 row in set (0.007 sec)


OceanBase(admin@test)>create table t1 (c1 bigint primary key, c2 varchar(10));
Query OK, 0 rows affected (0.102 sec)


OceanBase(admin@test)>create table t2 (c1 bigint primary key, c2 varchar(10)) charset=utf8mb4 collate=utf8mb4_bin;
Query OK, 0 rows affected (0.089 sec)

OceanBase(admin@test)>show create table t1\G
*************************** 1. row ***************************
       Table: t1
Create Table: CREATE TABLE `t1` (
  `c1` bigint(20) NOT NULL,
  `c2` varchar(10) DEFAULT NULL,
  PRIMARY KEY (`c1`)
) DEFAULT CHARSET = utf8mb4 ROW_FORMAT = DYNAMIC COMPRESSION = 'zstd_1.3.8' REPLICA_NUM = 2 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0
1 row in set (0.011 sec)

OceanBase(admin@test)>show create table t2\G
*************************** 1. row ***************************
       Table: t2
Create Table: CREATE TABLE `t2` (
  `c1` bigint(20) NOT NULL,
  `c2` varchar(10) COLLATE utf8mb4_bin DEFAULT NULL,
  PRIMARY KEY (`c1`)
) DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = DYNAMIC COMPRESSION = 'zstd_1.3.8' REPLICA_NUM = 2 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0
1 row in set (0.011 sec)

OceanBase(admin@test)>insert /*+ append enable_parallel_dml parallel(4) */ into t1 select random(), randstr(1, 6) from table(generator(1000000));
Query OK, 1000000 rows affected (8.629 sec)
Records: 1000000  Duplicates: 0  Warnings: 0

OceanBase(admin@test)>insert /*+ append enable_parallel_dml parallel(4) */ into t2 select * from t1;
Query OK, 1000000 rows affected (7.115 sec)
Records: 1000000  Duplicates: 0  Warnings: 0

OceanBase(admin@test)>select * from t1 order by c2 limit 999998, 1;
+----------------------+------+
| c1                   | c2   |
+----------------------+------+
| -8614823888367694260 | 5    |
+----------------------+------+
1 row in set (1.050 sec)

OceanBase(admin@test)>select * from t2 order by c2 limit 999998, 1;
+----------------------+------+
| c1                   | c2   |
+----------------------+------+
| -8614823888367694260 | 5    |
+----------------------+------+
1 row in set (0.876 sec)

OceanBase(admin@test)>select * from t1 order by c2 limit 999998, 1;
+----------------------+------+
| c1                   | c2   |
+----------------------+------+
| -8614823888367694260 | 5    |
+----------------------+------+
1 row in set (1.053 sec)

OceanBase(admin@test)>select * from t2 order by c2 limit 999998, 1;
+----------------------+------+
| c1                   | c2   |
+----------------------+------+
| -8614823888367694260 | 5    |
+----------------------+------+
1 row in set (0.844 sec)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

通过上面的对比可以看到，由于 t2 表使用的 collation 为 utf8mb4_bin，其排序耗时只需要0.844秒，使用了 utf8mb4_general_c1 的 t1 表排序耗时则需要 1.053 秒，差距明显。

在 Oracle 模式下，我们通过设置变量 NLS_SORT 为 binary 也可以达到相同效果。默认情况下，NLS_SORT 已经被设置为 binary。