Java微服务篇4——Elastic search_es实现case when

Java微服务篇4——Elastic search

1、Elastic search安装配置

Elastic search官方：https://www.elastic.co/cn/products/elasticsearch

Elastic search6.2.4 百度云：https://pan.baidu.com/s/1JyQok8Nija4gYhcjh-HWcw提取码：isa2

解压即可

在config/elasticsearch.yml修改

# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
1
2
3
4
5
6
7
8
9

启动bin目录下的elasticsearch.bat/elasticsearch.sh

访问

http://localhost:9200/
1

2、kibana安装

Kibana是一个基于Node.js的Elasticsearch索引库数据统计工具，可以利用Elasticsearch的聚合功能， 生成各种图表，如柱形图，线状图，饼图等。 而且还提供了操作Elasticsearch索引数据的控制台

kibana6.2.4 百度云：https://pan.baidu.com/s/1AlD34ZYvHqtVwaapRh3tPA提取码：p8gr

在config/kibana.yml修改

elasticsearch.url: "http://localhost:9200"
1

3、ik分词器

elasticsearch-analysis-ik-6.2.4 百度云：https://pan.baidu.com/s/1L4_fQIgLmLoLpKL8sXAAFw提取码：8k05

解压elasticsearch-analysis-ik-6.2.4.zip后,将解压后的文件夹拷贝到elasticsearch-6.2.4\plugins 下，并重命名文件夹为ik

重启即可

测试

GET /_analyze
{
  "text": "我是中国人"
}
1
2
3
4

GET /_analyze
{
  "analyzer": "ik_max_word",
  "text": "我是中国人"
}
1
2
3
4
5

GET /_analyze
{
  "analyzer": "ik_min_word",
  "text": "我是中国人"
}
1
2
3
4
5

4、elasticsearch-head

elasticsearch-head 百度云：https://pan.baidu.com/s/1JsOvr64gb5xNbkPPJafyag提取码：6khj

5、elastic search操作

映射属性数据类型

index

是否被索引

index的默认值就是true，也就是说你不进行任何配置，所有字段都会被索引，有些字段是我们不希望被索引的，比如商品的图片信息（URL），就需要手动设置index为false

stroe

是否额外存储

在lucene里，如果一个字段的store设置为false，那么在文档列表中就不会有这个字段的 值，用户的搜索结果中不会显示出来，但是在Elasticsearch中，即便store设置为false，也可以搜索到结果

原因是Elasticsearch在创建文档索引时，会将文档中的原始数据备份，保存到一个叫做 _source 的属性 中。而且我们可以通过过滤 _source 来选择哪些要显示，哪些不显示

如果设置store为true，就会在 _source 以外额外存储一份数据，多余，因此一般我们都会将store设 置为false，事实上，store的默认值就是false

在某些情况下，这对 store 某个领域可能是有意义的。例如，如果您的文档包含一个 title ，一个 date 和一个非常大的 content 字段，则可能只想检索the title 和the date 而不必从一个大 _source 字段中提取这些字段

boost

网站权重:网站权重是指搜索引擎给网站（包括网页）赋予一定的权威值，对网站（含网页）权威的评估评价。一 个网站权重越高，在搜索引擎所占的份量越大，在搜索引擎排名就越好。提高网站权重，不但利于网站（包括网 页）在搜索引擎的排名更靠前，还能提高整站的流量，提高网站信任度。所以提高网站的权重具有相当重要的意 义。 权重即网站在SEO中的重要性，权威性。英文：Page Strength。

• 权重不等于排名
• 权重对排名有着 非常大的影响
• 整站权重的提高有利于内页的排名

5.1、库操作（database）

# 添加索引库
PUT /索引库名
# 获取索引库信息
GET /索引库名
# 删除索引库
DELETE /索引库名
1
2
3
4
5
6

5.2、类型及映射操作（表）

创建数据库表需要设置字段约束，索引库也一样，在创建索引库的类型时，需要知道这个类型下 有哪些字段，每个字段有哪些约束信息，这就叫做 字段映射(mapping)

# 添加类型及映射1（向索引库追加类型及映射）
PUT winkto/_mapping/goods
{
  "properties": {
    "title": {
      "type": "text",
      "analyzer": "ik_max_word"
    },
    "images": {
      "type": "keyword",
      "index": "false"
    },
    "price": {
      "type": "float"
    }
  }
}
# 添加类型及映射2（在索引库创建时添加类型及映射）
PUT /winkto2
{
  "settings": {},
  "mappings": {
    "goods": {
      "properties": {
        "title": {
          "type": "text",
          "analyzer": "ik_max_word"
        }
      }
    }
  }
}
# 查看类型及映射
GET winkto/_mapping
GET winkto/_mapping/goods
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

5.3、追加数据

# 追加数据（随机id）
POST /winkto/goods/
{
  "title":"小米10",
  "images":"http://image.lagou.com/12479122.jpg",
  "price":2699.00
}
1
2
3
4
5
6
7

# 追加数据（指定id）
POST /winkto/goods/sadhsahduashfusahdi
{
  "title":"华为",
  "images":"http://image.lagou.com/12479122.jpg",
  "price":4699.00
}
1
2
3
4
5
6
7

# 获取数据
GET /winkto/goods/xqurVHsB1mHhNfuBSw0p
1
2

5.4、删除数据

DELETE /winkto/goods/sadhsahduashfusahdi
1

5.5、查询

语法

GET /索引库名/_search
{
    "query":{
        "查询类型":{
        	"查询条件":"查询条件值"
        }
    }
}
1
2
3
4
5
6
7
8

5.5.1、查询所有

GET /winkto/_search
{
    "query":{
        "match_all":{
        }
    }
}
1
2
3
4
5
6
7

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 1,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kBFMWHsBy-koX7e_VW4W",
        "_score": 1,
        "_source": {
          "title": "华为pro30",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 5699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

5.5.2、条件查询

GET /winkto/_search
{
    "query":{
        "match":{
          "title": "小米"
        }
    }
}
1
2
3
4
5
6
7
8

{
  "took": 16,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

match 类型查询，会把查询条件进行分词，然后进行查询,多个词条之间是or的关系

GET /winkto/_search
{
    "query":{
        "match":{
          "title": "小米华为"
        }
    }
}
1
2
3
4
5
6
7
8

{
  "took": 355,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kBFMWHsBy-koX7e_VW4W",
        "_score": 0.2876821,
        "_source": {
          "title": "华为pro30",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 5699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

但是有些时候我们希望分词关系为and

GET /winkto/_search
{
    "query":{
        "match":{
          "title": {
            "query": "小米华为",
            "operator": "and"
          }
        }
    }
}
1
2
3
4
5
6
7
8
9
10
11

{
  "took": 8,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.5753642,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 0.5753642,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

5.5.3、词条匹配

term 查询被用于精确值 匹配，这些精确值可能是数字、时间、布尔或者那些未分词的字符 串,keyword类型的字符串

GET /winkto/_search
{
    "query":{
        "term": {
          "price": 2699
        }
    }
}
1
2
3
4
5
6
7
8

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 1,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 1,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

5.5.4、布尔组合

bool 把各种其它查询通过 must （与）、 must_not （非）、 should （或）的方式进行组合

GET /winkto/_search
{
    "query":{
        "bool": {
          "must": [
            {
              "match": {
                "title": "小米"
              }
            }
          ],
          "must_not": [
            {
              "match": {
                "title": "华为"
              }
            }
          ],
          "should": [
            {
              "match": {
                "title": "10"
              }
            }
          ]
        }
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.5753642,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.5753642,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

5.5.5、范围查询

GET /winkto/_search
{
    "query":{
        "range": {
          "price": {
            "gte": 2000,
            "lte": 6000
          }
        }
    }
}
1
2
3
4
5
6
7
8
9
10
11

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": 1,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 1,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kBFMWHsBy-koX7e_VW4W",
        "_score": 1,
        "_source": {
          "title": "华为pro30",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 5699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 1,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

5.6、结果过滤

默认情况下，elasticsearch在搜索的结果中，会把文档中保存在 _source 的所有字段都返回。但有时只想获取其中的部分字段，此时可以添加 _source 的过滤

5.6.1、直接指定字段

GET /winkto/_search
{
  "_source": ["title","price"], 
  "query":{
      "match": {
        "title": "小米"
      }
  }
}
1
2
3
4
5
6
7
8
9

{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "price": 2699,
          "title": "小米10"
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 0.2876821,
        "_source": {
          "price": 2699,
          "title": "小米华为plusplus"
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

5.6.2、includes和excludes

• includes：来指定想要显示的字段
• excludes：来指定不想要显示的字段

GET /winkto/_search
{
  "_source": {
    "includes": ["title","images"],
    "excludes": ["price"]
  }, 
  "query":{
      "match": {
        "title": "小米"
      }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "images": "http://image.lagou.com/12479122.jpg",
          "title": "小米10"
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 0.2876821,
        "_source": {
          "images": "http://image.lagou.com/12479122.jpg",
          "title": "小米华为plusplus"
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

5.6.3、Filter

GET /winkto/_search
{
  "query":{
    "bool": {
      "must": [
        {
          "match": {
            "title": "小米"
          }
        }
      ],
      "filter": {
        "range": {
          "price": {
            "gte": 1000,
            "lte": 3000
          }
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 0.2876821,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

如果一次查询只有过滤，没有查询条件，不希望进行评分，可以使用 constant_score 取代只有 filter 语句的 bool 查询

GET /winkto/_search
{
  "query":{
    "constant_score": {
      "filter": {
        "range": {
          "price": {
            "gte": 1000,
            "lte": 3000
          }
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 1,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 1,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

5.7、排序

GET /winkto/_search
{
  "query":{
    "match_all": {}
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    }
  ]
}
1
2
3
4
5
6
7
8
9
10
11
12
13

{
  "took": 141,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": null,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kBFMWHsBy-koX7e_VW4W",
        "_score": null,
        "_source": {
          "title": "华为pro30",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 5699
        },
        "sort": [
          5699
        ]
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": null,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        },
        "sort": [
          2699
        ]
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": null,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        },
        "sort": [
          2699
        ]
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

5.8、分页

GET /winkto/_search
{
  "query":{
    "match_all": {}
  },
  "from": 0,
  "size": 2
}
1
2
3
4
5
6
7
8

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": 1,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 1,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kBFMWHsBy-koX7e_VW4W",
        "_score": 1,
        "_source": {
          "title": "华为pro30",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 5699
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

5.9、高亮显示

GET /winkto/_search
{
  "query":{
    "match": {
      "title": "小米"
    }
  },
  "highlight": {
    "pre_tags": "<em>",
    "post_tags": "</em>",
    "fields": {
      "title": {}
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{
  "took": 69,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "xqurVHsB1mHhNfuBSw0p",
        "_score": 0.2876821,
        "_source": {
          "title": "小米10",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        },
        "highlight": {
          "title": [
            "<em>小米</em>10"
          ]
        }
      },
      {
        "_index": "winkto",
        "_type": "goods",
        "_id": "kRFWWHsBy-koX7e_DW5_",
        "_score": 0.2876821,
        "_source": {
          "title": "小米华为plusplus",
          "images": "http://image.lagou.com/12479122.jpg",
          "price": 2699
        },
        "highlight": {
          "title": [
            "<em>小米</em>华为plusplus"
          ]
        }
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

5.10、聚合

聚合可以让我们极其方便的实现对数据的统计、分析

• 桶（group by）：是按照某种方式对数据进行分组，每一组数据在ES中称为一个 桶
• 度量（聚合的结果）：求平均值、最大、最小、求和等

预准备

PUT /car
{
  "mappings": {
    "orders": {
      "properties": {
        "color": {
          "type": "keyword"
        },
        "make": {
          "type": "keyword"
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

POST /car/orders/_bulk
{ "index": {}}
{ "price" : 10000, "color" : "红", "make" : "本田", "sold" : "2020-10-28" }
{ "index": {}}
{ "price" : 20000, "color" : "红", "make" : "本田", "sold" : "2020-11-05" }
{ "index": {}}
{ "price" : 30000, "color" : "绿", "make" : "福特", "sold" : "2020-05-18" }
{ "index": {}}
{ "price" : 15000, "color" : "蓝", "make" : "丰田", "sold" : "2020-07-02" }
{ "index": {}}
{ "price" : 12000, "color" : "绿", "make" : "丰田", "sold" : "2020-08-19" }
{ "index": {}}
{ "price" : 20000, "color" : "红", "make" : "本田", "sold" : "2020-11-05" }
{ "index": {}}
{ "price" : 80000, "color" : "红", "make" : "宝马", "sold" : "2020-01-01" }
{ "index": {}}
{ "price" : 25000, "color" : "蓝", "make" : "福特", "sold" : "2020-02-12" }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

5.10.1、聚合为桶

GET /car/_search
{
  "size": 0,
  "aggs": {
    "popular_colors": {
      "terms": {
        "field": "color"
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11

{
  "took": 48,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 8,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "popular_colors": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "红",
          "doc_count": 4
        },
        {
          "key": "绿",
          "doc_count": 2
        },
        {
          "key": "蓝",
          "doc_count": 2
        }
      ]
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

5.10.2、聚合操作

GET /car/_search
{
  "size": 0,
  "aggs": {
    "popular_colors": {
      "terms": {
        "field": "color"
      },
      "aggs": {
        "avg_price": {
          "avg": {
            "field": "price"
          }
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

{
  "took": 8,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 8,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "popular_colors": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "红",
          "doc_count": 4,
          "avg_price": {
            "value": 32500
          }
        },
        {
          "key": "绿",
          "doc_count": 2,
          "avg_price": {
            "value": 21000
          }
        },
        {
          "key": "蓝",
          "doc_count": 2,
          "avg_price": {
            "value": 20000
          }
        }
      ]
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

6、Elastic search集群

6.1、集群解决的问题

• 单台机器存储容量有限，无法实现高存储
• 单服务器容易出现单点故障，无法实现高可用
• 单服务的并发处理能力有限，无法实现高并发

6.2、数据分片

第一个问题就是数据量太大，单点存储量有限的问题，可以把数据拆分成多份，每一份存储到不同机器节点（node），从而实现减少每个节点数 据量的目的。这就是数据的分布式存储，也叫做： 数据分片（Shard）

数据分片解决了海量数据存储的问题，但是如果出现单点故障，那么分片数据就不再完整

给每个分片数据进行备 份，存储到其它节点，防止数据丢失，这就是数据备份，也叫 数据副本（replica）

数据备份可以保证高可用，但是每个分片备份一份，所需要的节点数量就会翻一倍，成本实在是太高

为了在高可用和成本间寻求平衡，首先对数据分片，存储到不同节点 然后对每个分片进行备份，放到对方节点，完成互相备份

6.3、集群搭建

node-0配置文件（node-1，node-2只需要修改node.name、http.port，transport.tcp.port，path.data，path.logs即可）

# ---------------------------------- Cluster -----------------------------------
# Use a descriptive name for your cluster:
cluster.name: my-application
# ------------------------------------ Node ------------------------------------
# Use a descriptive name for the node:
node.name: node-0
# Add custom attributes to the node:
#node.attr.rack: r1
# 是否可以为主节点
node.master: true
# ----------------------------------- Paths ------------------------------------
# Path to directory where to store the data (separate multiple locations by comma):
path.data: D:\es\elasticsearch-6.2.4-0\data
# Path to log files:
path.logs: D:\es\elasticsearch-6.2.4-0\data
# ----------------------------------- Memory -----------------------------------
# Lock the memory on startup:
#bootstrap.memory_lock: true
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
# Elasticsearch performs poorly when the system is swapping the memory.
# ---------------------------------- Network -----------------------------------
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 0.0.0.0
# Set a custom port for HTTP:
http.port: 9200
# For more information, consult the network module documentation.
# TCP协议对外端口 每个节点不一样，默认：9300
transport.tcp.port: 9201
# --------------------------------- Discovery ----------------------------------
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#discovery.zen.minimum_master_nodes: 
# For more information, consult the zen discovery module documentation.
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9201","127.0.0.1:9301","127.0.0.1:9401"]
discovery.zen.minimum_master_nodes: 2
# ---------------------------------- Gateway -----------------------------------
# Block initial recovery after a full cluster restart until N nodes are started:
#gateway.recover_after_nodes: 3
# For more information, consult the gateway module documentation.
# ---------------------------------- Various -----------------------------------
# Require explicit names when deleting indices:
#action.destructive_requires_name: true
# ---------------------------------- Http -----------------------------------
#允许跨域名访问
http.cors.enabled: true
#当设置允许跨域，默认为*,表示支持所有域名
http.cors.allow-origin: "*"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

创建索引winkto

PUT /winkto
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "items":{
      "properties": {
        "id": {
          "type": "keyword"
        },
        "title":{
          "type": "text",
          "analyzer": "ik_max_word"
        },
        "category":{
          "type": "keyword"
        },
        "brand": {
          "type": "keyword"
        },
        "images":{
          "type": "keyword",
          "index": false
        },
        "price":{
          "type": "double"
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

7、Elastic search API

预准备

PUT /winkto
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "items":{
      "properties": {
        "id": {
          "type": "keyword"
        },
        "title":{
          "type": "text",
          "analyzer": "ik_max_word"
        },
        "category":{
          "type": "keyword"
        },
        "brand": {
          "type": "keyword"
        },
        "images":{
          "type": "keyword",
          "index": false
        },
        "price":{
          "type": "double"
        }
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

7.1、添加数据

导入依赖

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-logging</artifactId>
    </dependency>

    <!--ES高级Rest Client-->
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>6.5.4</version>
    </dependency>

    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>6.5.4</version>
    </dependency>

    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
    </dependency>
</dependencies>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

实体类

public class Product {
    private Long id;
    private String title;
    private String category;
    private String brand;
    private Double price;
    private String images;
}
1
2
3
4
5
6
7
8

config

@Configuration
public class WinktoConfig {
    @Bean
    public ObjectMapper jackson(){
        return new ObjectMapper();
    }
}
1
2
3
4
5
6
7

测试

@SpringBootTest
class ElasticSearchApplicationTests {
    @Autowired
    ObjectMapper jackson;
    @Test
    void contextLoadsinsert() throws IOException {
        // 初始化HighLevel客户端
        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
                new HttpHost("127.0.0.1",9200),
                new HttpHost("127.0.0.1",9300),
                new HttpHost("127.0.0.1",9400)));
        // 文档数据
        Product product = new Product(1L,"华为P50、新款发布","手机","华为",5999.9,"http://image.huawei.com/1.jpg");
        IndexRequest indexRequest = new IndexRequest("winkto", "items", product.getId().toString());
        indexRequest.source(jackson.writeValueAsString(product), XContentType.JSON);
        IndexResponse response = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(response);
        // 关闭客户端
        restHighLevelClient.close();
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

结果

IndexResponse[index=winkto,type=items,id=1,version=3,result=created,seqNo=2,primaryTerm=1,shards={"total":2,"successful":2,"failed":0}]
1

7.2、查询数据

@Test
void contextLoadsSelect() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    GetRequest getRequest = new GetRequest("winkto","items","1");
    GetResponse documentFields = restHighLevelClient.get(getRequest,RequestOptions.DEFAULT);
    String sourceAsString = documentFields.getSourceAsString();
    Product product = jackson.readValue(sourceAsString, Product.class);
    System.out.println(product.toString());
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Product{id=1, title='华为P50、新款发布', category='手机', brand='华为', price=5999.9, images='http://image.huawei.com/1.jpg'}
1

7.3、修改文档

新增时，如果传递的id是已经存在的，则会完成修改操作，如果不存在，则是新增

7.4、删除文档

@Test
void contextLoadsDelete() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    DeleteRequest deleteRequest = new DeleteRequest("winkto", "items", "1");
    DeleteResponse delete = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12

7.5、match_all

@Test
void contextLoadsMatchAll() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.matchAllQuery());
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7.6、match

@Test
void contextLoadsMatch() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.matchQuery("title","新款"));
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7.7、range

@Test
void contextLoadsRange() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gt(2000).lt(4000));
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7.8、source过滤

默认情况下，索引库中所有数据都会返回，如果我们想只返回部分字段，可以通过source filter来控制

@Test
void contextLoadsSource() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gt(2000).lt(4000));
    // 第一个参数想要包含的字段，第二个参数不想要包含的字段
    searchSourceBuilder.fetchSource(new String[]{"title","price"},null);
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

7.9、排序

@Test
void contextLoadsSort() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gt(2000).lt(4000));
    // 降序排序
    searchSourceBuilder.sort("price", SortOrder.DESC);
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

7.10、分页

@Test
void contextLoadsSort() throws IOException {
    // 初始化HighLevel客户端
    RestHighLevelClient restHighLevelClient = new RestHighLevelClient(RestClient.builder(
            new HttpHost("127.0.0.1",9200),
            new HttpHost("127.0.0.1",9300),
            new HttpHost("127.0.0.1",9400)));
    SearchRequest searchRequest = new SearchRequest();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.matchAllQuery());
    // 降序排序
    searchSourceBuilder.sort("price", SortOrder.DESC);
    //
    searchSourceBuilder.from(0);
    searchSourceBuilder.size(2);
    searchRequest.source(searchSourceBuilder);
    SearchResponse search = restHighLevelClient.search(searchRequest);
    SearchHits hits = search.getHits();
    SearchHit[] hits1 = hits.getHits();
    for (SearchHit documentFields : hits1) {
        String sourceAsString = documentFields.getSourceAsString();
        Product product = jackson.readValue(sourceAsString, Product.class);
        System.out.println(product);
    }
    // 关闭客户端
    restHighLevelClient.close();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

8、Spring Data Elastic search

Spring Data 的使命是给各种数据访问提供统一的编程接口，不管是关系型数据库（如MySQL），还是 非关系数据库（如Redis），或者类似Elasticsearch这样的索引数据库。从而简化开发人员的代码，提 高开发效率。

• 支持Spring的基于 @Configuration 的java配置方式，或者XML配置方式
• 提供了用于操作ES的便捷工具类 ElasticsearchTemplate 。包括实现文档到POJO之间的自动智能映射
• 利用Spring的数据转换服务实现的功能丰富的对象映射
• 基于注解的元数据映射方式，而且可扩展以支持更多不同的数据格式，可以定义JavaBean：类名、属性
• 根据持久层接口自动生成对应实现方法，无需人工编写基本操作代码（类似mybatis，根据接口自 动得到实现）。当然，也支持人工定制查询

8.1、创建索引库

导入依赖

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

@Document(indexName = "winkto", type = "product", shards = 3, replicas = 1)
public class Product {
    @Id
    private Long id;
    @Field(type = FieldType.Text,analyzer = "ik_max_word")
    private String title;
    @Field(type = FieldType.Keyword)
    private String category;
    @Field(type = FieldType.Keyword)
    private String brand;
    @Field(type = FieldType.Double)
    private Double price;
    @Field(type = FieldType.Keyword)
    private String images;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

@SpringBootTest
class EsDataApplicationTests {

    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;

    @Test
    void contextLoadsIndex() {
        elasticsearchTemplate.createIndex(Product.class);
    }
}
1
2
3
4
5
6
7
8
9
10
11

application.yaml

spring:
  data:
    elasticsearch:
      cluster-name: my-application
      cluster-nodes: 127.0.0.1:9201,127.0.0.1:9301,127.0.0.1:9401
1
2
3
4
5

8.2、创建类型映射

@Test
void contextLoadsMapping() {
    elasticsearchTemplate.putMapping(Product.class);
}
1
2
3
4

8.3、索引数据CRUD

添加数据（单条）

@Test
void contextLoadsInsert() {
    Product product = new Product(1L, "小米手机7", "手机", "小米", 3299.00, "/13123.jpg");
    productMapper.save(product);
}
1
2
3
4
5

添加数据（多条）

@Test
void contextLoadsInsertAll() {
    List<Product> list = new ArrayList<>();
    list.add(new Product(2L, "坚果手机R1", "手机", "锤子", 3699.00, "/13123.jpg"));
    list.add(new Product(3L, "华为META10", "手机", "华为", 4499.00, "/13123.jpg"));
    list.add(new Product(4L, "小米Mix2S", "手机", "小米", 4299.00, "/13123.jpg"));
    list.add(new Product(5L, "荣耀V10", "手机", "华为", 2799.00, "/13123.jpg"));
    productMapper.saveAll(list);
}
1
2
3
4
5
6
7
8
9

根据id查询

@Test
void contextLoadsSelect() {
    Optional<Product> optionalProduct = productMapper.findById(1L);
    System.out.println(optionalProduct.orElse(null));
}
1
2
3
4
5

查询所有

@Test
void contextLoadsSelectAll() {
    Iterable<Product> all = productMapper.findAll();
    all.forEach(System.out::println);
}
1
2
3
4
5

根据id删除

@Test
void contextLoadsDelete() {
    productMapper.deleteById(1L);
}
1
2
3
4