赞
踩
前段时间,小刘从硬盘找了以前的笔记,总结,和大家分享一下
Elasticsearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎,基于RESTful web接口。Elasticsearch是用Java语言开发的,并作为Apache许可条款下的开放源码发布,是一种流行的企业级搜索引擎。Elasticsearch用于云计算中,能够达到实时搜索,稳定,可靠,快速,安装使用方便。官方客户端在Java、.NET(C#)、PHP、Python、Apache Groovy、Ruby和许多其他语言中都是可用的。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr,也是基于Lucene
就连Github都是用ElasticSearch做词条搜索
如:当系统数据量上了10亿、100亿条的时候,我们在做系统架构的时候通常会从以下角度去考虑问题:
1)用什么数据库好?(mysql、sybase、oracle、达梦、神通、mongodb、hbase…)
2)如何解决单点故障;(lvs、F5、A10、Zookeep、MQ)
3)如何保证数据安全性;(热备、冷备、异地多活)
4)如何解决检索难题;(数据库代理中间件:mysql-proxy、Cobar、MaxScale等;)
5)如何解决统计分析问题;(离线、近实时)
对于关系型数据,我们通常采用以下或类似架构去解决查询瓶颈和写入瓶颈:
解决要点:
1)通过主从备份解决数据安全性问题;
2)通过数据库代理中间件心跳监测,解决单点故障问题;
3)通过代理中间件将查询语句分发到各个slave节点进行查询,并汇总结果
对于Nosql数据库,以mongodb为例,其它原理类似:
解决要点:
1)通过副本备份保证数据安全性;
2)通过节点竞选机制解决单点问题;
3)先从配置库检索分片信息,然后将请求分发到各个节点,最后由路由节点合并汇总结果
我们知道,完全把数据放在内存中是不可靠的,实际上也不太现实,当我们的数据达到PB级别时,按照每个节点96G内存计算,在内存完全装满的数据情况下,我们需要的机器是:1PB=1024T=1048576G
节点数=1048576/96=10922个
实际上,考虑到数据备份,节点数往往在2.5万台左右。成本巨大决定了其不现实!
从前面讨论我们了解到,把数据放在内存也好,不放在内存也好,都不能完完全全解决问题。
全部放在内存速度问题是解决了,但成本问题上来了。
为解决以上问题,从源头着手分析,通常会从以下方式来寻找方法:
1、存储数据时按有序存储;
2、将数据和索引分离;
3、压缩数据;
这就引出了Elasticsearch。
在 Elasticsearch 中,文档归属于一种类型 type,而这些类型存在于索引 index 中,我们可以列一些简单的不同点,来类比传统关系型数据库:
Elasticsearch 集群可以包含多个索引 indices,每一个索引可以包含多个类型 types,每一个类型包含多个文档 documents,然后每个文档包含多个字段 Fields。而在 DB 中可以有多个数据库 Databases,每个库中可以有多张表 Tables,没个表中又包含多行Rows,每行包含多列Columns。
ElasticSearch的对象模型,跟关系型数据库模型相比:
索引(Index):相当于数据库,用于定义文档类型的存储;在同一个索引中,同一个字段只能定义一个数据类型;
文档类型(Type):相当于关系表,用于描述文档中的各个字段的定义;不同的文档类型,能够存储不同的字段,服务于不同的查询请求;
ElasticSearch的对象模型,跟关系型数据库模型相比:
索引(Index):相当于数据库,用于定义文档类型的存储;在同一个索引中,同一个字段只能定义一个数据类型;
文档类型(Type):相当于关系表,用于描述文档中的各个字段的定义;不同的文档类型,能够存储不同的字段,服务于不同的查询请求;
文档(Document):相当于关系表的数据行,存储数据的载体,包含一个或多个存有数据的字段;
字段(Field):文档的一个Key/Value对;
词(Term):表示文本中的一个单词;
标记(Token):表示在字段中出现的词,由该词的文本、偏移量(开始和结束)以及类型组成;
索引是由段(Segment)组成的,段存储在硬盘(Disk)文件中,段不是实时更新的,这意味着,段在写入磁盘后,就不再被更新。ElasticSearch引擎把被删除的文档的信息存储在一个单独的文件中,在搜索数据时,
ElasticSearch引擎首先从段中查询,再从查询结果中过滤被删除的文档,这意味着,段中存储着“被删除”的文档,这使得段中含有”正常文档“的密度降低。多个段可以通过段合并(Segment Merge)操作把“已删除”的文档将从段中物理删除,把未删除的文档合并到一个新段中,新段中没有”已删除文档“,因此,段合并操作能够提高索引的查找速度,但段合并是IO密集型的操作,需要消耗大量的硬盘IO。
三、SpringBoot集成ElasticSearch 介绍
下面介绍下 SpringBoot 如何通过 elasticsearch-rest-high-level-client 工具操作 ElasticSearch,这里需要说一下,为什么没有使用 Spring 家族封装的 spring-data-elasticsearch。
主要原因是灵活性和更新速度,Spring 将 ElasticSearch 过度封装,让开发者很难跟 ES 的 DSL 查询语句进行关联。再者就是更新速度,ES 的更新速度是非常快,但是 spring-data-elasticsearch 更新速度比较缓慢。
由于上面两点,所以选择了官方推出的 Java 客户端 elasticsearch-rest-high-level-client,它的代码写法跟 DSL 语句很相似,懂 ES 查询的使用其上手很快。
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0modelVersion> <parent> <groupId>org.springframework.bootgroupId> <artifactId>spring-boot-starter-parentartifactId> <version>2.2.4.RELEASEversion> <relativePath/> parent> <groupId>cloud.spiritmarkgroupId> <artifactId>springboot-elasticsearch-exampleartifactId> <version>0.0.1-SNAPSHOTversion> <name>springboot-elasticsearch-examplename> <description>Demo project for ElasticSearchdescription> <properties> <java.version>1.8java.version> properties> <dependencies> <dependency> <groupId>org.springframework.bootgroupId> <artifactId>spring-boot-starter-webartifactId> dependency> <dependency> <groupId>org.projectlombokgroupId> <artifactId>lombokartifactId> <optional>trueoptional> dependency> <dependency> <groupId>org.elasticsearch.clientgroupId> <artifactId>elasticsearch-rest-high-level-clientartifactId> <version>6.5.4version> dependency> <dependency> <groupId>org.elasticsearchgroupId> <artifactId>elasticsearchartifactId> <version>6.5.4version> dependency> dependencies> <build> <plugins> <plugin> <groupId>org.springframework.bootgroupId> <artifactId>spring-boot-maven-pluginartifactId> plugin> plugins> build>project>
(1)、application.yml 配置文件
为了方便更改连接 ES 的连接配置,所以我们将配置信息放置于 application.yaml 中:
#baseserver: port: 8080#springspring: application: name: springboot-elasticsearch#elasticsearchelasticsearch: schema: http address: 127.0.0.1:9200 connectTimeout: 5000 socketTimeout: 5000 connectionRequestTimeout: 5000 maxConnectNum: 100 maxConnectPerRoute: 100
(2)、java 连接配置类
这里需要写一个 Java 配置类读取 application 中的配置信息:
import org.apache.http.HttpHost;import org.elasticsearch.client.RestClient;import org.elasticsearch.client.RestClientBuilder;import org.elasticsearch.client.RestHighLevelClient;import org.springframework.beans.factory.annotation.Value;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import java.util.ArrayList;import java.util.List;/** * ElasticSearch 配置 */@Configurationpublic class ElasticSearchConfig { /** 协议 */ @Value("${elasticsearch.schema:http}") private String schema; /** 集群地址,如果有多个用“,”隔开 */ @Value("${elasticsearch.address}") private String address; /** 连接超时时间 */ @Value("${elasticsearch.connectTimeout:5000}") private int connectTimeout; /** Socket 连接超时时间 */ @Value("${elasticsearch.socketTimeout:10000}") private int socketTimeout; /** 获取连接的超时时间 */ @Value("${elasticsearch.connectionRequestTimeout:5000}") private int connectionRequestTimeout; /** 最大连接数 */ @Value("${elasticsearch.maxConnectNum:100}") private int maxConnectNum; /** 最大路由连接数 */ @Value("${elasticsearch.maxConnectPerRoute:100}") private int maxConnectPerRoute; @Bean public RestHighLevelClient restHighLevelClient() { // 拆分地址 List hostLists = new ArrayList<>(); String[] hostList = address.split(",");for (String addr : hostList) { String host = addr.split(":")[0]; String port = addr.split(":")[1]; hostLists.add(new HttpHost(host, Integer.parseInt(port), schema)); }// 转换成 HttpHost 数组 HttpHost[] httpHost = hostLists.toArray(new HttpHost[]{});// 构建连接对象 RestClientBuilder builder = RestClient.builder(httpHost);// 异步连接延时配置 builder.setRequestConfigCallback(requestConfigBuilder -> { requestConfigBuilder.setConnectTimeout(connectTimeout); requestConfigBuilder.setSocketTimeout(socketTimeout); requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeout);return requestConfigBuilder; });// 异步连接数配置 builder.setHttpClientConfigCallback(httpClientBuilder -> { httpClientBuilder.setMaxConnTotal(maxConnectNum); httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);return httpClientBuilder; });return new RestHighLevelClient(builder); }}
创建索引
创建名为 mydlq-user 的索引与对应 Mapping。
请求格式 :PUT /mydlq-user
{ "mappings": { "doc": { "dynamic": true, "properties": { "name": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "address": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "remark": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "age": { "type": "integer" }, "salary": { "type": "float" }, "birthDate": { "type": "date", "format": "yyyy-MM-dd" }, "createTime": { "type": "date" } } } }}
删除索引
删除 mydlq-user 索引。
DELETE /mydlq-user
import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;import org.elasticsearch.action.support.master.AcknowledgedResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.settings.Settings;import org.elasticsearch.common.xcontent.XContentBuilder;import org.elasticsearch.common.xcontent.XContentFactory;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class IndexService2 { @Autowired private RestHighLevelClient restHighLevelClient; /** * 创建索引 */ public void createIndex() { try { // 创建 Mapping XContentBuilder mapping = XContentFactory.jsonBuilder() .startObject() .field("dynamic", true) .startObject("properties") .startObject("name") .field("type","text") .startObject("fields") .startObject("keyword") .field("type","keyword") .endObject() .endObject() .endObject() .startObject("address") .field("type","text") .startObject("fields") .startObject("keyword") .field("type","keyword") .endObject() .endObject() .endObject() .startObject("remark") .field("type","text") .startObject("fields") .startObject("keyword") .field("type","keyword") .endObject() .endObject() .endObject() .startObject("age") .field("type","integer") .endObject() .startObject("salary") .field("type","float") .endObject() .startObject("birthDate") .field("type","date") .field("format", "yyyy-MM-dd") .endObject() .startObject("createTime") .field("type","date") .endObject() .endObject() .endObject(); // 创建索引配置信息,配置 Settings settings = Settings.builder() .put("index.number_of_shards", 1) .put("index.number_of_replicas", 0) .build(); // 新建创建索引请求对象,然后设置索引类型(ES 7.0 将不存在索引类型)和 mapping 与 index 配置 CreateIndexRequest request = new CreateIndexRequest("mydlq-user", settings); request.mapping("doc", mapping); // RestHighLevelClient 执行创建索引 CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT); // 判断是否创建成功 boolean isCreated = createIndexResponse.isAcknowledged(); log.info("是否创建成功:{}", isCreated); } catch (IOException e) { log.error("", e); } } /** * 删除索引 */ public void deleteIndex() { try { // 新建删除索引请求对象 DeleteIndexRequest request = new DeleteIndexRequest("mydlq-user"); // 执行删除索引 AcknowledgedResponse acknowledgedResponse = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT); // 判断是否删除成功 boolean siDeleted = acknowledgedResponse.isAcknowledged(); log.info("是否删除成功:{}", siDeleted); } catch (IOException e) { log.error("", e); } }}
增加文档信息
在索引 mydlq-user 中增加一条文档信息。
POST /mydlq-user/doc{ "address": "北京市", "age": 29, "birthDate": "1990-01-10", "createTime": 1579530727699, "name": "张三", "remark": "来自北京市的张先生", "salary": 100}
获取文档信息
获取 mydlq-user 的索引 id=1 的文档信息。
GET /mydlq-user/doc/1
更新文档信息
更新之前创建的 id=1 的文档信息。
PUT /mydlq-user/doc/1{ "address": "北京市海淀区", "age": 29, "birthDate": "1990-01-10", "createTime": 1579530727699, "name": "张三", "remark": "来自北京市的张先生", "salary": 100}
删除文档信息
删除之前创建的 id=1 的文档信息。
DELETE /mydlq-user/doc/1
import cloud.spiritmark.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.delete.DeleteRequest;import org.elasticsearch.action.delete.DeleteResponse;import org.elasticsearch.action.get.GetRequest;import org.elasticsearch.action.get.GetResponse;import org.elasticsearch.action.index.IndexRequest;import org.elasticsearch.action.index.IndexResponse;import org.elasticsearch.action.update.UpdateRequest;import org.elasticsearch.action.update.UpdateResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.xcontent.XContentType;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;import java.util.Date;@Slf4j@Servicepublic class IndexService { @Autowired private RestHighLevelClient restHighLevelClient; /** * 增加文档信息 */ public void addDocument() { try { // 创建索引请求对象 IndexRequest indexRequest = new IndexRequest("mydlq-user", "doc", "1"); // 创建员工信息 UserInfo userInfo = new UserInfo(); userInfo.setName("张三"); userInfo.setAge(29); userInfo.setSalary(100.00f); userInfo.setAddress("北京市"); userInfo.setRemark("来自北京市的张先生"); userInfo.setCreateTime(new Date()); userInfo.setBirthDate("1990-01-10"); // 将对象转换为 byte 数组 byte[] json = JSON.toJSONBytes(userInfo); // 设置文档内容 indexRequest.source(json, XContentType.JSON); // 执行增加文档 IndexResponse response = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT); log.info("创建状态:{}", response.status()); } catch (Exception e) { log.error("", e); } } /** * 获取文档信息 */ public void getDocument() { try { // 获取请求对象 GetRequest getRequest = new GetRequest("mydlq-user", "doc", "1"); // 获取文档信息 GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT); // 将 JSON 转换成对象 if (getResponse.isExists()) { UserInfo userInfo = JSON.parseObject(getResponse.getSourceAsBytes(), UserInfo.class); log.info("员工信息:{}", userInfo); } } catch (IOException e) { log.error("", e); } } /** * 更新文档信息 */ public void updateDocument() { try { // 创建索引请求对象 UpdateRequest updateRequest = new UpdateRequest("mydlq-user", "doc", "1"); // 设置员工更新信息 UserInfo userInfo = new UserInfo(); userInfo.setSalary(200.00f); userInfo.setAddress("北京市海淀区"); // 将对象转换为 byte 数组 byte[] json = JSON.toJSONBytes(userInfo); // 设置更新文档内容 updateRequest.doc(json, XContentType.JSON); // 执行更新文档 UpdateResponse response = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT); log.info("创建状态:{}", response.status()); } catch (Exception e) { log.error("", e); } } /** * 删除文档信息 */ public void deleteDocument() { try { // 创建删除请求对象 DeleteRequest deleteRequest = new DeleteRequest("mydlq-user", "doc", "1"); // 执行删除文档 DeleteResponse response = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT); log.info("删除状态:{}", response.status()); } catch (IOException e) { log.error("", e); } }}
执行查询示例前,先往索引中插入一批数据:
POST mydlq-user/_doc
{"name":"小白","address":"北京市海定区","remark":"低层员工","age":29,"salary":3000,"birthDate":"1990-11-11","createTime":"2019-11-11T08:18:00.000Z"}
POST _bulk
{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"刘一","address":"北京市丰台区","remark":"低层员工","age":30,"salary":3000,"birthDate":"1989-11-11","createTime":"2019-03-15T08:18:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"陈二","address":"北京市昌平区","remark":"中层员工","age":27,"salary":7900,"birthDate":"1992-01-25","createTime":"2019-11-08T11:15:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"张三","address":"北京市房山区","remark":"中层员工","age":28,"salary":8800,"birthDate":"1991-10-05","createTime":"2019-07-22T13:22:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"李四","address":"北京市大兴区","remark":"高层员工","age":26,"salary":9000,"birthDate":"1993-08-18","createTime":"2019-10-17T15:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"王五","address":"北京市密云区","remark":"低层员工","age":31,"salary":4800,"birthDate":"1988-07-20","createTime":"2019-05-29T09:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"赵六","address":"北京市通州区","remark":"中层员工","age":32,"salary":6500,"birthDate":"1987-06-02","createTime":"2019-12-10T18:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"孙七","address":"北京市朝阳区","remark":"中层员工","age":33,"salary":7000,"birthDate":"1986-04-15","createTime":"2019-06-06T13:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"周八","address":"北京市西城区","remark":"低层员工","age":32,"salary":5000,"birthDate":"1987-09-26","createTime":"2019-01-26T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吴九","address":"北京市海淀区","remark":"高层员工","age":30,"salary":11000,"birthDate":"1989-11-25","createTime":"2019-09-07T13:34:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"郑十","address":"北京市东城区","remark":"低层员工","age":29,"salary":5000,"birthDate":"1990-12-25","createTime":"2019-03-06T12:08:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"萧十一","address":"北京市平谷区","remark":"低层员工","age":29,"salary":3300,"birthDate":"1990-11-11","createTime":"2019-03-10T08:17:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"曹十二","address":"北京市怀柔区","remark":"中层员工","age":27,"salary":6800,"birthDate":"1992-01-25","createTime":"2019-12-03T11:09:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吴十三","address":"北京市延庆区","remark":"中层员工","age":25,"salary":7000,"birthDate":"1994-10-05","createTime":"2019-07-27T14:22:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"冯十四","address":"北京市密云区","remark":"低层员工","age":25,"salary":3000,"birthDate":"1994-08-18","createTime":"2019-04-22T15:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"蒋十五","address":"北京市通州区","remark":"低层员工","age":31,"salary":2800,"birthDate":"1988-07-20","createTime":"2019-06-13T10:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"苗十六","address":"北京市门头沟区","remark":"高层员工","age":32,"salary":11500,"birthDate":"1987-06-02","createTime":"2019-11-11T18:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"鲁十七","address":"北京市石景山区","remark":"高员工","age":33,"salary":9500,"birthDate":"1986-04-15","createTime":"2019-06-06T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"沈十八","address":"北京市朝阳区","remark":"中层员工","age":31,"salary":8300,"birthDate":"1988-09-26","createTime":"2019-09-25T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吕十九","address":"北京市西城区","remark":"低层员工","age":31,"salary":4500,"birthDate":"1988-11-25","createTime":"2019-09-22T13:34:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"丁二十","address":"北京市东城区","remark":"低层员工","age":33,"salary":2100,"birthDate":"1986-12-25","createTime":"2019-03-07T12:08:00.000Z"}
插入完成后再查询数据,查看之前插入的数据是否存在:
GET mydlq-user/_search
执行后得到下面记录:
{ "took": 2, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": 20, "max_score": 1, "hits": [ { "_index": "mydlq-user", "_type": "_doc", "_id": "BeN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "刘一", "address": "北京市丰台区", "remark": "低层员工", "age": 30, "salary": 3000, "birthDate": "1989-11-11", "createTime": "2019-03-15T08:18:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "BuN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "陈二", "address": "北京市昌平区", "remark": "中层员工", "age": 27, "salary": 7900, "birthDate": "1992-01-25", "createTime": "2019-11-08T11:15:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "B-N0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "张三", "address": "北京市房山区", "remark": "中层员工", "age": 28, "salary": 8800, "birthDate": "1991-10-05", "createTime": "2019-07-22T13:22:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "CON0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "李四", "address": "北京市大兴区", "remark": "高层员工", "age": 26, "salary": 9000, "birthDate": "1993-08-18", "createTime": "2019-10-17T15:00:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "CeN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "王五", "address": "北京市密云区", "remark": "低层员工", "age": 31, "salary": 4800, "birthDate": "1988-07-20", "createTime": "2019-05-29T09:00:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "CuN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "赵六", "address": "北京市通州区", "remark": "中层员工", "age": 32, "salary": 6500, "birthDate": "1987-06-02", "createTime": "2019-12-10T18:00:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "C-N0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "孙七", "address": "北京市朝阳区", "remark": "中层员工", "age": 33, "salary": 7000, "birthDate": "1986-04-15", "createTime": "2019-06-06T13:00:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "DON0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "周八", "address": "北京市西城区", "remark": "低层员工", "age": 32, "salary": 5000, "birthDate": "1987-09-26", "createTime": "2019-01-26T14:00:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "DeN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "吴九", "address": "北京市海淀区", "remark": "高层员工", "age": 30, "salary": 11000, "birthDate": "1989-11-25", "createTime": "2019-09-07T13:34:00.000Z" } }, { "_index": "mydlq-user", "_type": "_doc", "_id": "DuN0BW8B7BNodGwRFTRj", "_score": 1, "_source": { "name": "郑十", "address": "北京市东城区", "remark": "低层员工", "age": 29, "salary": 5000, "birthDate": "1990-12-25", "createTime": "2019-03-06T12:08:00.000Z" } } ] }}
精确查询
精确查询,查询地址为 北京市通州区 的人员信息:
查询条件不会进行分词,但是查询内容可能会分词,导致查询不到。之前在创建索引时设置 Mapping 中 address 字段存在 keyword 字段是专门用于不分词查询的子字段。
GET mydlq-user/_search{ "query": { "term": { "address.keyword": { "value": "北京市通州区" } } }}
精确查询-多内容查询
精确查询,查询地址为 北京市丰台区、北京市昌平区 或 北京市大兴区 的人员信息:
GET mydlq-user/_search{ "query": { "terms": { "address.keyword": [ "北京市丰台区", "北京市昌平区", "北京市大兴区" ] } }}
import cloud.spiritmark.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class TermQueryService { @Autowired private RestHighLevelClient restHighLevelClient; /** * 精确查询(查询条件不会进行分词,但是查询内容可能会分词,导致查询不到) */ public void termQuery() { try { // 构建查询条件(注意:termQuery 支持多种格式查询,如 boolean、int、double、string 等,这里使用的是 string 的查询) SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.termQuery("address.keyword", "北京市通州区")); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } } /** * 多个内容在一个字段中进行查询 */ public void termsQuery() { try { // 构建查询条件(注意:termsQuery 支持多种格式查询,如 boolean、int、double、string 等,这里使用的是 string 的查询) SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.termsQuery("address.keyword", "北京市丰台区", "北京市昌平区", "北京市大兴区")); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } }}
匹配查询全部数据与分页
匹配查询符合条件的所有数据,并且设置以 salary 字段升序排序,并设置分页:
GET mydlq-user/_search{ "query": { "match_all": {} }, "from": 0, "size": 10, "sort": [ { "salary": { "order": "asc" } } ]}
匹配查询数据
匹配查询地址为 通州区 的数据:
GET mydlq-user/_search{ "query": { "match": { "address": "通州区" } }}
词语匹配查询
词语匹配进行查询,匹配 address 中为 北京市通州区 的员工信息:
GET mydlq-user/_search{ "query": { "match_phrase": { "address": "北京市通州区" } }}
内容多字段查询
查询在字段 address、remark 中存在 北京 内容的员工信息:
GET mydlq-user/_search{ "query": { "multi_match": { "query": "北京", "fields": ["address","remark"] } }}
import cloud.spiritmark.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.MatchAllQueryBuilder;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.elasticsearch.search.sort.SortOrder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class MatchQueryService { @Autowired private RestHighLevelClient restHighLevelClient; /** * 匹配查询符合条件的所有数据,并设置分页 */ public Object matchAllQuery() { try { // 构建查询条件 MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery(); // 创建查询源构造器 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(matchAllQueryBuilder); // 设置分页 searchSourceBuilder.from(0); searchSourceBuilder.size(3); // 设置排序 searchSourceBuilder.sort("salary", SortOrder.ASC); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } } /** * 匹配查询数据 */ public Object matchQuery() { try { // 构建查询条件 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.matchQuery("address", "*通州区")); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } } /** * 词语匹配查询 */ public Object matchPhraseQuery() { try { // 构建查询条件 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.matchPhraseQuery("address", "北京市通州区")); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } } /** * 内容在多字段中进行查询 */ public Object matchMultiQuery() { try { // 构建查询条件 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.multiMatchQuery("北京市", "address", "remark")); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } }}
模糊查询所有以 三 结尾的姓名
GET mydlq-user/_search{ "query": { "fuzzy": { "name": "三" } }}
import cloud.spiritmark.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.unit.Fuzziness;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class FuzzyQueryService { @Autowired private RestHighLevelClient restHighLevelClient; /** * 模糊查询所有以 “三” 结尾的姓名 */ public Object fuzzyQuery() { try { // 构建查询条件 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.fuzzyQuery("name", "三").fuzziness(Fuzziness.AUTO)); // 创建查询请求对象,将查询对象配置到其中 SearchRequest searchRequest = new SearchRequest("mydlq-user"); searchRequest.source(searchSourceBuilder); // 执行查询,然后处理响应结果 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); // 根据状态和数据条数验证是否返回了数据 if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) { SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { // 将 JSON 转换成对象 UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class); // 输出查询信息 log.info(userInfo.toString()); } } } catch (IOException e) { log.error("", e); } }}
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。