Springboot整合Elasticsearch7.10.0 RestHighLevelClient_springbootelasticsearch7.10

参考文章：https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-document-index.html

一、elasticsearch官方文档说明

---

该文章，主要是介绍elasticsearch7.x的rest java客户端。

1.1 增（只看原理，后面有完整的工具类）

定义一个request

IndexRequest request = new IndexRequest(索引名称); 
request.id(文档id); 
String jsonString = "{" +
        "\"user\":\"kimchy\"," +
        "\"postDate\":\"2013-01-30\"," +
        "\"message\":\"trying out Elasticsearch\"" +
        "}";
request.source(jsonString, XContentType.JSON); 
12345678
1
2
3
4
5
6
7
8
9

执行request命令，分同步异步，一般使用同步

IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);
1
1
2

1.2 删（只看原理，后面有完整的工具类）

定义一个request
根据文档id进行删除

DeleteRequest request = new DeleteRequest(
        索引名称,    
        文档id); 
123
1
2
3
4

执行request命令

DeleteResponse deleteResponse = client.delete(request, RequestOptions.DEFAULT);
1
1
2

1.3 改（只看原理，后面有完整的工具类）

定义一个request

UpdateRequest request = new UpdateRequest(索引名称, 文档id);
String jsonString = "{" +
        "\"updated\":\"2017-01-01\"," +
        "\"reason\":\"daily update\"" +
        "}";
request.doc(jsonString, XContentType.JSON);
123456
1
2
3
4
5
6
7

执行request命令

UpdateResponse updateResponse = client.update(request, RequestOptions.DEFAULT);
1
1
2

1.4 查（只看原理，后面有完整的工具类）

查有非常多方式，elasticsearch作为一个开源搜索引擎，全文检索，结构化检索，数据分析，所以有很强大的搜索功能

定义一个request

SearchRequest searchRequest = new SearchRequest(); 
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); 
searchSourceBuilder.query(QueryBuilders.matchAllQuery()); 
searchRequest.source(searchSourceBuilder);
1234
1
2
3
4
5

同样具有分页功能

SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); 
sourceBuilder.query(QueryBuilders.termQuery("user", "kimchy")); 
sourceBuilder.from(0); 
sourceBuilder.size(5); 
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); 
12345
1
2
3
4
5
6

执行request命令

SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
1
1
2

二、elasticsearch工具类步骤

---

2.1 maven依赖

elasticsearch的版本要与客户端的版本一致

按照es官方文档描述elasticsearch-rest-client还需要依赖

The High Level Java REST Client depends on the following artifacts and their transitive dependencies:

org.elasticsearch.client:elasticsearch-rest-client
org.elasticsearch:elasticsearch
1234
    <properties>
        <java.version>1.8</java.version>
        <elasticsearch>7.5.1</elasticsearch>
    </properties>

    <dependencies>
        <!--SpringBoot整合Spring Data Elasticsearch的依赖-->

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>${elasticsearch}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elasticsearch}</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.62</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>commons-beanutils</groupId>
            <artifactId>commons-beanutils</artifactId>
            <version>1.8.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

实际上官方文档说 只引入一个依赖就可，实测可以。

注意看一下版本号下面的依赖，我用的最新的4.1.1

        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>4.1.1</version>
        </dependency>
1
2
3
4
5

2.2 elasticsearch的搜索条件

下图整理出部分搜索相关的

参考大神文章：https://blog.csdn.net/weixin_42408648/article/details/108199320

2.2.1 精确查询（必须完全匹配上）

• 思考： 有些精确匹配，需要加上 .keyword

单个匹配termQuery

//不分词查询 参数1： 字段名，参数2：字段查询值，因为不分词，所以汉字只能查询一个字，英语是一个单词.
QueryBuilder queryBuilder=QueryBuilders.termQuery("fieldName", "fieldlValue");
//分词查询，采用默认的分词器
QueryBuilder queryBuilder2 = QueryBuilders.matchQuery("fieldName", "fieldlValue");

多个匹配
//不分词查询，参数1： 字段名，参数2：多个字段查询值,因为不分词，所以汉字只能查询一个字，英语是一个单词.
QueryBuilder queryBuilder=QueryBuilders.termsQuery("fieldName", "fieldlValue1","fieldlValue2...");
//分词查询，采用默认的分词器
QueryBuilder queryBuilder= QueryBuilders.multiMatchQuery("fieldlValue", "fieldName1", "fieldName2", "fieldName3");
//匹配所有文件，相当于就没有设置查询条件
QueryBuilder queryBuilder=QueryBuilders.matchAllQuery();
1234567891011121314
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

2.2.2 模糊查询（只要包含即可）

//模糊查询常见的5个方法如下
//1.常用的字符串查询
QueryBuilders.queryStringQuery("fieldValue").field("fieldName");//左右模糊
//2.常用的用于推荐相似内容的查询
QueryBuilders.moreLikeThisQuery(new String[] {"fieldName"}).addLikeText("pipeidhua");//如果不指定filedName，则默认全部，常用在相似内容的推荐上
//3.前缀查询  如果字段没分词，就匹配整个字段前缀
QueryBuilders.prefixQuery("fieldName","fieldValue");
//4.fuzzy query:分词模糊查询，通过增加fuzziness模糊属性来查询,如能够匹配hotelName为tel前或后加一个字母的文档，fuzziness 的含义是检索的term 前后增加或减少n个单词的匹配查询
QueryBuilders.fuzzyQuery("hotelName", "tel").fuzziness(Fuzziness.ONE);
//5.wildcard query:通配符查询，支持* 任意字符串；？任意一个字符
QueryBuilders.wildcardQuery("fieldName","ctr*");//前面是fieldname，后面是带匹配字符的字符串
QueryBuilders.wildcardQuery("fieldName","c?r?");
123456789101112
1
2
3
4
5
6
7
8
9
10
11
12
13

2.2.3 范围查询

//闭区间查询
QueryBuilder queryBuilder0 = QueryBuilders.rangeQuery("fieldName").from("fieldValue1").to("fieldValue2");
//开区间查询
QueryBuilder queryBuilder1 = QueryBuilders.rangeQuery("fieldName").from("fieldValue1").to("fieldValue2").includeUpper(false).includeLower(false);//默认是true，也就是包含
//大于
QueryBuilder queryBuilder2 = QueryBuilders.rangeQuery("fieldName").gt("fieldValue");
//大于等于
QueryBuilder queryBuilder3 = QueryBuilders.rangeQuery("fieldName").gte("fieldValue");
//小于
QueryBuilder queryBuilder4 = QueryBuilders.rangeQuery("fieldName").lt("fieldValue");
//小于等于
QueryBuilder queryBuilder5 = QueryBuilders.rangeQuery("fieldName").lte("fieldValue");
123456789101112
1
2
3
4
5
6
7
8
9
10
11
12
13

2.2.4组合查询/多条件查询/布尔查询

QueryBuilders.boolQuery()
QueryBuilders.boolQuery().must();//文档必须完全匹配条件，相当于and
QueryBuilders.boolQuery().mustNot();//文档必须不匹配条件，相当于not
QueryBuilders.boolQuery().should();//至少满足一个条件，这个文档就符合should，相当于or
1234
1
2
3
4
5

2.3 初始化elasticsearch的账号密码

采用spring对RestHighLevelClient进行初始化

@Configuration
public class ElasticsearchConfig {
    @Bean
    public RestHighLevelClient restHighLevelClient(){
        RestHighLevelClient client=new RestHighLevelClient(
                RestClient.builder(new HttpHost("localhost", 9200, "http")));
        return client;
    }
}
123456789
1
2
3
4
5
6
7
8
9
10

2.4 精华：elasticsearch工具类，直接复制使用即可

/**
 * es 的工具类
 *
 * @author czchen
 * @version 1.0
 * @date 2020/8/25 14:37
 */
@Slf4j
@Component
public class EsUtil {

    @Autowired
    private RestHighLevelClient restHighLevelClient;


    /**
     * 关键字
     */
    public static final String KEYWORD = ".keyword";

    /**
     * 创建索引
     *
     * @param index 索引
     * @return
     */
    public boolean createIndex(String index) throws IOException {
        if(isIndexExist(index)){
            log.error("Index is exits!");
            return false;
        }
        //1.创建索引请求
        CreateIndexRequest request = new CreateIndexRequest(index);
        //2.执行客户端请求
        CreateIndexResponse response = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);

        log.info("创建索引{}成功",index);

        return response.isAcknowledged();
    }

    /**
     * 删除索引
     *
     * @param index
     * @return
     */
    public boolean deleteIndex(String index) throws IOException {
        if(!isIndexExist(index)) {
            log.error("Index is not exits!");
            return false;
        }
        //删除索引请求
        DeleteIndexRequest request = new DeleteIndexRequest(index);
        //执行客户端请求
        AcknowledgedResponse delete = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);

        log.info("删除索引{}成功",index);

        return delete.isAcknowledged();
    }



    /**
     * 判断索引是否存在
     *
     * @param index
     * @return
     */
    public boolean isIndexExist(String index) throws IOException {

        GetIndexRequest request = new GetIndexRequest(index);

        boolean exists = restHighLevelClient.indices().exists(request, RequestOptions.DEFAULT);

        return exists;
    }



    /**
     * 数据添加，正定ID
     *
     * @param jsonObject 要增加的数据
     * @param index      索引，类似数据库
     * @param id         数据ID, 为null时es随机生成
     * @return
     */
    public String addData(JSONObject jsonObject, String index, String id) throws IOException {

        //创建请求
        IndexRequest request = new IndexRequest(index);
        //规则 put /test_index/_doc/1
        request.id(id);
        request.timeout(TimeValue.timeValueSeconds(1));
        //将数据放入请求 json
        IndexRequest source = request.source(jsonObject, XContentType.JSON);
        //客户端发送请求
        IndexResponse response = restHighLevelClient.index(request, RequestOptions.DEFAULT);
        log.info("添加数据成功 索引为: {}, response 状态: {}, id为: {}",index,response.status().getStatus(), response.getId());
        return response.getId();
    }



    /**
     * 数据添加 随机id
     *
     * @param jsonObject 要增加的数据
     * @param index      索引，类似数据库
     * @return
     */
    public String addData(JSONObject jsonObject, String index) throws IOException {
        return addData(jsonObject, index, UUID.randomUUID().toString().replaceAll("-", "").toUpperCase());
    }

    /**
     * 通过ID删除数据
     *
     * @param index 索引，类似数据库
     * @param id    数据ID
     */
    public void deleteDataById(String index, String id) throws IOException {
        //删除请求
        DeleteRequest request = new DeleteRequest(index, id);
        //执行客户端请求
        DeleteResponse delete = restHighLevelClient.delete(request, RequestOptions.DEFAULT);
        log.info("索引为: {}, id为: {}删除数据成功",index, id);
    }


    /**
     * 通过ID 更新数据
     *
     * @param object     要增加的数据
     * @param index      索引，类似数据库
     * @param id         数据ID
     * @return
     */
    public void updateDataById(Object object, String index, String id) throws IOException {
        //更新请求
        UpdateRequest update = new UpdateRequest(index, id);

        //保证数据实时更新
        //update.setRefreshPolicy("wait_for");

        update.timeout("1s");
        update.doc(JSON.toJSONString(object), XContentType.JSON);
        //执行更新请求
        UpdateResponse update1 = restHighLevelClient.update(update, RequestOptions.DEFAULT);
        log.info("索引为: {}, id为: {}, 更新数据成功",index, id);
    }


    /**
     * 通过ID 更新数据,保证实时性
     *
     * @param object     要增加的数据
     * @param index      索引，类似数据库
     * @param id         数据ID
     * @return
     */
    public void updateDataByIdNoRealTime(Object object, String index, String id) throws IOException {
        //更新请求
        UpdateRequest update = new UpdateRequest(index, id);

        //保证数据实时更新
        update.setRefreshPolicy("wait_for");

        update.timeout("1s");
        update.doc(JSON.toJSONString(object), XContentType.JSON);
        //执行更新请求
        UpdateResponse update1 = restHighLevelClient.update(update, RequestOptions.DEFAULT);
        log.info("索引为: {}, id为: {}, 更新数据成功",index, id);
    }


    /**
     * 通过ID获取数据
     *
     * @param index  索引，类似数据库
     * @param id     数据ID
     * @param fields 需要显示的字段，逗号分隔（缺省为全部字段）
     * @return
     */
    public Map<String,Object> searchDataById(String index, String id, String fields) throws IOException {
        GetRequest request = new GetRequest(index, id);
        if (StringUtils.isNotEmpty(fields)){
            //只查询特定字段。如果需要查询所有字段则不设置该项。
            request.fetchSourceContext(new FetchSourceContext(true,fields.split(","), Strings.EMPTY_ARRAY));
        }
        GetResponse response = restHighLevelClient.get(request, RequestOptions.DEFAULT);
        Map<String, Object> map = response.getSource();
        //为返回的数据添加id
        map.put("id",response.getId());
        return map;
    }

    /**
     * 通过ID判断文档是否存在
     * @param index  索引，类似数据库
     * @param id     数据ID
     * @return
     */
    public  boolean existsById(String index,String id) throws IOException {
        GetRequest request = new GetRequest(index, id);
        //不获取返回的_source的上下文
        request.fetchSourceContext(new FetchSourceContext(false));
        request.storedFields("_none_");
        return restHighLevelClient.exists(request, RequestOptions.DEFAULT);
    }

    /**
     * 获取低水平客户端
     * @return
     */
    public RestClient getLowLevelClient() {
        return restHighLevelClient.getLowLevelClient();
    }


    /**
     * 高亮结果集 特殊处理
     * map转对象 JSONObject.parseObject(JSONObject.toJSONString(map), Content.class)
     * @param searchResponse
     * @param highlightField
     */
    public List<Map<String, Object>> setSearchResponse(SearchResponse searchResponse, String highlightField) {
        //解析结果
        ArrayList<Map<String,Object>> list = new ArrayList<>();
        for (SearchHit hit : searchResponse.getHits().getHits()) {
            Map<String, HighlightField> high = hit.getHighlightFields();
            HighlightField title = high.get(highlightField);

            hit.getSourceAsMap().put("id", hit.getId());

            Map<String, Object> sourceAsMap = hit.getSourceAsMap();//原来的结果
            //解析高亮字段,将原来的字段换为高亮字段
            if (title!=null){
                Text[] texts = title.fragments();
                String nTitle="";
                for (Text text : texts) {
                    nTitle+=text;
                }
                //替换
                sourceAsMap.put(highlightField,nTitle);
            }
            list.add(sourceAsMap);
        }
        return list;
    }


    /**
     * 查询并分页
     * @param index          索引名称
     * @param query          查询条件
     * @param size           文档大小限制
     * @param from           从第几页开始
     * @param fields         需要显示的字段，逗号分隔（缺省为全部字段）
     * @param sortField      排序字段
     * @param highlightField 高亮字段
     * @return
     */
    public List<Map<String, Object>> searchListData(String index,
                                                    SearchSourceBuilder query,
                                                    Integer size,
                                                    Integer from,
                                                    String fields,
                                                    String sortField,
                                                    String highlightField) throws IOException {
        SearchRequest request = new SearchRequest(index);
        SearchSourceBuilder builder = query;
        if (StringUtils.isNotEmpty(fields)){
            //只查询特定字段。如果需要查询所有字段则不设置该项。
            builder.fetchSource(new FetchSourceContext(true,fields.split(","),Strings.EMPTY_ARRAY));
        }
        from = from <= 0 ? 0 : from*size;
        //设置确定结果要从哪个索引开始搜索的from选项，默认为0
        builder.from(from);
        builder.size(size);
        if (StringUtils.isNotEmpty(sortField)){
            //排序字段，注意如果proposal_no是text类型会默认带有keyword性质，需要拼接.keyword
            builder.sort(sortField+".keyword", SortOrder.ASC);
        }
        //高亮
        HighlightBuilder highlight = new HighlightBuilder();
        highlight.field(highlightField);
        //关闭多个高亮
        highlight.requireFieldMatch(false);
        highlight.preTags("<span style='color:red'>");
        highlight.postTags("</span>");
        builder.highlighter(highlight);
        //不返回源数据。只有条数之类的数据。
        //builder.fetchSource(false);
        request.source(builder);
        SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
        log.error("=="+response.getHits().getTotalHits());
        if (response.status().getStatus() == 200) {
            // 解析对象
            return setSearchResponse(response, highlightField);
        }
        return null;
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307

2.5 个人测试-单元测试

package cn.stylefeng.guns.modular.es.entity;

import cn.hutool.core.bean.BeanUtil;
import cn.hutool.json.JSONObject;
import cn.hutool.json.JSONUtil;
import cn.stylefeng.guns.GunsApplication;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import javax.annotation.Resource;
import java.io.IOException;
import java.util.Map;

@Slf4j
@RunWith(SpringRunner.class)
@SpringBootTest(classes = GunsApplication.class)
public class UserTest {

    @Resource
    private ElasticsearchRestTemplate template;

    @Resource
    private RestHighLevelClient client;

    @Test
    public void testCreateIndexAndDoc() throws IOException {

        //1.创建索引请求
        CreateIndexRequest request = new CreateIndexRequest("user");
        //2.执行客户端请求
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        log.info("创建索引{}成功", response);
        System.out.println("response.isAcknowledged() = " + response.isAcknowledged());


    }

    @Test
    public void saveDoc() throws IOException {
        String index = "1";

        User user = new User();
        user.setId(1L);
        user.setAge(20);
        user.setName("狗蛋");
        user.setPassword("23243424");

        //创建请求
        IndexRequest request = new IndexRequest("user");

        //规则 put /test_index/_doc/1
        request.id(index);
        request.timeout(TimeValue.timeValueSeconds(1));
        //将数据放入请求 json
        request.source(user, XContentType.JSON);
        //客户端发送请求
        IndexResponse response = client.index(request, RequestOptions.DEFAULT);
        log.info("添加数据成功 索引为: {}, response 状态: {}, id为: {}",index,response.status().getStatus(), response.getId());
        log.info("=========={}",response.getId());

    }

    @Test
    public void saveDoc2() throws IOException {
        IndexRequest request = new IndexRequest("user");

        User user = new User();
        user.setId(1L);
        user.setAge(20);
        user.setName("狗蛋");
        user.setPassword("23243424");

//        String s = JSONUtil.pa

        request.id("1");
        String jsonString = "{" +
                "\"id\":\"kimchy\"," +
                "\"postDate\":\"2013-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}";


//        request.source(jsonString, XContentType.JSON);
        Map<String, Object> stringObjectMap = BeanUtil.beanToMap(user);
        request.source(stringObjectMap);

//        request.source()

        //客户端发送请求
        IndexResponse response = client.index(request, RequestOptions.DEFAULT);
//        IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);

        log.info("添加数据成功 索引为: {}, response 状态: {}, id为: {}","user",response.status().getStatus(), response.getId());
        log.info("=========={}",response.getId());


    }


    @Test
    public void testStrToBean() {
        User user = new User();
        user.setId(1L);
        user.setAge(20);
        user.setName("狗蛋");
        user.setPassword("23243424");

        JSONObject jsonObject = JSONUtil.parseObj(user);

        System.out.println(jsonObject.toString());
    }

    @Test
    public void saveDocRandomId() throws IOException {
        IndexRequest request = new IndexRequest("user");

        User user = new User();
        user.setId(2L);
        user.setAge(30);
        user.setName("狗蛋333");
        user.setPassword("23dsfasf243424");

//        request.id("1");

        Map<String, Object> stringObjectMap = BeanUtil.beanToMap(user);
        request.source(stringObjectMap);

//        request.source()

        //客户端发送请求
        IndexResponse response = client.index(request, RequestOptions.DEFAULT);
//        IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);

        log.info("添加数据成功 索引为: {}, response 状态: {}, id为: {}","user",response.status().getStatus(), response.getId());
        log.info("=========={}",response.getId());

    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150

三、ES的通用ElasticsearchRepository

因为是基于springboot,那就不再使用kibana进行建立索引,而是用springboot来创建.创建索引的api:
ElasticsearchRestTemplate.createIndex(实体类.class)
如,现在创建一个名为esBean的实体类

相当于建库:
@Document用于指定索引
@Field(type = FieldType.Text,analyzer = “ik_max_word”)用于指定类型

@Data
@NoArgsConstructor
@Document(indexName = "test")
public class esBean {
    @Id
    private int id;
    @Field(type = FieldType.Text,analyzer = "ik_max_word")
    private String name;
    @Field(type = FieldType.Text,analyzer = "ik_max_word")
    private String tags;
    public esBean(int id,String name,String tags){
        this.id=id;
        this.name=name;
        this.tags=tags;
    }
}



12345678910111213141516171819
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

service层:

public class esService  implements IesService {

    @Autowired
    private ElasticsearchRestTemplate template;
    @Override
    public void create(){
        template.createIndex(esBean.class);
    }
}
123456789
1
2
3
4
5
6
7
8
9
10

像这样,就会自动建立一个名为test的索引,表结果就是esBean的成员

存储数据

需要一个持久化层的接口去继承ElasticsearchRepository,接着在service层中,依赖注入,直接调用其saveAll()方法即可,如下:

持久层:(可以在这里做一些搜索查询)

@Service
public interface esMapper extends ElasticsearchRepository<esBean,Long> {

}

12345
1
2
3
4
5
6

service层:

可以看到,saveAll,findAll这些方法是原来就有的,无需自己定义

public class esService  implements IesService {

    @Autowired
    private ElasticsearchRestTemplate template;
    @Autowired
    esMapper mapper;
    @Override
    public Iterator<esBean> findAll() {
        return mapper.findAll().iterator();
    }
    public void create(){
        template.createIndex(esBean.class);
    }

    @Override
    public void saveAll(List<esBean> list) {
            mapper.saveAll(list);
    }
}
12345678910111213141516171819
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

controller层:

@Autowired
    IesService iesService;

 @GetMapping("/init1")
    public String init1(){
         iesService.create();
         // id name tags
        List<esBean>list=new ArrayList<>();
        list.add(new esBean(1,"张三锕爱","很帅,有很多人喜欢他"));
        list.add(new esBean(2,"李四酷狗","很帅,但是讨厌他"));
        list.add(new esBean(3,"王五王二爷","很丑,有是喜欢他"));
        list.add(new esBean(4,"张三王二婆","很帅,有没人喜欢他"));
        iesService.saveAll(list);
        return "success";
    }
123456789101112131415
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

ElasticsearchRestTemplate的基本api

SearchQuery 总的查询
BoolQueryBuilder bool查询,可在后面加上must,mustNot,should等等
MatchQueryBuilder 匹配查询
TermQueryBuilder 倒排索引查询
HighlightBuilder 高亮查询,用于设置要高亮的field
SearchHit 查询结果

实现文本高亮检索步骤

实体类:

@Data
@NoArgsConstructor
@Document(indexName = "test5")
public class esBean {
    @Id
    private int id;
    @Field(type = FieldType.Text,analyzer = "ik_max_word")
    private String name;
    @Field(type = FieldType.Text,analyzer = "ik_max_word")
    private String tags;
    public esBean(int id,String name,String tags){
        this.id=id;
        this.name=name;
        this.tags=tags;
    }
}

1234567891011121314151617
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

持久层

@Service
public interface esMapper extends ElasticsearchRepository<esBean,Long> {

}

123456
1
2
3
4
5
6

service:

public interface IesService {
    public Iterator<esBean> findAll();
    public void saveAll(List<esBean> list);
    public void create();
    public AggregatedPage testForHigh() throws IOException;
}
123456
@Service
public class esService  implements IesService {

    @Autowired
    private ElasticsearchRestTemplate template;
    @Autowired
    esMapper mapper;
    @Override
    public Iterator<esBean> findAll() {
        return mapper.findAll().iterator();
    }
    public void create(){
        template.createIndex(esBean.class);
    }

    @Override
    public void saveAll(List<esBean> list) {
            mapper.saveAll(list);
    }

     public AggregatedPage  testForHigh() throws IOException {
        String preTag = "<font color='#dd4b39'>";//google的色值
        String postTag = "</font>";
        BoolQueryBuilder boolQueryBuilder=new BoolQueryBuilder()
                .must(new MatchQueryBuilder("tags","很帅"));
        SearchQuery searchQuery=new NativeSearchQueryBuilder().   //总的查询
                withQuery(boolQueryBuilder).           //设置bool查询
                withHighlightFields(new HighlightBuilder.Field("name").preTags(preTag).postTags(postTag)).//设置高亮效果
                withHighlightFields(new HighlightBuilder.Field("tags").preTags(preTag).postTags(postTag)).build();
        AggregatedPage<esBean> ideas=template.queryForPage(searchQuery, esBean.class, new SearchResultMapper() {
            @Override
            public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> aClass, Pageable pageable) {
                List<esBean> list = new ArrayList<>();
                for(SearchHit  hit:searchResponse.getHits()){//获取遍历查询结果
                    if(searchResponse.getHits().getHits().length<=0)return null;
                    esBean bean=new esBean();
                    Map map=hit.getSourceAsMap();
                    System.out.println(map);
                    bean.setId((Integer)map.get("id"));
                    bean.setName((String)map.get("name"));
                    HighlightField name=hit.getHighlightFields().get("name");
                    if(name!=null){
                        bean.setName(name.fragments()[0].toString());   //得到高亮的结果
                    }
                    HighlightField tags=hit.getHighlightFields().get("tags");
                    if(tags!=null){
                        bean.setTags(tags.fragments()[0].toString());
                    }
                    list.add(bean);
                }
                if(list.size()>0)return new AggregatedPageImpl<>((List<T>)list);

                return null;
            }

            @Override
            public <T> T mapSearchHit(SearchHit searchHit, Class<T> aClass) {
                return null;
            }
        });
        ideas.get().forEach(model->{
            System.out.println(model);
        });
        return ideas;
    }

            @Override
            public <T> T mapSearchHit(SearchHit searchHit, Class<T> aClass) {
                return null;
            }
        });
        ideas.get().forEach(model->{
            System.out.println(model);
        });
        return ideas;
    }
}


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

controller:

@RestController
@RequestMapping("/elastic")
public class ElasticController {

    @Autowired
    IesService iesService;
    

    @GetMapping("/init1")
    public String init1(){
         iesService.create();
         // id name tags
        List<esBean>list=new ArrayList<>();
        list.add(new esBean(1,"张三锕爱","很帅,有很多人喜欢他"));
        list.add(new esBean(2,"李四酷狗","很帅,但是讨厌他"));
        list.add(new esBean(3,"王五王二爷","很丑,有是喜欢他"));
        list.add(new esBean(4,"张三王二婆","很帅,有没人喜欢他"));
        iesService.saveAll(list);
        return "success";
    }
    @GetMapping("/get1")
    public AggregatedPage get1() throws IOException {
        return iesService.testForHigh();
    }

}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

postman访问:

四、ElasticsearchRepository的使用和复杂查询实现

Spring Data 的强大之处，就在于你不用写任何DAO处理，自动根据方法名或类的信息进行CRUD操作。只要你定义一个接口，然后继承Repository提供的一些子接口，就能具备各种基本的CRUD功能。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-D7M8jsnj-1606058387968)(/Users/allen/Documents/TyporaFiles/gulimall/resource/%25E8%25B5%2584%25E6%2596%2599%25E6%2596%2587%25E6%25A1%25A3/%25E9%2598%25B6%25E6%25AE%25B5md/assets/1575806287671.png)]

其中ElasticsearchRepository接口功能最强大。该接口的方法包括：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-DV78cXpV-1606058387976)(/Users/allen/Documents/TyporaFiles/gulimall/resource/%25E8%25B5%2584%25E6%2596%2599%25E6%2596%2587%25E6%25A1%25A3/%25E9%2598%25B6%25E6%25AE%25B5md/assets/1575806405547.png)]

4.4.1. 新增

@Autowired
UserRepository userRepository;

@Test
void testAdd(){
    this.userRepository.save(new User(1l, "zhang3", 20, "123456"));
}
1
2
3
4
5
6
7

修改和新增是同一个接口，区分的依据就是id，这一点跟我们在页面发起PUT请求是类似的。

4.4.2. 删除

@Test
void testDelete(){
    this.userRepository.deleteById(1l);
}
1
2
3
4

4.5. 查询

4.5.1. 基本查询

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-l4Bx6eRd-1606058387984)(/Users/allen/Documents/TyporaFiles/gulimall/resource/%25E8%25B5%2584%25E6%2596%2599%25E6%2596%2587%25E6%25A1%25A3/%25E9%2598%25B6%25E6%25AE%25B5md/assets/1575848896764.png)]

查询一个：

@Test
void testFind(){
    System.out.println(this.userRepository.findById(1l).get());
}
1
2
3
4

4.5.2. 条件查询

Spring Data 的另一个强大功能，是根据方法名称自动实现功能。

比如：你的方法名叫做：findByTitle，那么它就知道你是根据title查询，然后自动帮你完成，无需写实现类。

当然，方法名称要符合一定的约定：

准备一组数据：

@Test
void testAddAll(){
    List<User> users = new ArrayList<>();
    users.add(new User(1l, "柳岩", 18, "123456"));
    users.add(new User(2l, "范冰冰", 19, "123456"));
    users.add(new User(3l, "李冰冰", 20, "123456"));
    users.add(new User(4l, "锋哥", 21, "123456"));
    users.add(new User(5l, "小鹿", 22, "123456"));
    users.add(new User(6l, "韩红", 23, "123456"));
    this.userRepository.saveAll(users);
}
1
2
3
4
5
6
7
8
9
10
11

在UserRepository中定义一个方法：

第一种写法：

public interface UserRepository extends ElasticsearchRepository<User, Long> {

    /**
     * 根据年龄区间查询
     * @param age1
     * @param age2
     * @return
     */
    List<User> findByAgeBetween(Integer age1, Integer age2);
}
1
2
3
4
5
6
7
8
9
10

测试：

@Test
void testFindByAgeBetween(){
    System.out.println(this.userRepository.findByAgeBetween(20, 30));
}
1
2
3
4

第二种写法：

这个就牛逼了，可以实现复杂查询

@Query("{\n" +
       "    \"range\": {\n" +
       "      \"age\": {\n" +
       "        \"gte\": \"?0\",\n" +
       "        \"lte\": \"?1\"\n" +
       "      }\n" +
       "    }\n" +
       "  }")
List<User> findByQuery(Integer age1, Integer age2);
1
2
3
4
5
6
7
8
9

测试：

@Test
void testFindByQuery(){
	System.out.println(this.userRepository.findByQuery(20, 30));
}
1
2
3
4

4.5.3. 自定义查询

@Test
void testNative(){
    // 初始化自定义查询对象
    NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
    // 构建查询
    queryBuilder.withQuery(QueryBuilders.matchQuery("name", "冰冰"));
    // 排序
    queryBuilder.withSort(SortBuilders.fieldSort("age").order(SortOrder.ASC));
    // 分页
    queryBuilder.withPageable(PageRequest.of(0, 2));
    // 高亮
    queryBuilder.withHighlightBuilder(new HighlightBuilder().field("name").preTags("<em>").postTags("</em>"));
    // 执行查询，获取分页结果集
    Page<User> userPage = this.userRepository.search(queryBuilder.build());
    // 总页数
    System.out.println(userPage.getTotalPages());
    // 总记录数
    System.out.println(userPage.getTotalElements());
    // 当前页数据
    System.out.println(userPage.getContent());
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

NativeSearchQueryBuilder：Spring提供的一个查询条件构建器，帮助构建json格式的请求体

Page<item>：默认是分页查询，因此返回的是一个分页的结果对象，包含属性：

• totalElements：总条数
• totalPages：总页数
• Iterator：迭代器，本身实现了Iterator接口，因此可直接迭代得到当前页的数据

搜索address中包含mill的所有人的年龄分布以及平均年龄，平均薪资

GET bank/_search
{
  "query": {
    "match": {
      "address": "Mill"
    }
  },
  "aggs": {
    "ageAgg": {
      "terms": {
        "field": "age",
        "size": 10
      }
    },
    "ageAvg": {
      "avg": {
        "field": "age"
      }
    },
    "balanceAvg": {
      "avg": {
        "field": "balance"
      }
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

java实现

    /**
     * 复杂检索:在bank中搜索address中包含mill的所有人的年龄分布以及平均年龄，平均薪资
     * @throws IOException
     */
    @Test
    public void searchData() throws IOException {
        //1. 创建检索请求
        SearchRequest searchRequest = new SearchRequest();

        //1.1）指定索引
        searchRequest.indices("bank");
        //1.2）构造检索条件
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(QueryBuilders.matchQuery("address","Mill"));

        //1.2.1)按照年龄分布进行聚合
        TermsAggregationBuilder ageAgg=AggregationBuilders.terms("ageAgg").field("age").size(10);
        sourceBuilder.aggregation(ageAgg);

        //1.2.2)计算平均年龄
        AvgAggregationBuilder ageAvg = AggregationBuilders.avg("ageAvg").field("age");
        sourceBuilder.aggregation(ageAvg);
        //1.2.3)计算平均薪资
        AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
        sourceBuilder.aggregation(balanceAvg);

        System.out.println("检索条件："+sourceBuilder);
        searchRequest.source(sourceBuilder);
        //2. 执行检索
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("检索结果："+searchResponse);

        //3. 将检索结果封装为Bean
        SearchHits hits = searchResponse.getHits();
        SearchHit[] searchHits = hits.getHits();
        for (SearchHit searchHit : searchHits) {
            String sourceAsString = searchHit.getSourceAsString();
            Account account = JSON.parseObject(sourceAsString, Account.class);
            System.out.println(account);

        }

        //4. 获取聚合信息
        Aggregations aggregations = searchResponse.getAggregations();

        Terms ageAgg1 = aggregations.get("ageAgg");

        for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
            String keyAsString = bucket.getKeyAsString();
            System.out.println("年龄："+keyAsString+" ==> "+bucket.getDocCount());
        }
        Avg ageAvg1 = aggregations.get("ageAvg");
        System.out.println("平均年龄："+ageAvg1.getValue());

        Avg balanceAvg1 = aggregations.get("balanceAvg");
        System.out.println("平均薪资："+balanceAvg1.getValue());


    }  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

可以尝试对比打印的条件和执行结果，和前面的ElasticSearch的检索语句和检索结果进行比较；

五、总结

据说ElasticsearchRepository并不好用，我还没深度使用，盲猜是复杂查询的时候嵌套和平行关系有点乱哈哈，那么问题来了，如果javaAPI里面能使用原生restAPI查询就好了，借助kibana直接写完就可以放到代码中执行，美滋滋。有空继续研究一下。