赞
踩
ElasticSearch作为基于Lucene的一款分布式全文检索服务器,可以通过暴露restfulAPI来操作索引、搜索,具备有实时搜索、稳定、可靠、快速、安装使用方便等特点,是目前使用最广泛的企业级搜索引擎。
本期的目的就是——在springBoot项目当中,使用elasticSearch的java客户端,来实现与ES服务进行交互,涉及的内容主要包括以下几部分:
1、基本资料准备;
2、索引(index)的增删;
3、文档(document)的增删改和简单查;
4、文档(document)的DSL查询(重点);
注意:所有的具体ES语法演示参照kibana来,结果展示参照head插件来和idea控制台,java代码交互借助idea来实现!
所用的ES、可视化管理插件kibana和es_head都用的7.6.2版本;
注意:springBoot整合的elasticSearch是个混合依赖,ES服务的版本为7.6.2,所以请确保自己的ES服务版本要对齐,避免版本冲突问题;
- <dependencies>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-web</artifactId>
- <version>2.3.2.RELEASE</version>
- </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-test</artifactId>
- <version>2.3.2.RELEASE</version>
- </dependency>
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>4.13.2</version>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
- <version>2.3.2.RELEASE</version>
- </dependency>
- </dependencies>
spring-boot-starter-data-elasticsearch当中的子依赖及版本:
- @SpringBootApplication
- public class ElasticSearchApplication {
- public static void main(String[] args) {
- SpringApplication.run(ElasticSearchApplication.class,args);
- }
- }
- @Configuration
- public class ElasticSearchClientConfig {
- @Bean
- public RestHighLevelClient restHighLevelClient(){
- //获取可以用来操作ES的java客户端
- RestHighLevelClient restHighLevelClient = new RestHighLevelClient(
- RestClient.builder(new HttpHost("192.168.245.129", 9200,"http")));
- return restHighLevelClient;
- }
- }
可以提前准备两个测试类(也可以后边再准备):
- import com.qf.ElasticSearchApplication;
- import org.elasticsearch.client.RestHighLevelClient;
- import org.junit.runner.RunWith;
- import org.springframework.beans.factory.annotation.Autowired;
- import org.springframework.boot.test.context.SpringBootTest;
- import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
-
- /**
- * ClassName: TestDemo1
- *
- * @author Guan
- * Description : 本类主要是做索引(创建、删除、查询)、文档(创建、删除、修改、简单查询)的
- * date: 2024/1/29 22:47
- */
- @RunWith(SpringJUnit4ClassRunner.class)
- @SpringBootTest(classes = ElasticSearchApplication.class)
- public class TestDemo1 {
- @Autowired
- private RestHighLevelClient restHighLevelClient;
-
- }
- import com.qf.ElasticSearchApplication;
- import org.elasticsearch.client.RestHighLevelClient;
- import org.junit.runner.RunWith;
- import org.springframework.beans.factory.annotation.Autowired;
- import org.springframework.boot.test.context.SpringBootTest;
- import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
-
- /**
- * ClassName: TestDemo2
- *
- * @author Guan
- * Description : 本类主要是做文档的DSL查询操作的!
- * date: 2024/1/29 22:50
- */
- @RunWith(SpringJUnit4ClassRunner.class)
- @SpringBootTest(classes = ElasticSearchApplication.class)
- public class TestDemo2 {
- @Autowired
- private RestHighLevelClient restHighLevelClient;
-
- }
请求语法(kibana中书写)
java Client实现
在TestDemo1当中继续写,映射的那部分内容可以直接从kibana当中拷贝:
- @DisplayName("创建索引,同时创建映射")
- @Test
- public void testCreatIndex() throws Exception{
- //获取创建索引请求对象
- CreateIndexRequest createIndexRequest = new CreateIndexRequest("book");
- //设置索引的初始化设置
- createIndexRequest.settings(Settings.builder().
- put("number_of_shards",2).
- put("number_of_replicas", 1));
- //在index当中创建映射
- createIndexRequest.mapping("{\n" +
- " \"_source\": {\n" +
- " \"excludes\": [\"description\"]\n" +
- " },\n" +
- " \"properties\": {\n" +
- " \"id\": {\n" +
- " \"type\": \"keyword\"\n" +
- " },\n" +
- " \"name\": {\n" +
- " \"type\": \"text\",\n" +
- " \"analyzer\": \"ik_max_word\",\n" +
- " \"search_analyzer\": \"ik_smart\"\n" +
- " },\n" +
- " \"price\": {\n" +
- " \"type\": \"float\"\n" +
- " },\n" +
- " \"description\": {\n" +
- " \"type\": \"text\",\n" +
- " \"analyzer\": \"ik_max_word\",\n" +
- " \"search_analyzer\": \"ik_smart\"\n" +
- " },\n" +
- " \"timestamp\": {\n" +
- " \"type\": \"date\",\n" +
- " \"format\": \"yyyy-MM-dd HH:mm:ss || yyyy-MM-dd\"\n" +
- " }\n" +
- " }\n" +
- "}", XContentType.JSON);
- //根据索引操作客户端、获取创建索引请求的响应对象
- CreateIndexResponse createIndexResponse = restHighLevelClient.
- indices().create(createIndexRequest, RequestOptions.DEFAULT);
- //查看响应结果
- System.out.println(createIndexResponse.isAcknowledged());
- }
操作结果:
ES语法
java Client实现
- @DisplayName("删除索引")
- @Test
- public void testCreateMapping() throws Exception{
- //获取删除索引请求对象
- DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest();
- deleteIndexRequest.indices("book");
- //根据索引操作客户端,获取响应对象
- AcknowledgedResponse deleteResponse = restHighLevelClient.indices().
- delete(deleteIndexRequest, RequestOptions.DEFAULT);
- System.out.println("是否删除成功? "+deleteResponse.isAcknowledged());
- }
操作结果:
ES语法
java client的实现
- @DisplayName("查看索引")
- @Test
- public void testQueryIndex() throws Exception {
- //获取查询索引请求对象
- GetIndexRequest getIndexRequest = new GetIndexRequest("book");
- //根据索引操作客户端、获取响应对象
- GetIndexResponse getIndexResponse = restHighLevelClient.indices().
- get(getIndexRequest, RequestOptions.DEFAULT);
- //解析响应结果
- Map<String, Settings> settings = getIndexResponse.getSettings();
- System.out.println(settings.toString());
- }
查询结果:
由于ES在7.x版本之后,已经废除了type的概念,所以可以用_doc来替代之前所用的type_name。
ES语法
java client实现
- @DisplayName("创建文档")
- @Test
- public void createDocument() throws Exception{
- //获取索引请求对象
- IndexRequest indexRequest = new IndexRequest("book","_doc","2");
- //创建文档
- indexRequest.source("{\n" +
- " \"id\": \"1002\",\n" +
- " \"name\": \"mybatis入门课程\",\n" +
- " \"decription\": \"mybatis入门课程主要是围绕着:原生jdbc案例分析、mybatis基本概念、mybatis的映射文件、mybatis的注解实现等几个方面入手;\",\n" +
- " \"timestamp\": \"2023-12-22\",\n" +
- " \"price\": 22.5\n" +
- "}",XContentType.JSON);
- //获取操作响应对象
- IndexResponse indexResponse = restHighLevelClient.
- index(indexRequest, RequestOptions.DEFAULT);
- //查看操作结果
- System.out.println(indexResponse.status());
- System.out.println(indexResponse.toString());
- }
ps:文件内容可以直接从kibana上边复制;
控制台显示结果:
head插件显示结果:
ES语法
注意添加的时候,不论是对index的指定、document内容的指定,都要注意键值之间都不要写空格+花括号{}千万别换行,否则容易出现错误开始异常;
java client的实现
- @DisplayName("批量添加文档")
- @Test
- public void testBulkAddDocument() throws Exception {
- //获取批量操作请求对象
- BulkRequest bulkRequest = new BulkRequest();
- //批量添加document
- bulkRequest.add(new IndexRequest("book", "_doc", "3").
- source("{\"id\":\"1003\",\"name\":\"安徒生童话\",\"decription\":\"《安徒生童话》是丹麦作家安徒生创作的童话集,共由166篇故事组成。该作爱憎分明,热情歌颂劳动人民、赞美他们的善良和纯洁的优秀品德;无情地揭露和批判王公贵族们的愚蠢、无能、贪婪和残暴。其中较为闻名的故事有:《小人鱼》《丑小鸭》《卖火柴的小女孩》、《拇指姑娘》等;\",\"timestamp\":\"2020-06-01\",\"price\":18.9}", XContentType.JSON));
- bulkRequest.add(new IndexRequest("book","_doc","4").
- source("{\"id\":\"1004\",\"name\":\"ElasticSearch\",\"description\":\"Elasticsearch 是一个分布式搜索引擎,底层基于 Lucene 实现。Elasticsearch 屏蔽了Lucene的底层细节,提供了分布式特性,同时对外提供了 Restful API。Elasticsearch以其易用性迅速赢得了许多用户 被用在网站搜索、日志分析等诸多方面。由于 ES强大的横向扩展能力, 甚至很多人也会直接把 ES 当做 NoSQL 来用;\",\"timestamp\":\"2019-11-11\",\"price\":25.7}",XContentType.JSON));
- //获取操作结果
- BulkResponse bulkResponse = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
- //查看操作结果
- System.out.println("是否失败?"+bulkResponse.hasFailures());
- }
操作结果:
控制台结果:
head插件显示结果:
ES语法
此处不论是put还是post,都可以实现document的修改操作!
java client实现
- @DisplayName("修改id为4的文档")
- @Test
- public void testUpdateDocument() throws Exception{
- //获取修改文档的请求对象
- UpdateRequest updateRequest = new UpdateRequest("book","_doc","4");
- //修改内容
- updateRequest.doc("{\n" +
- " \"id\": \"1004\",\n" +
- " \"name\": \"ElasticSearch入门教程\",\n" +
- " \"decription\": \"本课程主要是围绕着ES的基本概念、ES的下载安装、相关插件的学习、基本语法和java整合ES服务等展开;\",\n" +
- " \"timestamp\": \"2023-12-22\",\n" +
- " \"price\": 27.5\n" +
- "}",XContentType.JSON);
- //执行修改操作,获取响应对象
- UpdateResponse update = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
- //查看响应结果
- System.out.println(update.status());
- System.out.println(update.toString());
-
- }
具体操作结果:
控制台:
head插件:
ES语法
java client实现
- @DisplayName("删除文档")
- @Test
- public void testDeleteDocument() throws Exception{
- //创建删除文档请求对象
- DeleteRequest deleteRequest = new DeleteRequest("book","_doc","4");
- //获取响应对象
- DeleteResponse delete = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
- //解析响应结果
- System.out.println(delete.status());
- System.out.println(delete.getResult());
- }
具体操作结果:
ES语法
java client实现
- @DisplayName("根据id简单查询稳定")
- @Test
- public void testQueryDocumentById() throws Exception{
- //获取查询文档id
- GetRequest getRequest = new GetRequest("book","_doc","1");
- //获取查询响应结果
- GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
- //解析查询结果
- System.out.println(getResponse.getSourceAsString());
- }
结果:
控制台结果:
DSL(Domain Specific Language)是ES提出的基于json的搜索方式,在搜索的时候,只需要传入特定的json格式数据,就可以完成不同的搜索需求。
DSL的搜索方式功能比URI的形式要更强大,一般在项目当中使用DSL的方式开完成搜索会比较多。注意:接下来的所有实现代码都是放在TestDemo2当中写的。
ES语法:
javaClient的实现:
- import com.qf.ElasticSearchApplication;
- import org.apache.lucene.search.TotalHits;
- import org.elasticsearch.action.search.SearchRequest;
- import org.elasticsearch.action.search.SearchResponse;
- import org.elasticsearch.client.RequestOptions;
- import org.elasticsearch.client.RestHighLevelClient;
- import org.elasticsearch.index.query.QueryBuilders;
- import org.elasticsearch.search.SearchHit;
- import org.elasticsearch.search.SearchHits;
- import org.elasticsearch.search.builder.SearchSourceBuilder;
- import org.junit.After;
- import org.junit.Before;
- import org.junit.Test;
- import org.junit.jupiter.api.DisplayName;
- import org.junit.runner.RunWith;
- import org.springframework.beans.factory.annotation.Autowired;
- import org.springframework.boot.test.context.SpringBootTest;
- import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
-
- import java.text.SimpleDateFormat;
-
- /**
- * ClassName: TestDemo2
- *
- * @author Guan
- * Description : 本类主要是做文档的DSL查询操作的!
- * date: 2024/1/29 22:50
- */
- @RunWith(SpringJUnit4ClassRunner.class)
- @SpringBootTest(classes = ElasticSearchApplication.class)
- public class TestDemo2 {
- @Autowired
- private RestHighLevelClient restHighLevelClient;
-
- //提前准备好搜索的请求和响应变量
- SearchRequest searchRequest;
- SearchResponse searchResponse;
-
- @DisplayName("测试前作准备工作:获取searchRequest对象")
- @Before
- public void before(){
- searchRequest = new SearchRequest();
- searchRequest.indices("book");
- }
-
- @DisplayName("测试(搜索)过程结束之后,用于解析响应对象")
- @After
- public void after() throws Exception{
- //获取搜索匹配的结果
- SearchHits hits = searchResponse.getHits();
- //获取搜索的总记录(不是数)
- TotalHits totalHits = hits.getTotalHits();
- System.out.println(totalHits.toString());
- //获取匹配的文档
- SearchHit[] searchHits = hits.getHits();
- //涉及到日期对象,要做日期对象的转化
- SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
- for (SearchHit hit : searchHits) {
- //获取文档的id
- String id = hit.getId();
- //获取源文档的内容
- String sourceAsString = hit.getSourceAsString();
- System.out.println("id: "+id+" ,内容为:"+sourceAsString);
- }
- }
-
- @DisplayName("查询所有记录")
- @Test
- public void testSearchAll() throws Exception{
- //创建搜索源对象
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- //设置搜索规则--查询所有
- searchSourceBuilder.query(QueryBuilders.matchAllQuery());
- //设置搜索源
- searchRequest.source(searchSourceBuilder);
- //执行搜索操作
- searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
- }
- }
查询结果:
ES语法:
java client实现:
- @DisplayName("分页查询")
- @Test
- public void testPageSearch() throws Exception{
- //创建搜索源对象
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- searchSourceBuilder.query(QueryBuilders.matchAllQuery());
- //设置从第几条记录开始查询
- searchSourceBuilder.from(1);
- //设置查几条记录
- searchSourceBuilder.size(2);
- //设置查询结果按照价格升序排序
- searchSourceBuilder.sort("price", SortOrder.ASC);
- //设置搜索源
- searchRequest.source(searchSourceBuilder);
- //执行查询结果
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
查询结果:
match Query即全文检索,它是一种将搜索内容先分词,然后再使用各个词条去索引当中搜索记录的方式。
涉及的特定json串:query----后边跟搜索的关键字;operator----后边跟or或者and,表示搜索内容只要含有分词后的一个词条就可以检索出来,还是搜索内容必须包含所有分词;
ES语法:
javaClient实现:
- @DisplayName("全文检索:fild_name为name,检索内容为'入门课程',检索结果必须包含所有分词")
- @Test
- public void testMatchSearch() throws Exception{
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- searchSourceBuilder.query(
- QueryBuilders.matchQuery("name","入门课程").operator(Operator.AND));
- searchRequest.source(searchSourceBuilder);
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
控制台结果:
match全文检索是针对某个field来展开的,需要在某个field当中去做匹配;而multi_match是把关键字放在了多个field当中去匹配;
ES语法:
javaClient的实现:
- @DisplayName("multi_match全文检索:将'入门课程'放在属性name和description当中检索")
- @Test
- public void testMultiMatchSearch() throws Exception{
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- searchSourceBuilder.query(QueryBuilders.multiMatchQuery("入门课程","name","description"));
- searchRequest.source(searchSourceBuilder);
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
控制台显示结果:
bool查询对应lucene当中的boolQuery,它是一种支持将多个查询条件组合起来检索的方式。它有三个参数:must(多个条件必须都满足),should(只要满足一个条件就行),must not(必须不在所有条件内);
ES语法:
java client实现:
- @DisplayName("bool查询:eg--查询所有name当中有“课程”,并且价格在20-30范围内的记录")
- @Test
- public void testBoolSearch() throws Exception {
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- //创建bool查询的构建对象
- BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
- //设置查询名字必须含有“课程”
- boolQueryBuilder.must(QueryBuilders.matchQuery("name","课程"));
- boolQueryBuilder.must(QueryBuilders.rangeQuery("price").gte(20).lte(30));
- //将bool查询规则应用在搜索源上
- searchSourceBuilder.query(boolQueryBuilder);
- //设置搜索源
- searchRequest.source(searchSourceBuilder);
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
过滤查询,本质上是DSL的补充语法。过滤查询在过滤的时候,是对某些搜索条件结果进行过滤,并不会进行任何匹配分数的计算。相对于query而言,filter的效率要更高一些,因为query需要计算搜索匹配相关度分数,同样的,query也更适合一些复杂条件的搜索。
ES语法:
java client实现:
- @DisplayName("过滤查询")
- @Test
- public void testFilterQuery() throws Exception{
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
- boolQueryBuilder.must(QueryBuilders.matchQuery("name","课程"));
- //过滤查询
- boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(20).lte(30));
- searchSourceBuilder.query(boolQueryBuilder);
- searchRequest.source(searchSourceBuilder);
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
控制台显示结果:
高亮查询不是逻辑结构层面的查询,不属于搜索条件,而是一种显示逻辑,它可以将关键字内容在查询结果当中进行高亮显示。
ES语法:
java client的实现:
- //修改这部分代码,增加了一个高亮显示
- @DisplayName("测试(搜索)过程结束之后,用于解析响应对象")
- @After
- public void after() throws Exception{
- //获取搜索匹配的结果
- SearchHits hits = searchResponse.getHits();
- //获取搜索的总记录(不是数)
- TotalHits totalHits = hits.getTotalHits();
- System.out.println(totalHits.toString());
- //获取匹配的文档
- SearchHit[] searchHits = hits.getHits();
- //涉及到日期对象,要做日期对象的转化
- SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
- for (SearchHit hit : searchHits) {
- //获取文档的id
- String id = hit.getId();
- //获取源文档的内容
- String sourceAsString = hit.getSourceAsString();
- System.out.println("id: "+id+" ,内容为:"+sourceAsString);
-
- //增加:高亮查询结果显示
- //获取所有高亮显示设置
- Map<String, HighlightField> highlightFields =hit.getHighlightFields();
- if (highlightFields != null){
- //获取被高亮显示的field
- HighlightField highlightField = highlightFields.get("name");
- Text[] fragments = highlightField.getFragments();
- System.out.println("高亮字段:"+ fragments[0].toString());
- }
- }
- }
-
-
- @DisplayName("highlight查询")
- @Test
- public void testHighlight() throws Exception{
- SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
- //设置查询条件
- searchSourceBuilder.query(QueryBuilders.matchQuery("name","课程"));
- //设置高亮查询(设置高亮显示)
- //获取highlight构建对象
- HighlightBuilder highlightBuilder = new HighlightBuilder();
- highlightBuilder.preTags("\"<font color='red'>\"");
- highlightBuilder.postTags("</font>");
- highlightBuilder.fields().add(new HighlightBuilder.Field("name"));
- searchSourceBuilder.highlighter(highlightBuilder);
-
- searchRequest.source(searchSourceBuilder);
- searchResponse = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
- }
控制台结果显示:
1、使用javaClient和ES进行交互的时候,最好提前在kibana当中检查一下语法的正确性,再复制到idea当中作为source之类,可以有效降低代码的错误性;
2、涉及到与索引操作相关的请求分别是:CreateIndexRequest(创建索引)、GetIndexRequest(查询索引)、DeleteIndexRequest(删除索引);
3、涉及到与文档操作相关的请求分别是:IndexRequest(获取索引请求,然后通过source方法去创建文档)、DeleteRequest(删除指定的文档)、UpdateRequest(修改文档,通过.doc方法修改)、GetRequest(根据id简单查询文档);
4、在涉及到批量操作文档的时候:注意{}不要换行,花括号当中的json格式key和value中间不要加空格,批量操作的请求是BulkRequest;
5、在使用DSL查询的时候,它是基于json的查询方式,有很多特定的json串来指定查询的规则;
请求和响应对象分别是SearchRequest和SearchResponse,查询的过程当中都需要去通过SearchSourceBuilder这个搜索源的构建者对象,来设置搜索的条件,然后再把搜索源应用在请求上去做查询;
6 、DSL当中的filter查询一般是和bool查询一起使用,它是在前几步查询的基础上,对查询结果做过滤,并没有做相关度的计算;
7、DSL的hightlight查询,虽然叫做查询,但是其实不是查询的逻辑操作,而是对搜索内容当中的关键字去做了高亮显示,它是一种显示操作,所以当涉及到设置搜索源的时候,需要单独在query之后,searchSourceBuilder通过.highligter(HighlightBuilder的对象)设置结果的高亮查询;
8 、最后,如果有问题或者想法,欢迎多多留言评论哦!
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。