赞
踩
官网:https://lucene.apache.org/
Lucene是apache下的一个开源的全文检索引擎工具包。它为软件开发人员提供一个简单易用的工具包(类库),以方便的在目标系统中实现全文检索的功能
1.比如百度和一些电商一些带搜索并且数据量大的网站,在搜素时会需要去搜索很多条数据,效率会大大降低。将要搜索的数据加入Lucene索引库,Lucene可以很多搜索算法大大节省搜索的效率
2.在搜索的时候,通常会有很多关键字,数据库只能通过like %搜索内容%的方式来查询,会过滤掉很多有用的数据。使用Lucene,可以使用分词器将搜索的内容拆分成多个词语,可以查询到更多的数据
3.Lucene不仅可以加入数据库数据的索引,还能加入文档,web网页等各种各样的方式
一篇文章一行一行的去查,去扫描。数据库中一张表,一个个数据去查。常用在文本编辑器的查找功能和数据库的查询
优点:
查询准确率高
缺点:
数据越多,查询越慢
一个字典通过,搜先去查看目录,找到需要的那个目录,然后通过目录找到正文部分。会将搜索条件切分词,去除重复部分,大写字母转小写,去除停用词(a,an,the,的,得,地)再将一句话拆分成一个个词,然后去创建目录 ,在数据量大查询的时候使用
优点:
查询准确率高
查询效率快
缺点:
索引文件额外占用磁盘空间
会有一个文档集合,文档集合里面是一个个文档
每个文档是通过一个key:value的方式存储的
key | value |
---|---|
id | 1 |
name | 联想thinkbook 14 Pro |
price | 6000 |
例如上面那个文档会生成关键字:
1,联想,think,book,thinkbook,14,pro,6000等
这些关键字会指向这个文档的DocumentId(例如 111)
然后用户搜索其中的关键字,会找拿到对应的DocumentId
将我们文档(也就是数据)显示出来
查询的时候会创建查询
name:联想 查到有联想关键字的所有文档 以 111(文档Id)->222这样子的方式排序
name:联想 AND thinkbook 查到有联想关键字和thinkbook关键字的所有文档
elasticsearch是一个基于Lucene的搜索服务器,对Lucene进行了resful的封装,操作方便简单,搜索效率快
elasticsearch解压既可用,安装迅速,可扩展,兼容了很多插件
https://www.elastic.co/cn/products/elasticsearch
https://github.com/elastic/elasticsearch
在根目录下创建一个 data 文件夹装索引
config 文件下 配置 elasticsearch.yml
#集群名称 cluster.name: server #节点名称 node.name: ch-node-1 #当前节点所绑定的本机IP network.host: 127.0.0.1 #设置对外服务的http端口,默认为 9200 http.port: 9200 #集群结点之间通信端口 transport.tcp.port: 9300 #指定该节点是否有资格被选举成为master结点,默认是true,如果原来的master宕机会重新选举新的master node.master: true #指定该节点是否存储索引数据,默认为true node.data: true #设置集群中master节点的初始列表 #discovery.zen.ping.unicast.hosts: ["0.0.0.0:9300", "0.0.0.0:9301","0.0.0.0:9302"] #主结点数量的最少值 discovery.zen.minimum_master_nodes: 1 #是否允许成为协调节点 node.ingest: true #设置为true可以锁住ES使用的内存,避免内存与swap分区交换数据 bootstrap.memory_lock: false #单机允许的最大存储结点数,通常单机启动一个结点建议设置为1,开发环境如果单机启动多个节点可设置大于1 node.max_local_storage_nodes: 2 #path.data是根目录的data文件路径 #path.logs是根目录的logs文件的路径 path.data: D:\a-elasticsearch\elasticsearch01\data path.logs: D:\a-elasticsearch\elasticsearch01\logs #跨域的设置 http.cors.enabled: true http.cors.allow-origin: /.*/
注意 : 做集群需要改端口
head插件是ES的一个可视化管理插件,用来监视ES的状态,并通过head客户端和ES服务进行交互,比
如创建映射、创建索引等,head的项目地址在https://github.com/mobz/elasticsearch-head
首先准备 pom 文件的内容,将es和es客户端的jar导入
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.13.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.13.0</version>
</dependency>
在 application.yml 文件中配置ip地址
elasticsearch:
hostlist: ${eshostlist:127.0.0.1:9200} #多个结点中间用逗号分隔
在项目中创建一个 config 包,里面创建es的配置类
@Configuration public class ElasticsearchConfig { @Value("${chuhang.elasticsearch.hostlist}") private String hostlist; @Bean public RestHighLevelClient restHighLevelClient(){ //解析hostlist配置信息 String[] split = hostlist.split(","); //创建HttpHost数组,其中存放es主机和端口的配置信息 HttpHost[] httpHostArray = new HttpHost[split.length]; for(int i=0;i<split.length;i++){ String item = split[i]; httpHostArray[i] = new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http"); } //创建RestHighLevelClient客户端 return new RestHighLevelClient(RestClient.builder(httpHostArray)); } //低级客户端,不常用 @Bean public RestClient restClient(){ //解析hostlist配置信息 String[] split = hostlist.split(","); //创建HttpHost数组,其中存放es主机和端口的配置信息 HttpHost[] httpHostArray = new HttpHost[split.length]; for(int i=0;i<split.length;i++){ String item = split[i]; httpHostArray[i] = new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http"); } return RestClient.builder(httpHostArray).build(); } }
然后就可以创建 test 包,写代码了
@Autowired private RestHighLevelClient restHighLevelClient; /** * 创建索引 */ @Test public void CreateIndex() throws IOException { //1.创建新建索引请求对象 CreateIndexRequest indexRequest = new CreateIndexRequest("course"); //2.通过CreateIndexRequest设置索引参数 indexRequest.settings(Settings.builder().put("number_of_shards",1));//分片 indexRequest.settings(Settings.builder().put("number_of_replicas", 0));//副本 //3.创建映射 indexRequest.mapping("{\n" + " \"properties\": {\n" + " \"description\": {\n" + " \"type\": \"text\",\n" + " \"analyzer\": \"ik_max_word\",\n" + " \"search_analyzer\": \"ik_smart\"\n" + " },\n" + " \"name\": {\n" + " \"type\": \"text\",\n" + " \"analyzer\": \"ik_max_word\",\n" + " \"search_analyzer\": \"ik_smart\"\n" + " },\n" + " \"pic\": {\n" + " \"type\": \"text\",\n" + " \"index\": false\n" + " },\n" + " \"price\": {\n" + " \"type\": \"float\"\n" + " },\n" + " \"studymodel\": {\n" + " \"type\": \"keyword\"\n" + " },\n" + " \"timestamp\": {\n" + " \"type\": \"date\",\n" + " \"format\": \"yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis\"\n" + " }\n" + " }\n" + "}", XContentType.JSON); //4.通过ES客户端RestHighLevelClient对象创建索引操作客户端IndicesClient IndicesClient indicesClient = restHighLevelClient.indices(); //5.通过IndicesClient对象执行新建索引请求,并且获得响应 CreateIndexResponse indexResponse = indicesClient.create(indexRequest, RequestOptions.DEFAULT); //6.查看是否创建成功 boolean acknowledged = indexResponse.isAcknowledged(); System.out.println("结果:"+acknowledged); }
/** * 添加文档 */ @Test public void testAddDoc() throws IOException { //1.准备json数据 Map<String, Object> jsonMap = new HashMap<>(); jsonMap.put("name", "spring cloud实战"); jsonMap.put("description", "本课程主要从四个章节进行讲解: 1.微服务架构入门 2.spring cloud基础入门 3.实战Spring Boot 4.注册中eureka。"); jsonMap.put("studymodel", "201001"); SimpleDateFormat dateFormat =new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); jsonMap.put("timestamp", dateFormat.format(new Date())); jsonMap.put("price", 5.6f); jsonMap.put("pic","group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg"); //2.索引请求对象 IndexRequest indexRequest = new IndexRequest("course"); //3.将数据放进请求中 indexRequest.source(jsonMap); //4.执行请求,并且获得响应 IndexResponse indexResponse = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT); //5.查看结果 DocWriteResponse.Result result = indexResponse.getResult(); System.out.println(result); }
/** * 查询文档:根据id查 */ @Test public void getDoc() throws IOException { //1.创建查询请求对象 GetRequest getRequest = new GetRequest("course","6EBZQYEB_RHHuuGGK2AP"); //2.得到响应的结果对象 GetResponse getResponse = restHighLevelClient.get(getRequest,RequestOptions.DEFAULT); boolean exists = getResponse.isExists(); //3.取值 Map<String, Object> sourceAsMap = getResponse.getSourceAsMap(); System.out.println("结果:"+sourceAsMap); }
/** * 更新文档 */ @Test public void updateDoc() throws IOException { //1.创建更新的请求对象 UpdateRequest updateRequest = new UpdateRequest(); //新 updateRequest.index("course").id("6EBZQYEB_RHHuuGGK2AP"); //2.将要更新的数据添加到更新的请求对象 updateRequest.doc("name","Spring Cloud实战"); //3.得到更新的响应结果 UpdateResponse update = restHighLevelClient.update(updateRequest,RequestOptions.DEFAULT); RestStatus status = update.status(); System.out.println(status); }
/**
* 根据文档id删除文档
*/
@Test
public void testDelDoc() throws IOException {
//1.删除索引请求对象
DeleteRequest deleteRequest = new DeleteRequest();
deleteRequest.index("course").id("6EBZQYEB_RHHuuGGK2AP");
//2.响应对象
DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest,
RequestOptions.DEFAULT);
//3.获取响应结果
DocWriteResponse.Result result = deleteResponse.getResult();
System.out.println(result);
}
/** * 查询所有的记录 * @throws IOException */ @Test public void testSearchAll() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 查询所有 searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
/** * 搜索分页查询 * @throws IOException */ @Test public void testSearchPage() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 查询所有 searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //设置分页参数 //页码 int page=1; //每页记录数 int size = 1; //计算出记录其实下标 int from = (page-1)*size; searchSourceBuilder.from(from); searchSourceBuilder.size(size); //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
/** * TermQuery查询 * @throws IOException */ @Test public void testTermQuery() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 设置搜索方式: termQuery 方式 searchSourceBuilder.query(QueryBuilders.termQuery("name","spring"));//只有换成"spring"才能找到数据 //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
/** * termQuery查询:精确匹配id * @throws IOException */ @Test public void testTermQueryByIds() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); String[] ids = new String[]{"2","3"}; //2.1 指定搜索的方式:termQuery搜索 searchSourceBuilder.query(QueryBuilders.termsQuery("_id",ids)); //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
/** * MatchQuery查询 * @throws IOException */ @Test public void testMatchQuery() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 指定搜索的方式:MatchQuery // searchSourceBuilder.query(QueryBuilders.matchQuery("description","spring开发").operator(Operator.OR)); searchSourceBuilder.query(QueryBuilders.matchQuery("description", "前台页面开发框架架构").minimumShouldMatch("80%")); //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
/** * MultihMatchQuery查询 * @throws IOException */ @Test public void testMultiQuery() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 指定搜索的方式:MatchQuery searchSourceBuilder.query(QueryBuilders.multiMatchQuery("spring css","name","description").minimumShouldMatch("50%").field("name",10)); //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
@Test public void testBoolQuery() throws IOException { //1.创建search请求对象,并且指定搜索的索引库名称 SearchRequest searchRequest = new SearchRequest("course"); //2.搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //2.1 指定搜索的方式:MatchQuery MultiMatchQueryBuilder mq = QueryBuilders.multiMatchQuery("spring css", "name", "description").minimumShouldMatch("50%").field("name", 10); //2.1.2 再定义一个termQuery TermQueryBuilder tq = QueryBuilders.termQuery("studymodel", "201001"); //2.1.3 最后定义一个booleanQuery BoolQueryBuilder bq = QueryBuilders.boolQuery(); bq.must(mq); // bq.must(tq); bq.filter(QueryBuilders.termQuery("studymodel","201001")); bq.filter(QueryBuilders.rangeQuery("price").gte(60).lte(100)); //2.1 设置搜索方式 searchSourceBuilder.query(bq); searchSourceBuilder.sort("studymodel", SortOrder.DESC);//降序 searchSourceBuilder.sort("price", SortOrder.ASC);//升序 //2.2 过滤字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{}); //高亮设置 HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.preTags("<tag>");//设置前缀 highlightBuilder.postTags("</tag>");//设置后缀 highlightBuilder.fields().add(new HighlightBuilder.Field("name")); searchSourceBuilder.highlighter(highlightBuilder); //3.向搜索请求对象中设置搜索源 searchRequest.source(searchSourceBuilder); //4.执行搜索请求,获取响应 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //5.搜索的结果 SearchHits hits = searchResponse.getHits(); //5.1 匹配到的总记录数 TotalHits totalHits = hits.getTotalHits(); //5.2 匹配到的文档 SearchHit[] searchHits = hits.getHits(); //5.3 循环取值 for (SearchHit hit : searchHits) { //注意:不能获取被过滤掉的字段信息! String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId();//文档的主键(主键不属于文档内的内容字段) float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); //取出高亮字段内容 Map<String, HighlightField> highlightFields = hit.getHighlightFields(); if(highlightFields!=null){ HighlightField nameField = highlightFields.get("name"); if(nameField!=null){ Text[] fragments = nameField.getFragments(); StringBuffer stringBuffer = new StringBuffer(); for (Text str : fragments) { stringBuffer.append(str.string()); } name = stringBuffer.toString(); } } String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。