赞
踩
感谢狂神老师的奉献 笔记记录 B站狂神说Java的ElasticSearch课程:
狂神老师上课视频
Elaticsearch,简称为es,es是一个开源的高扩展的分布式全文检索引擎,它可以近乎实时的存储、检索数据;本身扩展性很好,可以扩展到上百台服务器,处理PB级别(大数据时代)的数据。es也使用java开发并使用Lucene作为其核心来实现所有索引和搜索的功能,但是它的目的是通过简单的RESTful API来隐藏Lucene的复杂性,从而让全文搜索变得简单
官网Elaticsearch官网
下载地址:https://www.elastic.co/cn/downloads
历史版本下载:https://www.elastic.co/cn/downloads/past-releases/
解压即可(注意:尽量将ElasticSearch相关工具放在统一目录下)
解压目录如下:
双击图中的启动程序
浏览器输入http://localhost:9200/
注意:安装之前要安装node.js
https://github.com/mobz/elasticsearch-head
解压即可(尽量将ElasticSearch相关工具放在统一目录下)
启动成功访问http://localhost:9100/
开启跨域:http.cors.enabled: true
所有人访问:http.cors.allow-origin: “*”
对于我们初学者来说
索引 可以看做 “数据库”
类型 可以看做 “表”
文档 可以看做 “库中的数据(表中的行)
Kibana是一个针对ElasticSearch的开源分析及可视化平台,用来搜索、查看交互存储在Elasticsearch索引中的数据。使用Kibana ,可以通过各种图表进行高级数据分析及展示。Kibana让海量数据更容易理解。它操作简单,基于浏览器的用户界面可以快速创建仪表板( dashboard )实时显示Elasticsearch查询动态。设置Kibana非常简单。无需编码或者额外的基础架构,几分钟内就可以完成Kibana安装并启动Elasticsearch索引监测。
下载地址:https://www.elastic.co/cn/downloads/
编辑器打开kibana解压目录/config/kibana.yml,添加
i18n.locale: “zh-CN”
输入http://localhost:5601访问页面
之前说elasticsearch是面向文档的,那么就意味着索引和搜索数据的最小单位是文档,elasticsearch中,文档有几个重要属性:
自我包含,一篇文档同时包含字段和对应的值,也就是同时包含key:value !
可以是层次型的,一个文档中包含自文档,复杂的逻辑实体就是这么来的! {就是一个json对象 ! fastjson进行自动转换 !}
灵活的结构,文档不依赖预先定义的模式,我们知道关系型数据库中,要提前定义字段才能使用,在elasticsearch中,对于字段是非常灵活的,有时候,我们可以忽略该字段,或者动态的添加一个新的字段。
尽管我们可以随意的新增或者忽略某个字段,但是,每个字段的类型非常重要,比如一个年龄字段类型,可以是字符串也可以是整形。因为elasticsearch会保存字段和类型之间的映射及其他的设置。这种映射具体到每个映射的每种类型,这也是为什么在elasticsearch中,类型有时候也称为映射类型。
类型是文档的逻辑容器,就像关系型数据库一样,表格是行的容器。类型中对于字段的定义称为映射,比如name映射为字符串类型。我们说文档是无模式的,它们不需要拥有映射中所定义的所有字段,比如新增一个字段,那么elasticsearch是怎么做的呢?
elasticsearch会自动的将新字段加入映射,但是这个字段的不确定它是什么类型,elasticsearch就开始猜,如果这个值是18,那么elasticsearch会认为它是整形。但是elasticsearch也可能猜不对,所以最安全的方式就是提前定义好所需要的映射,这点跟关系型数据库殊途同归了,先定义好字段,然后再使用,别整什么幺蛾子。
索引是映射类型的容器, elasticsearch中的索引是一个非常大的文档集合。 索引存储了映射类型的字段和其他设置。然后它们被存储到了各个分片上了。我们来研究下分片是如何工作的。
下载地址:https://github.com/medcl/elasticsearch-analysis-ik/releases
解压即可(但是我们需要解压到ElasticSearch的plugins目录ik文件夹下)
ik是我们自己创建的
具体操作移步狂神老师的笔记观看https://www.kuangstudy.com/bbs/1354069127022583809
注意最好选择jdk1.8模式,勾上以下文件
注意springboot与elasticSearch有版本对应关系
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.5.11</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.es.jd</groupId> <artifactId>studyuse</artifactId> <version>0.0.1-SNAPSHOT</version> <name>studyuse</name> <description>Demo project for Spring Boot</description> <properties> <java.version>1.8</java.version> <repackage.classifier/> <spring-native.version>0.10.6</spring-native.version> <elasticsearch.version>7.12.0</elasticsearch.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <!-- jsoup解析页面 --> <!-- 解析网页 爬视频可 研究tiko --> <dependency> <groupId>org.jsoup</groupId> <artifactId>jsoup</artifactId> <version>1.10.2</version> </dependency> <!-- fastjson --> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.70</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.experimental</groupId> <artifactId>spring-native</artifactId> <version>${spring-native.version}</version> </dependency> <!-- devtools热部署 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <configuration> <excludes> <exclude> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </exclude> </excludes> <classifier>${repackage.classifier}</classifier> <image> <builder>paketobuildpacks/builder:tiny</builder> <env> <BP_NATIVE_IMAGE>true</BP_NATIVE_IMAGE> </env> </image> </configuration> </plugin> <plugin> <groupId>org.springframework.experimental</groupId> <artifactId>spring-aot-maven-plugin</artifactId> <version>${spring-native.version}</version> <executions> <execution> <id>test-generate</id> <goals> <goal>test-generate</goal> </goals> </execution> <execution> <id>generate</id> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> </plugins> </build> <repositories> <repository> <id>spring-releases</id> <name>Spring Releases</name> <url>https://repo.spring.io/release</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>spring-releases</id> <name>Spring Releases</name> <url>https://repo.spring.io/release</url> <snapshots> <enabled>false</enabled> </snapshots> </pluginRepository> </pluginRepositories> <profiles> <profile> <id>native</id> <properties> <repackage.classifier>exec</repackage.classifier> <native-buildtools.version>0.9.4</native-buildtools.version> </properties> <dependencies> <dependency> <groupId>org.graalvm.buildtools</groupId> <artifactId>junit-platform-native</artifactId> <version>${native-buildtools.version}</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.graalvm.buildtools</groupId> <artifactId>native-maven-plugin</artifactId> <version>${native-buildtools.version}</version> <executions> <execution> <id>test-native</id> <phase>test</phase> <goals> <goal>test</goal> </goals> </execution> <execution> <id>build-native</id> <phase>package</phase> <goals> <goal>build</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles> </project>
package com.es.jd.studyuse.util; import com.es.jd.studyuse.pojo.Content; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import org.springframework.stereotype.Component; import java.io.IOException; import java.net.URL; import java.util.ArrayList; import java.util.List; @Component public class HtmlParseUtil { public List<Content> parseJD(String keywords) throws IOException { //获取请求 https://search.jd.com/Search?keyword=java //前提需要联网 String url=" https://search.jd.com/Search?keyword="+keywords; //解析网页(Jsoup返回Doucument就是js 页面对象) Document document = Jsoup.parse(new URL(url), 30000); Element element = document.getElementById("J_goodsList"); System.out.println(element.html()); //获取所有的li元素 Elements elements = element.getElementsByTag("li"); List<Content> goodsList = new ArrayList<>(); //获取元素中的内容 for (Element el : elements) { String img = el.getElementsByTag("img").eq(0).attr("data-lazy-img"); String price = el.getElementsByClass("p-price").eq(0).text(); String title = el.getElementsByClass("p-name").eq(0).text(); goodsList.add(new Content(title,img,price)); } return goodsList; } }
package com.es.jd.studyuse.util; import com.es.jd.studyuse.pojo.Content; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import org.springframework.stereotype.Component; import java.io.IOException; import java.net.URL; import java.util.ArrayList; import java.util.List; @Component public class HtmlParseUtil { public List<Content> parseJD(String keywords) throws IOException { //获取请求 https://search.jd.com/Search?keyword=java //前提需要联网 String url=" https://search.jd.com/Search?keyword="+keywords; //解析网页(Jsoup返回Doucument就是js 页面对象) Document document = Jsoup.parse(new URL(url), 30000); Element element = document.getElementById("J_goodsList"); System.out.println(element.html()); //获取所有的li元素 Elements elements = element.getElementsByTag("li"); List<Content> goodsList = new ArrayList<>(); //获取元素中的内容 for (Element el : elements) { String img = el.getElementsByTag("img").eq(0).attr("data-lazy-img"); String price = el.getElementsByClass("p-price").eq(0).text(); String title = el.getElementsByClass("p-name").eq(0).text(); goodsList.add(new Content(title,img,price)); } return goodsList; } }
package com.es.jd.studyuse.cotroller; import com.es.jd.studyuse.service.ContentService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; import java.io.IOException; import java.util.List; import java.util.Map; //请求编写 @RestController public class ContentController { @Autowired private ContentService contentService; @GetMapping("/pase/{keyword}") public Boolean parse(@PathVariable("keyword") String keyword) throws IOException { return contentService.parseContent(keyword); } @GetMapping("/search/{keyword}/{pageNo}/{pageSize}") public List<Map<String,Object>> search(@PathVariable("keyword") String keyword, @PathVariable("pageNo") int pageNo, @PathVariable("pageSize") int pageSize) throws IOException { return contentService.searchPage(keyword,pageNo,pageSize); } //3.获取这些数据是实现高亮功能· @GetMapping("/searchHighLight/{keyword}/{pageNo}/{pageSize}") public List<Map<String,Object>> searchHighLight(@PathVariable("keyword") String keyword, @PathVariable("pageNo") int pageNo, @PathVariable("pageSize") int pageSize) throws IOException { return contentService.searchHighLight(keyword,pageNo,pageSize); } }
package com.es.jd.studyuse.cotroller;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
@Controller
public class IndexContreoller {
@GetMapping({"/","/index"})
public String index(){
return "index";
}
}
package com.es.jd.studyuse.pojo; import lombok.Data; @Data public class Content { private String title; private String img; private String price; public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public String getImg() { return img; } public void setImg(String img) { this.img = img; } public String getPrice() { return price; } public void setPrice(String price) { this.price = price; } public Content() { } public Content(String title, String img, String price) { this.title = title; this.img = img; this.price = price; } @Override public String toString() { return "Content{" + "title='" + title + '\'' + ", img='" + img + '\'' + ", price='" + price + '\'' + '}'; } }
package com.es.jd.studyuse.service; import com.alibaba.fastjson.JSON; import com.es.jd.studyuse.pojo.Content; import com.es.jd.studyuse.util.HtmlParseUtil; import org.elasticsearch.action.bulk.BulkRequest; import org.elasticsearch.action.bulk.BulkResponse; import org.elasticsearch.action.index.IndexRequest; import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.common.text.Text; import org.elasticsearch.common.unit.TimeValue; import org.elasticsearch.common.xcontent.XContentType; import org.elasticsearch.index.query.MatchPhraseQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.search.SearchHit; import org.elasticsearch.search.SearchHits; import org.elasticsearch.search.builder.SearchSourceBuilder; import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; import org.elasticsearch.search.fetch.subphase.highlight.HighlightField; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import java.io.IOException; import java.util.ArrayList; import java.util.List; import java.util.Map; import java.util.concurrent.TimeUnit; //业务编写 @Service public class ContentService { @Autowired private RestHighLevelClient restHighLevelClient; //1.解析数据放入es索引中 public Boolean parseContent(String keywords) throws IOException { List<Content> contents = new HtmlParseUtil().parseJD(keywords); //把查询到的数据放入索引库中 BulkRequest bulkRequest = new BulkRequest(); bulkRequest.timeout("2m"); for (Content content : contents) { bulkRequest.add( // 这里是数据信息 new IndexRequest("jd") .source(JSON.toJSONString(content),XContentType.JSON) ); } BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT); return !bulk.hasFailures(); } //获取这些数据实现搜索功能 public List<Map<String,Object>> searchPage(String keyword,int pageNo,int pageSize) throws IOException { if (pageNo<=1){ pageNo = 1; } //创建搜索库请求 SearchRequest searchRequest = new SearchRequest("jd"); //创建添加条件对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //给条件对象赋值 searchSourceBuilder.from(pageNo); searchSourceBuilder.size(pageSize); //创建精确查找对象 MatchPhraseQueryBuilder matchPhraseQueryBuilder = QueryBuilders.matchPhraseQuery("title", keyword); //放入查找对象中 searchSourceBuilder.query(matchPhraseQueryBuilder); //设置反应时间 searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); //执行搜索 将条件对象放入请求对象中 searchRequest.source(searchSourceBuilder); //客户端执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //解析结果 List<Map<String,Object>> list = new ArrayList<>(); SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { list.add(hit.getSourceAsMap()); } return list; } //获取这些数据实现高亮功能 public List<Map<String,Object>> searchHighLight(String keyword,int pageNo,int pageSize) throws IOException { if (pageNo<=1){ pageNo = 1; } //创建搜索库请求 SearchRequest searchRequest = new SearchRequest("jd"); //创建添加条件对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //给条件对象赋值 searchSourceBuilder.from(pageNo); searchSourceBuilder.size(pageSize); //创建精确查找对象 MatchPhraseQueryBuilder matchPhraseQueryBuilder = QueryBuilders.matchPhraseQuery("title", keyword); //放入查找对象中 searchSourceBuilder.query(matchPhraseQueryBuilder); //设置反应时间 searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); //设置高亮 HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.field("title"); highlightBuilder.requireFieldMatch(false);//多个高亮显示! highlightBuilder.preTags("<span style='color:red'>"); highlightBuilder.postTags("</span>"); //放入高亮对象 searchSourceBuilder.highlighter(highlightBuilder); //执行搜索 将条件对象放入请求对象中 searchRequest.source(searchSourceBuilder); //客户端执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); //解析结果 List<Map<String,Object>> list = new ArrayList<>(); SearchHits hits = searchResponse.getHits(); for (SearchHit hit : hits) { //获得高亮显示 Map<String, HighlightField> highlightFields = hit.getHighlightFields(); HighlightField title = highlightFields.get("title"); Map<String, Object> sourceAsMap = hit.getSourceAsMap();//原来的结果! //解析高亮的字段 将原来的字段换为我们高亮的字段 if (title!=null){ Text[] fragments = title.fragments(); String newTitle = ""; //将高亮字段替换到原来的字段 for (Text text : fragments) { newTitle += text; } sourceAsMap.put("title",newTitle);//高亮字段替换掉原来的内容即可 } list.add(sourceAsMap); } return list; } }
<!DOCTYPE html> <html xmlns:th="http://www.thymeleaf.org"> <head> <meta charset="utf-8"/> <title>小白Java-ES仿京东实战</title> <link rel="stylesheet" th:href="@{/css/style.css}"/> </head> <body class="pg"> <div class="page" id="app"> <div id="mallPage" class=" mallist tmall- page-not-market "> <!-- 头部搜索 --> <div id="header" class=" header-list-app"> <div class="headerLayout"> <div class="headerCon "> <!-- Logo--> <h1 id="mallLogo"> <img th:src="@{/images/jdlogo.png}" alt=""> </h1> <div class="header-extra"> <!--搜索--> <div id="mallSearch" class="mall-search"> <form name="searchTop" class="mallSearch-form clearfix"> <fieldset> <legend>小白搜索</legend> <div class="mallSearch-input clearfix"> <div class="s-combobox" id="s-combobox-685"> <div class="s-combobox-input-wrap"> <input v-model="keyword" type="text" autocomplete="off" value="dd" id="mq" class="s-combobox-input" aria-haspopup="true"> </div> </div> <button type="submit" @click.prevent="searchKey" id="searchbtn">小松搜索</button> </div> </fieldset> </form> <ul class="relKeyTop"> <li><a>小白学Java</a></li> <li><a>小白学前端</a></li> <li><a>小白学Linux</a></li> <li><a>小白学大数据</a></li> <li><a>小白学理财</a></li> </ul> </div> </div> </div> </div> </div> <!-- 商品详情页面 --> <div id="content"> <div class="main"> <!-- 品牌分类 --> <form class="navAttrsForm"> <div class="attrs j_NavAttrs" style="display:block"> <div class="brandAttr j_nav_brand"> <div class="j_Brand attr"> <div class="attrKey"> 品牌 </div> <div class="attrValues"> <ul class="av-collapse row-2"> <li><a href="#"> 小白</a></li> <li><a href="#"> Java </a></li> </ul> </div> </div> </div> </div> </form> <!-- 排序规则 --> <div class="filter clearfix"> <a class="fSort fSort-cur">综合<i class="f-ico-arrow-d"></i></a> <a class="fSort">人气<i class="f-ico-arrow-d"></i></a> <a class="fSort">新品<i class="f-ico-arrow-d"></i></a> <a class="fSort">销量<i class="f-ico-arrow-d"></i></a> <a class="fSort">价格<i class="f-ico-triangle-mt"></i><i class="f-ico-triangle-mb"></i></a> </div> <!-- 商品详情 --> <div class="view grid-nosku"> <div class="product" v-for="result in results"> <div class="product-iWrap"> <!--商品封面--> <div class="productImg-wrap"> <a class="productImg"> <img :src="result.img"> </a> </div> <!--价格--> <p class="productPrice"> <em>{{result.price}}</em> </p> <!--标题--> <p class="productTitle"> <a v-html="result.title"></a> </p> <!-- 店铺名 --> <div class="productShop"> <span>店铺: 小白学Java </span> </div> <!-- 成交信息 --> <p class="productStatus"> <span>月成交<em>999笔</em></span> <span>评价 <a>3</a></span> </p> </div> </div> </div> </div> </div> </div> </div> <!--前端使用Vue,实现前后端分离--> <script th:src="@{/js/axios.min.js}"></script> <script th:src="@{/js/vue.min.js}"></script> <script> new Vue({ el: '#app', data: { keyword: '',//搜索的关键字 results: []//搜索结果 }, methods: { searchKey() { var keyword = this.keyword; console.log(keyword); //对接后端的接口 axios.get('searchHighLight/'+keyword+"/1/10").then(response=>{ console.log(response.data); this.results = response.data;//绑定数据绑定成功 }) } } }) </script> </body> </html>
链接:https://pan.baidu.com/s/1TaPzubDuR19FhgA6XiY8WQ
提取码:xsgo
`更多相关学习请看狂神学习笔记
狂神学习笔记
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。