赞
踩
ElasticSearch在Java中使用,在此感谢狂神老师,狂神大神ES课程
提示:以下是本篇文章正文内容,下面案例可供参考
关于ElasticSearch和Kibana的安装在其他文章中有写,接下来是针对head可视化界面的安装
ElasticSearch-Head下载
进入安装的文件夹下在cmd指令下使用cnpm install
使用npm run start启动
cnpm install
npm run start
进入localhost:9100发现:与9200产生了跨域问题
在ElasticSearch下配置跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
再次连接,成功
可以把es当作一个数据库(索引),在索引中加入文档(数据库中的数据)
查询在Kibana中看,这个可视化页面当作数据库中查看数据就行
在这里安装选用的7.17.3版本,下载即用,方便快捷
参考:Kibana与ElasticSearch7.17.3的安装与使用
所有的ElasticSearch是在Restful风格下进行操作的
es_db索引为其他课程所创建,可以不看(自己测试用)
PUT /test3/_doc/1
{
"name":"llyyff",
"age":123,
"birthday":"2000-02-03"
}
GET /test3
如果自己的文档字段没有指定,那么es就会给我们配置默认字段类型
#添加数据与获取数据 PUT /es_learn/user/1 { "name":"es学习", "age":23, "desc":"又菜又爱玩", "tags":["吃饭","睡觉","打球"] } PUT /es_learn/user/2 { "name":"小红", "age":21, "desc":"又玩又菜", "tags":["学习","唱歌","睡觉"] } PUT /es_learn/user/3 { "name":"小刚", "age":18, "desc":"大佬", "tags":["学习","看视频","打球"] } GET /es_learn/_search
#不建议使用PUT PUT /es_learn/user/3 { "desc":"Java大牛,24岁博士后" } #不建议使用POST POST /es_learn/user/1 { "doc":{ "desc":"争取做24岁Java大牛" } } #建议使用POST _update POST /es_learn/user/1/_update { "doc":{ "desc":"争取去阿里腾讯" } }
使用PUT与POST会发现数据被覆盖,之前就算不被修改的数据也会消失
使用POST_update发现仅仅修改需要修改的数据,不影响原来的数据
#查询数据 精确匹配(必须为数据准确才能匹配到,未来使用的是只要包含有其中的字就查询
GET /es_learn/user/_search?q=name:小红
排序、分页、高亮、模糊查询、精准查询
我们之后使用java操作es,所有的方法和对象就是这里面的key
数据下标还是从0开始的,和学的所有数据结构是一样的
/search/{current}/{pageSize}
must(and),所有的条件都要符合 where id=1 and name=xxx
should(or),两个条件满足其一
must_not,不满足条件的都会出来
过滤器filter
gt 大于
gre 大于等于
lt 小于
lte 小于等于
term查询是直接通过倒排索引指定的词条进程精确的查找
关于分词:
term,会直接查询精确的
match,会使用分词器解析!(先分析文档,然后再通过分析的文档进行查询!)
正常的是可以被拆分的
keyword类型,字段不会被拆分,查什么是什么
这些Mysql也可以做,但是Mysql的效率低
匹配
按照条件匹配
精确匹配
区间范围匹配
匹配字段过滤
多条件查询
高亮查询
<properties> <java.version>1.8</java.version> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <spring-boot.version>2.3.7.RELEASE</spring-boot.version> <!--自定义es版本,保证和本地一致--> <elascticsearch.version>7.17.3</elascticsearch.version> </properties> <dependencies> <!--jsoup解析网页--> <dependency> <groupId>org.jsoup</groupId> <artifactId>jsoup</artifactId> <version>1.14.3</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>2.0.7</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> <exclusions> <exclusion> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> </exclusion> </exclusions> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-dependencies</artifactId> <version>${spring-boot.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.8.1</version> <configuration> <source>1.8</source> <target>1.8</target> <encoding>UTF-8</encoding> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <version>2.3.7.RELEASE</version> <configuration> <mainClass>com.lyf.ElasticsearchLearnApplication</mainClass> </configuration> <executions> <execution> <id>repackage</id> <goals> <goal>repackage</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
//spring的两步骤: //1.找对象 //2.放到spring中待用 //3.如果是SpringBoot 就先分析源码 //xxxxAutoConfiguration xxxProperties //集成ElasticSearch //如果是集群构建多个,不是集群构建一个 public class ElasticSearchConfig { @Configuration public class ElasticSearchClientConfig { @Bean public RestHighLevelClient restHighLevelClient(){ RestHighLevelClient client=new RestHighLevelClient( RestClient.builder( new HttpHost("127.0.0.1",9200,"http"))); return client; } } }
源码中提供的对象
虽然这里导入3个类,但都是静态内部类,核心类就一个
在测试类中编写测试
@SpringBootTest class ElasticsearchLearnApplicationTests { @Autowired @Qualifier("restHighLevelClient") private RestHighLevelClient client; //测试索引的创建 Request @Test void testCreateIndex() throws IOException { //1.创建索引请求 CreateIndexRequest request=new CreateIndexRequest("lyf_index"); //2.客户端执行创建请求 IndicesClient ,请求后获得相应 CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT); System.out.println(createIndexResponse); } }
返回结果
创建成功
//测试获取索引
@Test
void testExisIndex() throws IOException {
GetIndexRequest request=new GetIndexRequest("lyf_index");
//判断该索引是否存在
boolean exists=client.indices().exists(request,RequestOptions.DEFAULT);
System.out.println(exists);
}
//测试删除索引
@Test
void testDeleteIndex() throws IOException{
DeleteIndexRequest request=new DeleteIndexRequest("lyf_index");
//删除
AcknowledgedResponse delete=client.indices().delete(request,RequestOptions.DEFAULT);
System.out.println(delete);
}
//测试添加文档 @Test void testAddDocument() throws IOException { //创建对象 User user=new User("lyf",3); //创建请求 IndexRequest request=new IndexRequest("lyf_index"); //规则 put/lyf_index/_doc/1 request.id("1"); request.timeout(TimeValue.timeValueSeconds(1)); request.timeout("1s"); //将我们的数据放入请求 json request.source(JSON.toJSONString(user), XContentType.JSON); //客户端发送请求 IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT); System.out.println(indexResponse.toString()); System.out.println(indexResponse.status()); }
//获取文档,判断是否存在 get /index/doc/1 @Test void testIsExists() throws IOException { GetRequest getRequest=new GetRequest("lyf_index","1"); //不获取返回的_source的上下文了 getRequest.fetchSourceContext(new FetchSourceContext(false)); getRequest.storedFields("_none_"); boolean exists = client.exists(getRequest, RequestOptions.DEFAULT); System.out.println(exists); } //获得文档的信息 @Test void testGetDocument() throws IOException{ GetRequest getRequest=new GetRequest("lyf_index","1"); //不获取返回的_source的上下文了 GetResponse getResponse=client.get(getRequest,RequestOptions.DEFAULT); System.out.println(getResponse.getSourceAsString()); //打印文档的内容 System.out.println(getResponse); //返回的全部与命令是一样的 }
//更新文档的信息
@Test
void testUpdateDocument() throws IOException{
UpdateRequest updateRequest=new UpdateRequest("lyf_index","1");
updateRequest.timeout("1s");
User user=new User("lyf学ES",18);
updateRequest.doc(JSON.toJSONString(user), XContentType.JSON);
UpdateResponse updateResponse=client.update(updateRequest,RequestOptions.DEFAULT);
UpdateResponse updateresponse = client.update(updateRequest, RequestOptions.DEFAULT);
System.out.println(updateresponse);
}
//删除文档的信息
@Test
void testDeleteRequest() throws IOException{
DeleteRequest request=new DeleteRequest("lyf_index","1");
request.timeout("1s");
DeleteResponse deleteResponse=client.delete(request, RequestOptions.DEFAULT);
System.out.println(deleteResponse.status());
}
删除成功
//特殊的,真的项目一般都会批量插入数据 @Test void testBulkRequest() throws IOException { BulkRequest bulkRequest = new BulkRequest(); bulkRequest.timeout("10s"); ArrayList<User> userArrayList = new ArrayList<>(); userArrayList.add(new User("lyf1", 3)); userArrayList.add(new User("lyf2", 4)); userArrayList.add(new User("lyf3", 5)); userArrayList.add(new User("lyf4", 6)); userArrayList.add(new User("lyf5", 7)); userArrayList.add(new User("lyf6", 8)); for (int i = 0; i < userArrayList.size(); i++) { bulkRequest.add( new IndexRequest("lyf_index") .id("" + (i + 1)) //不写id生成随机id .source(JSON.toJSONString(userArrayList.get(i)), XContentType.JSON)); } BulkResponse bulkResponse=client.bulk(bulkRequest,RequestOptions.DEFAULT); bulkResponse.hasFailures(); //是否失败,如果是false返回成功 }
//查询 //SearchRequest 搜索请求 //SearchSourceBuilder 条件构造 //HighlightBuilder 构建高亮 //TermQueryBuilder 精确查询 //MatchAllQueryBuilder 查询所有 //xxx QueryBuilder 对应我们刚才看到的所有命令 @Test void testSearch() throws IOException { SearchRequest searchRequest = new SearchRequest("lyf_index"); //构建查询条件 SearchSourceBuilder sourceBuilder=new SearchSourceBuilder(); //构建高亮 sourceBuilder.highlighter(); //查询条件 我们可以使用QueryBuilder进行快速匹配 //QueryBuilders.matchAllQuery(); 匹配所有 //QueryBuilders.termQuery("name","lyf"); 精确查询 //TermQueryBuilder termQueryBuilder= QueryBuilders.termQuery("name","lyf1"); MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery(); sourceBuilder.query(matchAllQueryBuilder); //分页 // sourceBuilder.from(); // sourceBuilder.size(); sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); searchRequest.source(sourceBuilder); SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT); //数据都封装在hit当中,所以从hit中获取数据 System.out.println(JSON.toJSONString(searchResponse.getHits())); System.out.println("========================================"); for (SearchHit hit : searchResponse.getHits().getHits()) { System.out.println(hit.getSourceAsMap()); } }
<properties> <java.version>1.8</java.version> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <spring-boot.version>2.3.7.RELEASE</spring-boot.version> <!--自定义es版本依赖,与本地一致--> <elasticsearch.version>7.17.3</elasticsearch.version> </properties> <dependencies> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>2.0.7</version> </dependency> <!--导入elasticsearch--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> <exclusions> <exclusion> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> </exclusion> </exclusions> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-dependencies</artifactId> <version>${spring-boot.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.8.1</version> <configuration> <source>1.8</source> <target>1.8</target> <encoding>UTF-8</encoding> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <version>2.3.7.RELEASE</version> <configuration> <mainClass>es.lyf.JingdongLearnApplication</mainClass> </configuration> <executions> <execution> <id>repackage</id> <goals> <goal>repackage</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
创建Controller返回index页面(页面请关注狂神老师课程,课程在上面)
@Controller
public class pageController {
@RequestMapping("/index")
public String goIndex(){
return "index";
}
}
数据问题?数据库获取,消息队列中获取中,都可以成为数据源,爬虫!
爬取数据:(获取请求返回的页面信息,筛选出想要的数据就剋了)
jsoup包
1.导入依赖
<!--jsoup解析网页-->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.14.3</version>
</dependency>
2.编写HtmlParstUtil,网页解析工具类(必须联网)
public class HtmlParseUtil { public static void main(String[] args) throws IOException { new HtmlParseUtil().parseJD("java").forEach(System.out::println); } public List<Content> parseJD(String keywords) throws IOException{ //获得请求 https://search.jd.com/Search?keyword=java //前提,需要联网! String url="https://search.jd.com/Search?keyword=java"; //解析网页 超过30秒就会报错 Document document = Jsoup.parse(new URL(url),30000); // 所有你在js中可以使用的方法,这里都能用 //获取前端网页下id为J_goodsList下的数据 Element element=document.getElementById("J_goodsList"); //获取到的前端id为J_goodsList下的数据 //System.out.println(element.html()); //获取所有的li元素 Elements elements = element.getElementsByTag("li"); //获取元素中的内容 这里的el就是每一个li标签了 ArrayList<Content> goodsList=new ArrayList<>(); for (Element el : elements) { //关于这种图片特别多的网站,所有图片都是懒加载的,为了提高性能。先显示网页再加载图片 //data-lazy-img //取img属性下的src不可以,要取source-data-lazy-img String img=el.getElementsByTag("img").eq(0).attr("data-lazy-img"); //获取到li标签下第一个img属性下的src中的数据 String price=el.getElementsByClass("p-price").eq(0).text(); //获取类名为"p-price"的价格并且变为文档 String title=el.getElementsByClass("p-name").eq(0).text(); System.out.println(img); System.out.println(price); System.out.println(title); System.out.println("-------------------------------------------------------------"); Content content=new Content(title,img,price); goodsList.add(content); } return goodsList; } }
ContentController
@RestController public class ContentController { @Autowired private ContentService contentService; //解析,获取指定的数据 @GetMapping("/parse/{keyword}") public Boolean parse(@PathVariable("keyword")String keyword)throws Exception{ return contentService.parseContent(keyword); } @RequestMapping("/search/{keyword}/{pageNo}/{pageSize}") public List<Map<String,Object>> search(@PathVariable("keyword") String keyword, @PathVariable("pageNo") int pageNo, @PathVariable("pageSize") int pageSize) throws IOException { System.out.println(keyword); System.out.println(pageNo); System.out.println(pageSize); return contentService.searchHighlightPage(keyword,pageNo,pageSize); } }
ContentService
@Service public class ContentServiceImpl implements ContentService { @Autowired private RestHighLevelClient restHighLevelClient; //1.解析数据放入es索引中 public Boolean parseContent(String keywords) throws Exception{ List<Content> contents=new HtmlParseUtil().parseJD(keywords); //把查询到的数据放入es中 BulkRequest bulkRequest = new BulkRequest(); bulkRequest.timeout("2m"); for(int i=0;i<contents.size();i++){ bulkRequest.add( new IndexRequest("jd_goods") .source(JSON.toJSONString(contents.get(i)), XContentType.JSON) ); } BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT); return !bulk.hasFailures(); } //2.获取数据实现搜索功能 分页 public List<Map<String,Object>> searchPage(String keyword,int pageNo,int pageSize) throws IOException { if (pageNo<=1) { pageNo=1; } //条件搜索 SearchRequest searchRequest = new SearchRequest("jd_goods"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); //分页 sourceBuilder.from(pageNo); sourceBuilder.size(pageSize); //精准匹配 TermQueryBuilder termQueryBuilder= QueryBuilders.termQuery("title",keyword); sourceBuilder.query(termQueryBuilder); sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); //60s内没刷新则代表处问题 //执行搜索 searchRequest.source(sourceBuilder); SearchResponse searchResponse=restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT); //解析结果 ArrayList<Map<String,Object>> list=new ArrayList<>(); for (SearchHit hit : searchResponse.getHits().getHits()) { list.add(hit.getSourceAsMap()); } return list; } //2.获取数据实现搜索功能 分页 public List<Map<String,Object>> searchHighlightPage(String keyword,int pageNo,int pageSize) throws IOException { if (pageNo<=1) { pageNo=1; } //条件搜索 SearchRequest searchRequest = new SearchRequest("jd_goods"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); //分页 sourceBuilder.from(pageNo); sourceBuilder.size(pageSize); //精准匹配 TermQueryBuilder termQueryBuilder= QueryBuilders.termQuery("title",keyword); sourceBuilder.query(termQueryBuilder); sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); //60s内没刷新则代表处问题 //高亮 HighlightBuilder highlightBuilder=new HighlightBuilder(); highlightBuilder.field("title"); highlightBuilder.requireFieldMatch(false); //多个高亮显示 highlightBuilder.preTags("<span style='color:red'>"); highlightBuilder.postTags("</span>"); sourceBuilder.highlighter(highlightBuilder); //执行搜索 searchRequest.source(sourceBuilder); SearchResponse searchResponse=restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT); //解析结果 ArrayList<Map<String,Object>> list=new ArrayList<>(); for (SearchHit hit : searchResponse.getHits().getHits()) { Map<String, HighlightField> highlightFields=hit.getHighlightFields(); HighlightField title = highlightFields.get("title"); Map<String, Object> sourceAsMap = hit.getSourceAsMap(); //原来的结果 //解析高亮的字段,将原来的字段换为我们高亮的字段即可! if(title!=null){ Text[] fragments=title.fragments(); String n_title=""; for (Text text : fragments) { n_title +=text; sourceAsMap.put("title",n_title); //高亮字段替换掉原来的内容 } } list.add(hit.getSourceAsMap()); } return list; } }
ElasticSearchConfiguration
@Configuration
public class ElasticSearchClientConfig {
@Bean
public RestHighLevelClient restHighLevelClient(){
RestHighLevelClient client=new RestHighLevelClient(
RestClient.builder(
new HttpHost("127.0.0.1",9200,"http")));
return client;
}
}
创建新的索引"jd_goods"
运行localhost:9090/parse/java,查看ES发现数据全部数据已经全部加载
遇到的问题Elasticsearch built-in security features are not enabled
在ES的conf下的elasticsearch.yml文件
network.host: 0.0.0.0
xpack.security.enabled: false
查询结果
js中导入axios.min.js、vue.min.js、jquery.min.js
<!--修改的数据--> <div class="page" id="app"> 遍历商品信息 <div class="product" v-for="result in results"> <div class="product-iWrap"> <!--商品封面--> <div class="productImg-wrap"> <a class="productImg"> <img :src="result.img"> </a> </div> <!--价格--> <p class="productPrice"> <em><b>¥</b>{{result.price}}}</em> </p> <!--标题--> <p class="productTitle"> <a v-html="result.title"> </a> <!--解析新修改的高亮标签--> </p> <!--前端使用Vue,完成前后端分离--> <script th:src="@{/js/axios.min.js}"></script> <script th:src="@{/js/vue.min.js}"></script> <script> new Vue({ el:'#app', data:{ keyword:'', //搜索的关键字 results:[] //搜索的结果 }, methods:{ searchKey(){ var keyword=this.keyword; console.log(keyword); //对接后端的接口 axios.post('search/'+keyword+"/1/30").then(response=>{ this.results=response.data; //绑定数据成功 }) } } })
结果展示
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。