ElasticSearch分布式搜索引擎从入门到实战应用（实战篇-仿京东首页搜索商品高亮显示）_模拟实战京东搜索效果

1、熟悉SpringBoot集成ElasticSearch

1.1、官方指导文档

elasticsearch官方指导文档：https://www.elastic.co/guide/index.html

推荐使用REST风格操作es，可以直接根据REST Client客户端官方指导文档即可：
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/index.html

1.2、创建集成项目配置

1、引入springboot集成es客户端依赖

<!--springboot集成es-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
1
2
3
4
5

2、统一版本

	<!--此处version最好改为2.2.5.RELEASE，不要太高，也不要太低-->
	<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>


	<!--版本统一-->
    <properties>
        <java.version>1.8</java.version>
        <elasticsearch.version>7.6.1</elasticsearch.version>
    </properties>
1
2
3
4
5
6
7
8
9
10
11
12
13
14

3、导入后续会用到的关键依赖

		<!--lombok插件依赖，需要先安装lombok插件引入依赖才不build报错-->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        
        <!--fastjson-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.70</version>
        </dependency>
1
2
3
4
5
6
7
8
9
10
11
12
13

4、创建并编写配置类

@Configuration
public class ElasticSearchRestClientConfig {
    // 向spring容器中注入Rest高级客户端
    //方法名最好和返回类型保持一直，后续自动匹配装载时方便
    @Bean
    public RestHighLevelClient restHighLevelClient(){
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(new HttpHost("127.0.0.1",9200,"http"))
        );
        return client;
    }
}
1
2
3
4
5
6
7
8
9
10
11
12

5、创建并编写测试实体类

@Data //生成setter和getter方法
@NoArgsConstructor //生成无参构造函数
@AllArgsConstructor //生成带参构造函数
public class User implements Serializable {
    private String name;
    private Integer age;
}
1
2
3
4
5
6
7

1.3、测试索引-增删查

• 首先启动elasticsearch和es-head服务和插件
• 然后要启动项目的主启动类SpringbootElasticsearchApiApplication，因为要把RestHighLevelClient注入到spring容器中，在测试前一定一定要做这一步，后续的测试才不会报错，血的教训！！！
• 测试建议写在test包下的SpringbootElasticsearchApplicationTests类中

6.1、创建索引

@SpringBootTest
class SpringbootElasticsearchApplicationTests {

    @Autowired
    RestHighLevelClient restHighLevelClient;


    @Test
    public void testPUTCreateIndex() throws IOException {
        //创建索引请求对象，同时可初始化索引名
        CreateIndexRequest request = new CreateIndexRequest("yxj_index");
        //创建索引响应对应，默认类型
        CreateIndexResponse reponse = restHighLevelClient.indices().create(request,RequestOptions.DEFAULT);

        System.out.println(reponse.isAcknowledged());//根据响应状态，索引是够创建成功
        System.out.println(reponse);//查询响应对象信息
        restHighLevelClient.close();//用完一定要关闭客户端
    }

}

控制台结果：
true
org.elasticsearch.client.indices.CreateIndexResponse@5565235d
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

6.2、获取索引，并判断其是否存在

	@Test
    public void testGETIndexAndIsExists() throws IOException {
        //创建获取索引请求对象
        GetIndexRequest request = new GetIndexRequest("yxj_index");
        //创建获取索引响应对象
        GetIndexResponse response = restHighLevelClient.indices().get(request, RequestOptions.DEFAULT);
        //判断索引是否存在
        boolean exits = restHighLevelClient.indices().exists(request, RequestOptions.DEFAULT);

        System.out.println(response.getIndices());//输出索引信息(暂时还没数据)
        System.out.println(exits);//判断是否存在
        restHighLevelClient.close();//用完一定要关闭客户端
    }


控制台结果：
[Ljava.lang.String;@36790bec
true
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

6.3、删除索引

	@Test
    public void testDeleteIndex() throws IOException {
        //创建删除索引的请求对象
        DeleteIndexRequest request = new DeleteIndexRequest("yxj_index");
        //创建删除索引的响应对象
        AcknowledgedResponse response = restHighLevelClient.indices().delete(request,RequestOptions.DEFAULT);

        System.out.println(response.isAcknowledged());//判断删除是否成功
        restHighLevelClient.close();
    }

控制台结果：
true
1
2
3
4
5
6
7
8
9
10
11
12
13

1.4、测试文档-增删改查

1、添加文档

	/**
     * 向索引中添加文档信息
     */
    @Test
    void testAddDocument() throws IOException{
        //创建对象
        User user = new User("一宿君",21);
        //创建请求，链接索引库
        IndexRequest request = new IndexRequest("yxj_index");
        //规则  PUT /yxj_index/_doc/1
        request.id("1");
        request.timeout("1s");//设置超时时间为1s
        request.timeout(TimeValue.timeValueMinutes(1));//这两种方式应该都可以

        //将数据放入request请求中(json格式)
        request.source(JSON.toJSONString(user), XContentType.JSON);

        //客户端发送请求，获取响应的结果信息
        IndexResponse response = restHighLevelClient.index(request,RequestOptions.DEFAULT);

        System.out.println(response.status());//获取操作文档的状态
        System.out.println(response);//获取文档操作相应信息
        restHighLevelClient.close();
    }



控制台结果：
CREATED
IndexResponse[index=yxj_index,type=_doc,id=1,version=1,result=created,seqNo=0,primaryTerm=1,shards={"total":2,"successful":1,"failed":0}]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

2、获取文档信息

/**
     * 获取文档信息
     */
    @Test
    void testGetDocumntAndIsExits() throws IOException {
        //创建获取文档请求，指定索引名和文档id
        GetRequest request = new GetRequest("yxj_index","1");
        //过滤掉_source文档上下文，我们只需要判断文档是否存在，不需要获取内容，可以提高效率
        //request.fetchSourceContext(new FetchSourceContext(false));
        //不获取任何字段
        //request.storedFields("_none_");

        //获取值钱，先判断该文档是否存在（提高效率）
        boolean exists = restHighLevelClient.exists(request, RequestOptions.DEFAULT);

        if(exists){
            System.out.println("文档存在。。。");
            //发送请求获取响应对象(此处发送请求，如果使用上述的request过滤掉上下文，是获取不到内容的，可以把上述过滤注释掉)
            GetResponse response = restHighLevelClient.get(request,RequestOptions.DEFAULT);

            System.out.println(response.getSourceAsString());//获取文档全部内容，转换为字符串
            System.out.println(response);//获取全部相应信息（和Kibana的命令操作是一致的）
        }else {
            System.out.println("文档不存在！！！");
        }

        restHighLevelClient.close();//关闭客户端
    }


控制台结果：
文档存在。。。
{"age":21,"name":"一宿君"}
{"_index":"yxj_index","_type":"_doc","_id":"1","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"age":21,"name":"一宿君"}}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

3、文档更新

/**
     * 文档的更新
     */
    @Test
    void testUpdateDocument() throws IOException {
        //创建更新请求
        UpdateRequest request = new UpdateRequest("yxj_index","1");
        //创建更新数据
        User user = new User("一宿君Java",19);
        //将数据放入请求中，转换为JSON格式
        request.doc(JSON.toJSONString(user),XContentType.JSON);
        //发送请求
        UpdateResponse response = restHighLevelClient.update(request, RequestOptions.DEFAULT);

        System.out.println(response.status());//查询更新状态是否成功
        restHighLevelClient.close();//关闭客户端
    }

控制台结果：
OK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

4、文档的删除

	/**
     * 文档的删除
     * @throws IOException
     */
    @Test
    void testDeleteDocument() throws IOException {
        //创建删除请求
        DeleteRequest request = new DeleteRequest("yxj_index", "1");

        //发送请求
        DeleteResponse response = restHighLevelClient.delete(request, RequestOptions.DEFAULT);

        System.out.println(response.status());//查询更新状态是否成功
        restHighLevelClient.close();//关闭客户端
    }

控制台结果：
OK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

5、批量插入文档数据

	/**
     * 批处理插入文档数据
     * @throws IOException
     */
    @Test
    void testBulkInsertDocument() throws IOException {
        //创建批量出入请求对象
        BulkRequest request = new BulkRequest();
        request.timeout("1s");

        //创建集合文档数据
        List<User> userList = new ArrayList<>();
        userList.add(new User("一宿君1", 1));
        userList.add(new User("一宿君2", 2));
        userList.add(new User("一宿君3", 3));
        userList.add(new User("一宿君4", 4));
        userList.add(new User("一宿君5", 5));
        userList.add(new User("一宿君6", 6));

        //批量请求处理
        for(int i=0;i<userList.size();i++){
            request.add(
                    //此处是新建索引，或者指定已存在索引
                    new IndexRequest("bulk_index")
                            //id不指定的话会随机生成
                            .id(""+(i+1))
                            .source(JSON.toJSONString(userList.get(i)),XContentType.JSON)
            );
        }

        //提交请求
        BulkResponse response = restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
        System.out.println(response.status());//判断批处理插入处理的状态
        restHighLevelClient.close();//关闭客户端
    }

控制台结果：
OK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

6、文档带条件查询

	/**
     * 查询
     * SearchRequest 搜索请求
     *      SearchSourceBuilder 条件构造
     *          HighlightBuilder 高亮
     *          TermQueryBuilder 精确查询
     *          MatchAllQueryBuilder 匹配所有条件
     *          MatchQueryBuilder 匹配指定条件
     *          xxxQueryBuilder ...
     *      source : 添加构建条件
     *      indices ：指定查询索引 
     * @throws IOException
     */
	@Test
    void testHasConditionSearch() throws IOException {
        //创建查询条件请求对象
        SearchRequest request = new SearchRequest();
        //构建查询条件对象
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        /**
         * 查询可分为两种：
         * match匹配查询
         *      matchAll：匹配查询所有 MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
         * term精确查询
         */
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name","一宿君");
        //TermQueryBuilder queryBuilder = QueryBuilders.termQuery("name","一宿君");

        //将查询条件对象放入 请求构建查询条件对象中
        searchSourceBuilder.query(matchQueryBuilder);
        //设置高亮
        searchSourceBuilder.highlighter(new HighlightBuilder());
        //设置分页（当前第0页，每页显示3条数据）
        searchSourceBuilder.from(0);
        searchSourceBuilder.size(3);

        //将构建查询条件对象放入到请求查询条件对象中
        request.source(searchSourceBuilder);

        //此处是指定索引，如果不指定会遍历所有的索引
        request.indices("bulk_index");

        //客户单发送请求
        SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);

        System.out.println(response.status());//查看查询的状态
        System.out.println(response);//打印全部响应信息

        //获取查询结果集，并遍历
        SearchHits hits = response.getHits();//此处获取到的是整个hits标签，包含全部信息
        System.out.println(JSON.toJSONString(hits));//将结果集转换为JSON格式
        System.out.println("============================================================");

        //此处的hits内部才是包含数据
        for(SearchHit documentFields:hits.getHits()){
            System.out.println(documentFields.getSourceAsString());//这个是获取字符串格式
            //System.out.println(documentFields.getSourceAsMap());//这个是获取map集合对格式
        }

        restHighLevelClient.close();//关闭客户端
    }

控制台结果：
OK
{"took":19,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":6,"relation":"eq"},"max_score":0.22232392,"hits":[{"_index":"bulk_index","_type":"_doc","_id":"1","_score":0.22232392,"_source":{"age":1,"name":"一宿君1"}},{"_index":"bulk_index","_type":"_doc","_id":"2","_score":0.22232392,"_source":{"age":2,"name":"一宿君2"}},{"_index":"bulk_index","_type":"_doc","_id":"3","_score":0.22232392,"_source":{"age":3,"name":"一宿君3"}}]}}
{"fragment":true,"hits":[{"fields":{},"fragment":false,"highlightFields":{},"id":"1","matchedQueries":[],"primaryTerm":0,"rawSortValues":[],"score":0.22232392,"seqNo":-2,"sortValues":[],"sourceAsMap":{"name":"一宿君1","age":1},"sourceAsString":"{\"age\":1,\"name\":\"一宿君1\"}","sourceRef":{"fragment":true},"type":"_doc","version":-1},{"fields":{},"fragment":false,"highlightFields":{},"id":"2","matchedQueries":[],"primaryTerm":0,"rawSortValues":[],"score":0.22232392,"seqNo":-2,"sortValues":[],"sourceAsMap":{"name":"一宿君2","age":2},"sourceAsString":"{\"age\":2,\"name\":\"一宿君2\"}","sourceRef":{"fragment":true},"type":"_doc","version":-1},{"fields":{},"fragment":false,"highlightFields":{},"id":"3","matchedQueries":[],"primaryTerm":0,"rawSortValues":[],"score":0.22232392,"seqNo":-2,"sortValues":[],"sourceAsMap":{"name":"一宿君3","age":3},"sourceAsString":"{\"age\":3,\"name\":\"一宿君3\"}","sourceRef":{"fragment":true},"type":"_doc","version":-1}],"maxScore":0.22232392,"totalHits":{"relation":"EQUAL_TO","value":6}}
============================================================
{"age":1,"name":"一宿君1"}
{"age":2,"name":"一宿君2"}
{"age":3,"name":"一宿君3"}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

2、ElasticSearch实战-仿京东首页查询高亮

2.1、创建项目

静态界面资源包：

链接：https://pan.baidu.com/s/1L8_NtjVLMmOooK2m-L0Tlw
提取码：9gjc

配置application.properties配置文件：

#修改端口号
server.port=9090
#关闭thymeleaf缓存
spring.thymeleaf.cache=false
1
2
3
4

导入相关依赖（特别注意版本号）：

	<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.wbs</groupId>
    <artifactId>springboot-elasticsearch-jd</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>springboot-elasticsearch-jd</name>

    <!--版本统一-->
    <properties>
        <java.version>1.8</java.version>
        <elasticsearch.version>7.6.1</elasticsearch.version>
    </properties>


    <dependencies>
        <!--集成es-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <!--thymeleaf-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--fastjson-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.70</version>
        </dependency>

        <!--热部署-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <!--lombok-->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

编写IndexController层：

@Controller
public class IndexController {

    @RequestMapping({"/","/index"})
    public String toIndex(){
        return "index";
    }
}
1
2
3
4
5
6
7
8

启动项目，直接访问地址localhost:9090，首先保证我们的项目能正常启动和访问到首页：

2.2、基础爬虫拉取数据（jsoup）

数据获取的方式有很多种：

• 数据库
• 消息队列
• 缓存
• 爬虫
• 等等……

1、首先导入jsoup依赖包

		<!-- jsoup解析页面 -->
        <!-- 只能解析网页，想爬视频的话可以研究tiko -->
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.10.2</version>
        </dependency>
1
2
3
4
5
6
7

2、进入京东首页搜索商品关键字

查看地址栏地址：

https://search.jd.com/Search?keyword=Java&enc=utf-8
1

3、审查网页元素

4、编写工具类爬取数据（获取请求返回的页面信息，筛选出可用的）

public class HtmlParseUtilTest {
    public static void main(String[] args) throws IOException {
        //1、请求url
        String url = "https://search.jd.com/Search?keyword=Java&enc=utf-8";
        //2、解析网页（jsoup解析返回的就是浏览器Document对象，可以操作网页中所有的html元素）
        Document document = Jsoup.parse(new URL(url), 30000);

        //3、通过上述审查网页元素中的商品列表id，获取元素
        Element element = document.getElementById("J_goodsList");

        //4、获取element元素中ul下的每一个所有li元素
        Elements elements = element.getElementsByTag("li");

        //5、获取li元素下的商品属性：img、price、name、……
        for (Element el : elements) {
            System.out.println("img-src:" + el.getElementsByTag("img").eq(0).attr("src"));//获取li元素下的第一章照片
            System.out.println("name:" + el.getElementsByClass("p-name").eq(0).text());//获取商品名字
            System.out.println("price:" + el.getElementsByClass("p-price").eq(0).text());//获取商品价格
            System.out.println("shopname:" + el.getElementsByClass("hd-shopname").eq(0).text());//获取商品出版社
            System.out.println("================================================================================================");
        }
        
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

上述的情况是以为大型网站图片比较多，一般使用的都是图片延迟加载（也就是懒加载的方式）渲染图片，这样可以高相应速度。

更改图片获取属性为 ：data-lazy-img

5、编写实体类，存放商品属性信息

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Product implements Serializable {
    private String name;
    private String img;
    private String price;
    private String shopname;
    
    //……属性可以根据需要添加，这里只罗列几个关键属性即可
}
1
2
3
4
5
6
7
8
9
10
11

6、编写修改解析网页工具类，获取树

public class HtmlParseUtil {
    public static void main(String[] args) throws IOException {
        new HtmlParseUtil().parseJD("Java").forEach(System.out::println);
    }

    public List<Product> parseJD(String keyword) throws IOException {
        //1、请求url
        String url = "https://search.jd.com/Search?keyword=" + keyword +"&enc=utf-8";
        //2、解析网页（jsoup解析返回的就是浏览器Document对象，可以操作网页中所有的html元素）
        Document document = Jsoup.parse(new URL(url), 30000);

        //3、通过上述审查网页元素中的商品列表id，获取元素
        Element element = document.getElementById("J_goodsList");

        //4、获取element元素中ul下的每一个所有li元素
        Elements elements = element.getElementsByTag("li");

        //5、创建存储数据集合
        ArrayList<Product> productArrayList = new ArrayList<>();

        //6、获取li元素下的商品属性：img、price、name、shopname，并添加到集合中
        for (Element el : elements) {
            String img = el.getElementsByTag("img").eq(0).attr("data-lazy-img");//获取li元素下的第一章照片
            String name = el.getElementsByClass("p-name").eq(0).text();//获取商品名字
            String price = el.getElementsByClass("p-price").eq(0).text();//获取商品价格
            String shopname = el.getElementsByClass("hd-shopname").eq(0).text();//获取商品出版社

            //创建商品实体类
            Product product  = new Product(img,name,price,shopname);
            //添加到集合中
            productArrayList.add(product);
        }
        //返回集合
        return productArrayList;
    }

}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

注意：

执行查看结果：

2.3、编写service业务逻辑层接口及实现类

//接口
@Service
public interface ProductService {


    //爬取数据存入es中
    public Boolean parseProductSafeEs(String keyword) throws IOException;

    //分页查询
    public List<Map<String,Object>> searchProduct(String keyword, int pageNum, int pageSize) throws IOException;

}




//实现类
@Service
public class ProductServiceImpl implements ProductService {

    @Autowired
    RestHighLevelClient restHighLevelClient;

    @Override
    public Boolean parseProductSafeEs(String keyword) throws IOException {
        //获取数据集
        List<Product> productList = new HtmlParseUtil().parseJD(keyword);
        //创建批处理请求对象
        BulkRequest request = new BulkRequest();
        //设置超时时间为
        request.timeout("2s");

        //将批量数据存入es中
        for (int i = 0; i < productList.size(); i++) {
            request.add(
                    new IndexRequest("jd_pro_index")
                            .id("" + (i+1))
                            .source(JSON.toJSONString(productList.get(i)), XContentType.JSON)
            );
        }

        //提交请求
        BulkResponse response = restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
        boolean bool = response.hasFailures();//是否是失败，true代表失败，false代表成功
        //restHighLevelClient.close();
        return !bool;
    }

    @Override
    public List<Map<String, Object>> searchProduct(String keyword, int pageNum, int pageSize) throws IOException {
        if(pageNum < 0){
            pageNum  = 0;
        }

        //创建查询请求对象
        SearchRequest request = new SearchRequest("jd_pro_index");

        //构建查询条件对象
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        //构建查询条件(采用精确查询，根据keyword关键字查询name字段)
        //TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", keyword);
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name", keyword);

        //将查询条件放入到构建查询条件对象中
        searchSourceBuilder.query(matchQueryBuilder);
        //设置超时时间为60S
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        //分页
        searchSourceBuilder.from(pageNum);//当前页
        searchSourceBuilder.size(pageSize);//每页显示数量

        //将搜索条件源放入到搜索请求中
        request.source(searchSourceBuilder);

        //客户端发送请求,获取响应结果
        SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);

        //关闭客户端
        //restHighLevelClient.close();

        //解析获取结数据集
        SearchHits searchHits = response.getHits();

        //创建list集合
        List<Map<String,Object>> mapList = new ArrayList<>();

        //循环遍历数据集，封装到list集合中
        for (SearchHit hit : searchHits.getHits()) {
            Map<String,Object> hitMap = hit.getSourceAsMap();
            mapList.add(hitMap);
        }

        return mapList;
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

2.4、编写Controller前端访问层

注意：此处所有的方法都不要关闭RestHighLevelClient客户端，否则其他方法会无法继续访问，同时报IO异常。

@Controller
public class ProductController {

    @Autowired
    RestHighLevelClient restHighLevelClient;

    @Autowired
    ProductService productService;

    /**
     * 创建索引
     * @throws IOException
     */
    @RequestMapping("/createIndex")
    @ResponseBody
    public String creatIndex() throws IOException {
        CreateIndexRequest request = new CreateIndexRequest("jd_pro_index");
        CreateIndexResponse response = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);

        System.out.println(response.isAcknowledged());

        if(response.isAcknowledged()){
            return "创建成功!";
        }else{
            return "创建失败!";
        }

    }

    /**
     * 删除索引
     * @throws IOException
     */
    @RequestMapping("/deleteIndex")
    @ResponseBody
    public String deleteIndex() throws IOException {
        DeleteIndexRequest request = new DeleteIndexRequest("jd_pro_index");
        AcknowledgedResponse response = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);

        System.out.println(response.isAcknowledged());


        if(response.isAcknowledged()){
            return "删除成功!";
        }else{
            return "删除失败!";
        }

    }


    /**
     * 从京东首页爬取数据存入到es中
     * @param keyword
     * @return
     * @throws IOException
     */
    @RequestMapping("/toSafeEs/{keyword}")
    @ResponseBody
    public String parseProductSafeEs(@PathVariable("keyword") String keyword) throws IOException {
        if(productService.parseProductSafeEs(keyword)){
            return "爬取数据成功存入es中！";
        }
        return "爬取数据失败";
    }


    @RequestMapping("/searchEsDoc/{keyword}/{pageNum}/{pageSize}")
    @ResponseBody
    public List<Map<String, Object>> searchProduct(
            @PathVariable("keyword") String keyword,
            @PathVariable("pageNum") int pageNum,
            @PathVariable("pageSize") int pageSize) throws IOException {
        List<Map<String, Object>> mapList = productService.searchProduct(keyword, pageNum, pageSize);
        if (mapList != null){
            return mapList;
        }
        return null;
    }

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

2.5、测试接口

创建索引

爬取数据存入es中

查询数据：

2.6、前后端分离（简单使用Vue）

• 下载vue依赖：用于渲染前端页面
• 下载axios依赖：用于ajax请求后端接口

vue和axios都可以去官网下载，跟狂神学了一个小技巧，在本地新建一个英文目录文件夹，直接cmd进入该目录下（前提是安装了nodejs）：

#如果之前没有初始化过，可以先执行初始化
npm init

#下载vue
npm install vue

#下载axios
npm install axios
1
2
3
4
5
6
7
8

修改index.xml首页：

<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org">

<head>
    <meta charset="utf-8"/>
    <title>一宿君Java-ES仿京东实战</title>
    <link rel="stylesheet" th:href="@{/css/style.css}"/>
    <script th:src="@{/js/jquery.min.js}"></script>
</head>

<body class="pg">
<div class="page" id="app">
    <div id="mallPage" class=" mallist tmall- page-not-market ">

        <!-- 头部搜索 -->
        <div id="header" class=" header-list-app">
            <div class="headerLayout">
                <div class="headerCon ">
                    <!-- Logo-->
                    <h1 id="mallLogo">
                        <img th:src="@{/images/jdlogo.png}" alt="">
                    </h1>

                    <div class="header-extra">

                        <!--搜索-->
                        <div id="mallSearch" class="mall-search">
                            <form name="searchTop" class="mallSearch-form clearfix">
                                <fieldset>
                                    <legend>天猫搜索</legend>
                                    <div class="mallSearch-input clearfix">
                                        <div class="s-combobox" id="s-combobox-685">
                                            <div class="s-combobox-input-wrap">
                                                <input type="text" v-model="keyword" autocomplete="off" value="dd" id="mq"
                                                       class="s-combobox-input" aria-haspopup="true">
                                            </div>
                                        </div>
                                        <button type="submit" @click.prevent="searchDocument" id="searchbtn">搜索</button>
                                    </div>
                                </fieldset>
                            </form>
                            <ul class="relKeyTop">
                                <li><a>一宿君Java</a></li>
                                <li><a>一宿君前端</a></li>
                                <li><a>一宿君Linux</a></li>
                                <li><a>一宿君大数据</a></li>
                                <li><a>一宿君博客</a></li>
                            </ul>
                        </div>
                    </div>
                </div>
            </div>
        </div>

        <!-- 商品详情页面 -->
        <div id="content">
            <div class="main">
                <!-- 品牌分类 -->
                <form class="navAttrsForm">
                    <div class="attrs j_NavAttrs" style="display:block">
                        <div class="brandAttr j_nav_brand">
                            <div class="j_Brand attr">
                                <div class="attrKey">
                                    品牌
                                </div>
                                <div class="attrValues">
                                    <ul class="av-collapse row-2">
                                        <li><a href="#"> 一宿君 </a></li>
                                        <li><a href="#"> Java </a></li>
                                    </ul>
                                </div>
                            </div>
                        </div>
                    </div>
                </form>

                <!-- 排序规则 -->
                <div class="filter clearfix">
                    <a class="fSort fSort-cur">综合<i class="f-ico-arrow-d"></i></a>
                    <a class="fSort">人气<i class="f-ico-arrow-d"></i></a>
                    <a class="fSort">新品<i class="f-ico-arrow-d"></i></a>
                    <a class="fSort">销量<i class="f-ico-arrow-d"></i></a>
                    <a class="fSort">价格<i class="f-ico-triangle-mt"></i><i class="f-ico-triangle-mb"></i></a>
                </div>

                <!-- 商品详情 -->
                <div class="view grid-nosku">

                    <div class="product" v-for="result in results">
                        <div class="product-iWrap">
                            <!--商品封面-->
                            <div class="productImg-wrap">
                                <a class="productImg">
                                    <img :src="result.img">
                                </a>
                            </div>
                            <!--价格-->
                            <p class="productPrice">
                                <em v-text="result.price"></em>
                            </p>
                            <!--标题-->
                            <p class="productTitle">
                                <a v-html="result.name"></a>
                            </p>
                            <!-- 店铺名 -->
                            <div class="productShop">
                                <span v-text="result.shopname"></span>
                            </div>
                            <!-- 成交信息 -->
                            <p class="productStatus">
                                <span>月成交<em>999笔</em></span>
                                <span>评价 <a>3</a></span>
                            </p>
                        </div>
                    </div>
                </div>
            </div>
        </div>
    </div>
</div>

<!--引入vue和axios依赖-->
<script th:src="@{/js/axios.min.js}"></script>
<script th:src="@{/js/vue.min.js}"></script>

<script>
    new Vue({
        el: '#app',
        data: {
            pageNum: 0,//当前页码
            pageSize:8,//每页显示数据条数
            keyword: '',//查询条件关键字
            results: []//查询结果
        },
        methods: {
            searchDocument(){
                var keyword = this.keyword;
                var pageNum = this.pageNum;
                var pageSize = this.pageSize;
                //console.log(this.keyword);
                axios.get("searchEsDoc/" + keyword + "/" + pageNum + "/" + pageSize).then(response=>{
                    console.log(response.data);
                    this.results = response.data;
                });
            }
        }
    });
</script>

</body>
</html>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151

我们查看关键之处：

编写vue实例：

<!--引入vue和axios依赖-->
<script th:src="@{/js/axios.min.js}"></script>
<script th:src="@{/js/vue.min.js}"></script>

<script>
    new Vue({
        el: '#app',
        data: {
            pageNum: 0,//当前页码
            pageSize:8,//每页显示数据条数
            keyword: '',//查询条件关键字
            results: []//查询结果
        },
        methods: {
            searchDocument(){
                var keyword = this.keyword;
                var pageNum = this.pageNum;
                var pageSize = this.pageSize;
                //console.log(this.keyword);
                axios.get("searchEsDoc/" + keyword + "/" + pageNum + "/" + pageSize).then(response=>{
                    console.log(response.data);
                    this.results = response.data;
                });
            }
        }
    });
</script>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

div最外层挂载点：

<div class="page" id="app">
1

首页搜索框：”

遍历商品数据集：

重新启动项目，访问地址localhost:9090，搜索关键字

查询成功，只剩下最后一步高亮显示关键字！！！

2.7、高亮显示关键字

添加新的接口：

/分页高亮显示
public List<Map<String,Object>> searchHighlightProduct(String keyword, int pageNum, int pageSize) throws IOException;
1
2

实现类业务逻辑基层代码：

    @Override
    public List<Map<String, Object>> searchHighlightProduct(String keyword, int pageNum, int pageSize) throws IOException {
        if(pageNum < 0){
            pageNum  = 0;
        }

        //创建查询请求对象
        SearchRequest request = new SearchRequest("jd_pro_index");

        //构建查询条件对象
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        //构建查询条件(采用精确查询，根据keyword关键字查询name字段)
        //TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", keyword);
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name", keyword);

        //将查询条件放入到构建查询条件对象中
        searchSourceBuilder.query(matchQueryBuilder);
        //设置超时时间为60S
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        //分页
        searchSourceBuilder.from(pageNum);//当前页
        searchSourceBuilder.size(pageSize);//每页显示数量


        //设置高亮
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("name");//要高亮的字段
        highlightBuilder.requireFieldMatch(false);//多字段高亮显示，此处瓜关闭
        highlightBuilder.preTags("<span style='color:red'>");
        highlightBuilder.postTags("</span>");

        searchSourceBuilder.highlighter(highlightBuilder);




        //将搜索条件源放入到搜索请求中
        request.source(searchSourceBuilder);
        //客户端发送请求,获取响应结果
        SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
        //关闭客户端
        //restHighLevelClient.close();

        //解析获取结数据集
        SearchHits searchHits = response.getHits();
        //创建list集合
        List<Map<String,Object>> mapList = new ArrayList<>();

        //循环遍历数据集，封装到list集合中
        for (SearchHit hit : searchHits.getHits()) {
            //这是原来的字段集合
            Map<String,Object> sourceMap = hit.getSourceAsMap();

            //获取高亮字段集合
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            HighlightField highlightField = highlightFields.get("name");
            //解析高亮的字段，实际上解析的是字段的的片段，也就是我们搜索的“关键字”部分要高亮
            if(highlightField != null){
                //获取字段中高亮片段集合，将其原来的集合中的片段替换掉
                Text[] fragments = highlightField.getFragments();
                String n_fragments = "";
                for (Text fragment : fragments) {
                    n_fragments += fragment;
                }
                sourceMap.put("name",n_fragments);//高亮字段替换掉原来的字段
            }

            mapList.add(sourceMap);
        }

        return mapList;
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

重新启动项目，访问搜索关键：

大功告成，妥了，撒花！！！
狂神真狂！！！