devbox
IT小白
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

Elasticsearch生产实战（ik分词器、拼音分词、自动补全、自动纠错）_es拼音分词器

作者：IT小白 | 2024-08-11 23:53:22

赞

踩

es拼音分词器

目录

一.IK分词器

1.IK分词器介绍

2.安装

3.使用

 4.自定义词库

二.拼音分词器

1.拼音分词器介绍

2.安装

三.自动补全

1.效果演示

2.实战

四.自动纠错

1.场景描述

2.DSL实现

3.java实现

五.仿京东实战

一.IK分词器

1.IK分词器介绍

        默认的中文分词是将每个字看成一个词，这显然是不符合要求的，所以我们需要安装中
文分词器来解决这个问题。

        IK分词是一款国人开发的相对简单的中文分词器。虽然开发者自2012年之后就不在维护
了，但在工程应用中IK算是比较流行的一款！我们今天就介绍一下IK中文分词器的使用。

2.安装

ik分词器下载地址
https://github.com/medcl/elasticsearch-analysis-ik/releases

链接：https://pan.baidu.com/s/1z49plwtgCzxprTibFviLlw 
提取码：c8xg

（1）先将其解压，将解压后的elasticsearch文件夹重命名文件夹为ik

（2）将ik文件夹拷贝到elasticsearch/plugins 目录下

（3）重新启动es，即可加载IK分词器

3.使用

下面看下效果吧~，如果没有IK分词器会将所有的字都拆分当成词语

postman请求测试【最小切分】：http://127.0.0.1:9200/testindex/_analyze

post请求，参数为：{"analyzer": "ik_smart", "text": "北京天安门" }

 postman请求测试【最小切分】： http://127.0.0.1:9200/testindex/_analyze

post请求参数为：{"analyzer": "ik_max_word", "text": "北京天安门" }

 4.自定义词库

现在测试ik分词器中没有配置过的特殊词语时，还是会对所有字分词，这时候需要自定义词库

如下：

http://127.0.0.1:9200/testindex/_analyze

 {"analyzer": "ik_max_word", "text": "你个老6" }

（1）进入elasticsearch/plugins/ik/config目录

（2）新建一个my.dic文件，编辑内容：你个老6

修改IKAnalyzer.cfg.xml（在ik/config目录下）

 重新启动elasticsearch，测试分词效果

二.拼音分词器

1.拼音分词器介绍

pinyin 分词器可以让用户输入拼音，就能查找到相关的关键词。比如在某个商城搜索中，输入 shuihu，就能匹配到水壶。这样的体验还是非常好的。

2.安装

拼音分词器安装地址：

Releases · medcl/elasticsearch-analysis-pinyin · GitHub

从github上下载对应es版本的拼音分词器，解压到plugins目录下，重命名为pinyin，重启es即可生效

 测试

创建索引

  1. #创建索引
  2. PUT /medcl/
  3. {
  4.     "settings" : {
  5.         "analysis" : {
  6.             "analyzer" : {
  7.                 "pinyin_analyzer" : {
  8.                     "tokenizer" : "my_pinyin"
  9.                     }
  10.             },
  11.             "tokenizer" : {
  12.                 "my_pinyin" : {
  13.                     "type" : "pinyin",
  14.                     "keep_separate_first_letter" : false,
  15.                     "keep_full_pinyin" : true,
  16.                     "keep_original" : true,
  17.                     "limit_first_letter_length" : 16,
  18.                     "lowercase" : true,
  19.                     "remove_duplicated_term" : true
  20.                 }
  21.             }
  22.         }
  23.     }
  24. }
  1. #查看分词效果
  2. GET /medcl/_analyze
  3. {
  4.   "text": ["汤兵兵"],
  5.   "analyzer": "pinyin_analyzer"
  6. }
  1. {
  2.   "tokens" : [
  3.     {
  4.       "token" : "tang",
  5.       "start_offset" : 0,
  6.       "end_offset" : 0,
  7.       "type" : "word",
  8.       "position" : 0
  9.     },
  10.     {
  11.       "token" : "汤兵兵",
  12.       "start_offset" : 0,
  13.       "end_offset" : 0,
  14.       "type" : "word",
  15.       "position" : 0
  16.     },
  17.     {
  18.       "token" : "tbb",
  19.       "start_offset" : 0,
  20.       "end_offset" : 0,
  21.       "type" : "word",
  22.       "position" : 0
  23.     },
  24.     {
  25.       "token" : "bing",
  26.       "start_offset" : 0,
  27.       "end_offset" : 0,
  28.       "type" : "word",
  29.       "position" : 1
  30.     }
  31.   ]
  32. }

创建映射

  1. #创建映射
  2. POST /medcl/_mapping
  3. {
  4.   "properties": {
  5.     "name": {
  6.       "type": "keyword",
  7.       "fields": {
  8.         "pinyin": {
  9.           "type": "text",
  10.           "store": false,
  11.           "term_vector": "with_offsets",
  12.           "analyzer": "pinyin_analyzer",
  13.           "boost": 10
  14.         }
  15.       }
  16.     }
  17.   }
  18. }

批量添加数据

  1. #批量添加数据
  2. POST /medcl/_bulk
  3. {"index": {"_index": "medcl","_id": "1"}}
  4. {"name":"汤兵兵"}
  5. {"index": {"_index": "medcl","_id": "2"}}
  6. {"name":"阚阳阳"}
  7. {"index": {"_index": "medcl","_id": "3"}}
  8. {"name":"汤一辰"}
  9. {"index": {"_index": "medcl","_id": "4"}}
  10. {"name":"汤得明"}
  11. {"index": {"_index": "medcl","_id": "5"}}
  12. {"name":"张继琴"}
  13. {"index": {"_index": "medcl","_id": "6"}}
  14. {"name":"阚佳武"}
  15. {"index": {"_index": "medcl","_id": "7"}}
  16. {"name":"施玉芬"}
  17. {"index": {"_index": "medcl","_id": "8"}}
  18. {"name":"陆毅"}
  19. {"index": {"_index": "medcl","_id": "9"}}
  20. {"name":"刘德华"}

测试拼音搜索

  1.  
  2. #拼音分词搜索
  3. GET /medcl/_search
  4. {
  5.   "query": {
  6.     "match": {
  7.       "name.pinyin": "tbb"
  8.     }
  9.   }
  10. }

拼音分词器可选参数配置

  1. ** 可选参数 **
  2.  
  3. keep_first_letter ： 启用此选项时，例如：刘德华 > ldh，默认值：true
  4. keep_separate_first_letter ： 启用该选项时，将保留第一个字母分开，例如：刘德华 > l，d，h，默认：假的，注意：查询结果也许是太模糊，由于长期过频
  5. limit_first_letter_length ： 设置 first_letter 结果的最大长度，默认值：16
  6. keep_full_pinyin ： 当启用该选项，例如：刘德华 > [liu，de，hua]，默认值：true
  7. keep_joined_full_pinyin ： 启用此选项时，例如：刘德华 > [liudehua]，默认值：false
  8. keep_none_chinese ： 在结果中保留非中文字母或数字，默认值：true
  9. keep_none_chinese_together ： 保持非中国信一起，默认值：true，如：DJ 音乐家 - > DJ，yin，yue，jia，当设置为 false，例如：DJ 音乐家 - > D，J，yin，yue，jia，注意：keep_none_chinese 必须先启动
  10. keep_none_chinese_in_first_letter ： 第一个字母不能是中文，例如：刘德华 AT2016- > ldhat2016，default：true
  11. keep_none_chinese_in_joined_full_pinyin ： 保持非中文字母加入完整拼音，例如：刘德华 2016- > liudehua2016，默认：false
  12. none_chinese_pinyin_tokenize ： 打破非中国信成单独的拼音项，如果他们拼音，默认值：true，如：liudehuaalibaba13zhuanghan- > liu，de，hua，a，li，ba，ba，13，zhuang，han，注意： keep_none_chinese 和 keep_none_chinese_together 应首先启用
  13. keep_original ： 当启用此选项时，也将保留原始输入，默认值：false
  14. lowercase ： 小写非中文字母，默认值：true
  15. trim_whitespace ： 默认值：true
  16. remove_duplicated_term ： 当启用此选项时，将删除重复项以保存索引，例如：de 的 > de，默认值：false 注意：位置相关查询可能受影响

三.自动补全

1.效果演示

汉字自动补全

拼音自动补全

 拼音首字母自动补全

2.实战

（1）数据准备

  1. #创建索引
  2. PUT /product_completion_index/
  3. {
  4.   "settings": {
  5.     "number_of_shards": 1,
  6.     "number_of_replicas": 2,
  7.     "analysis": {
  8.       "analyzer": {
  9.         "ik_pinyin_analyzer": {
  10.           "type": "custom",
  11.           "tokenizer": "ik_smart",
  12.           "filter": "pinyin_filter"
  13.         }
  14.       },
  15.       "filter": {
  16.         "pinyin_filter": {
  17.           "type": "pinyin",
  18.           "keep_first_letter": true,
  19.           "keep_separate_first_letter": false,
  20.           "keep_full_pinyin": true,
  21.           "keep_original": true,
  22.           "limit_first_letter_length": 16,
  23.           "lowercase": true,
  24.           "remove_duplicated_term": true
  25.         }
  26.       }
  27.     }
  28.   }
  29. }
  30. #创建映射
  31. POST /product_completion_index/_mapping
  32. {
  33.   "properties": {
  34.     "name": {
  35.       "type": "keyword"
  36.     },
  37.     "searchkey": {
  38.       "type": "completion",
  39.       "analyzer": "ik_pinyin_analyzer"
  40.     }
  41.   }
  42. }
  43.  
  44. #批量新增数据
  45. POST /product_completion_index/_bulk
  46. {"index":{"_index":"product_completion_index","_id":"1"}}
  47. {"name":"小米(MI)","searchkey":"小米手机"}
  48. {"index":{"_index":"product_completion_index","_id":"2"}}
  49. {"searchkey":"小米10","name":"小米(MI)"}
  50. {"index":{"_index":"product_completion_index","_id":"3"}}
  51. {"searchkey":"小米电视","name":"小米(MI)"}
  52. {"index":{"_index":"product_completion_index","_id":"4"}}
  53. {"searchkey":"小米路由器","name":"小米(MI)"}
  54. {"index":{"_index":"product_completion_index","_id":"5"}}
  55. {"searchkey":"小米9","name":"小米(MI)"}
  56. {"index":{"_index":"product_completion_index","_id":"6"}}
  57. {"searchkey":"小米手机","name":"小米(MI)"}
  58. {"index":{"_index":"product_completion_index","_id":"7"}}
  59. {"searchkey":"小米耳环","name":"小米(MI)"}
  60. {"index":{"_index":"product_completion_index","_id":"8"}}
  61. {"searchkey":"小米8","name":"小米(MI)"}
  62. {"index":{"_index":"product_completion_index","_id":"9"}}
  63. {"searchkey":"小米10Pro","name":"小米(MI)"}
  64. {"index":{"_index":"product_completion_index","_id":"10"}}
  65. {"searchkey":"小米笔记本","name":"小米(MI)"}
  66. {"index":{"_index":"product_completion_index","_id":"11"}}
  67. {"searchkey":"小米摄像头","name":"小米(MI)"}
  68. {"index":{"_index":"product_completion_index","_id":"12"}}
  69. {"searchkey":"小米电饭煲","name":"小米(MI)"}
  70. {"index":{"_index":"product_completion_index","_id":"13"}}
  71. {"searchkey":"小米充电宝","name":"小米(MI)"}
  72. {"index":{"_index":"product_completion_index","_id":"14"}}
  73. {"searchkey":"adidas男鞋","name":"adidas男鞋"}
  74. {"index":{"_index":"product_completion_index","_id":"15"}}
  75. {"searchkey":"adidas女鞋","name":"adidas女鞋"}
  76. {"index":{"_index":"product_completion_index","_id":"16"}}
  77. {"searchkey":"adidas外套","name":"adidas外套"}
  78. {"index":{"_index":"product_completion_index","_id":"17"}}
  79. {"searchkey":"adidas裤子","name":"adidas裤子"}
  80. {"index":{"_index":"product_completion_index","_id":"18"}}
  81. {"searchkey":"adidas官方旗舰店","name":"adidas官方旗舰店"}
  82. {"index":{"_index":"product_completion_index","_id":"19"}}
  83. {"searchkey":"阿迪达斯袜子","name":"阿迪达斯袜子"}
  84. {"index":{"_index":"product_completion_index","_id":"20"}}
  85. {"searchkey":"阿迪达斯外套","name":"阿迪达斯外套"}
  86. {"index":{"_index":"product_completion_index","_id":"21"}}
  87. {"searchkey":"阿迪达斯运动鞋","name":"阿迪达斯运动鞋"}
  88. {"index":{"_index":"product_completion_index","_id":"22"}}
  89. {"searchkey":"耐克外套","name":"耐克外套"}
  90. {"index":{"_index":"product_completion_index","_id":"23"}}
  91. {"searchkey":"耐克运动鞋","name":"耐克运动鞋"}

（2）测试

  1. #汉字自动补全
  2. GET product_completion_index/_search
  3. {
  4. "from": 0,
  5. "size": 100,
  6. "suggest": {
  7.  "czbk-suggest": {
  8.   "prefix": "小米",
  9.   "completion": {
  10.    "field": "searchkey",
  11.    "size": 20,
  12.    "skip_duplicates": true
  13.   }
  14.  }
  15. }
  16. }
  17. // 结果
  18. {
  19.   "took" : 2,
  20.   "timed_out" : false,
  21.   "_shards" : {
  22.     "total" : 1,
  23.     "successful" : 1,
  24.     "skipped" : 0,
  25.     "failed" : 0
  26.   },
  27.   "hits" : {
  28.     "total" : {
  29.       "value" : 0,
  30.       "relation" : "eq"
  31.     },
  32.     "max_score" : null,
  33.     "hits" : [ ]
  34.   },
  35.   "suggest" : {
  36.     "czbk-suggest" : [
  37.       {
  38.         "text" : "xm",
  39.         "offset" : 0,
  40.         "length" : 2,
  41.         "options" : [
  42.           {
  43.             "text" : "小米10",
  44.             "_index" : "product_completion_index",
  45.             "_type" : "_doc",
  46.             "_id" : "2",
  47.             "_score" : 1.0,
  48.             "_source" : {
  49.               "searchkey" : "小米10",
  50.               "name" : "小米(MI)"
  51.             }
  52.           },
  53.           {
  54.             "text" : "小米10Pro",
  55.             "_index" : "product_completion_index",
  56.             "_type" : "_doc",
  57.             "_id" : "9",
  58.             "_score" : 1.0,
  59.             "_source" : {
  60.               "searchkey" : "小米10Pro",
  61.               "name" : "小米(MI)"
  62.             }
  63.           },
  64.           {
  65.             "text" : "小米8",
  66.             "_index" : "product_completion_index",
  67.             "_type" : "_doc",
  68.             "_id" : "8",
  69.             "_score" : 1.0,
  70.             "_source" : {
  71.               "searchkey" : "小米8",
  72.               "name" : "小米(MI)"
  73.             }
  74.           },
  75.           {
  76.             "text" : "小米9",
  77.             "_index" : "product_completion_index",
  78.             "_type" : "_doc",
  79.             "_id" : "5",
  80.             "_score" : 1.0,
  81.             "_source" : {
  82.               "searchkey" : "小米9",
  83.               "name" : "小米(MI)"
  84.             }
  85.           },
  86.           {
  87.             "text" : "小米充电宝",
  88.             "_index" : "product_completion_index",
  89.             "_type" : "_doc",
  90.             "_id" : "13",
  91.             "_score" : 1.0,
  92.             "_source" : {
  93.               "searchkey" : "小米充电宝",
  94.               "name" : "小米(MI)"
  95.             }
  96.           },
  97.           {
  98.             "text" : "小米手机",
  99.             "_index" : "product_completion_index",
  100.             "_type" : "_doc",
  101.             "_id" : "1",
  102.             "_score" : 1.0,
  103.             "_source" : {
  104.               "name" : "小米(MI)",
  105.               "searchkey" : "小米手机"
  106.             }
  107.           },
  108.           {
  109.             "text" : "小米摄像头",
  110.             "_index" : "product_completion_index",
  111.             "_type" : "_doc",
  112.             "_id" : "11",
  113.             "_score" : 1.0,
  114.             "_source" : {
  115.               "searchkey" : "小米摄像头",
  116.               "name" : "小米(MI)"
  117.             }
  118.           },
  119.           {
  120.             "text" : "小米电视",
  121.             "_index" : "product_completion_index",
  122.             "_type" : "_doc",
  123.             "_id" : "3",
  124.             "_score" : 1.0,
  125.             "_source" : {
  126.               "searchkey" : "小米电视",
  127.               "name" : "小米(MI)"
  128.             }
  129.           },
  130.           {
  131.             "text" : "小米电饭煲",
  132.             "_index" : "product_completion_index",
  133.             "_type" : "_doc",
  134.             "_id" : "12",
  135.             "_score" : 1.0,
  136.             "_source" : {
  137.               "searchkey" : "小米电饭煲",
  138.               "name" : "小米(MI)"
  139.             }
  140.           },
  141.           {
  142.             "text" : "小米笔记本",
  143.             "_index" : "product_completion_index",
  144.             "_type" : "_doc",
  145.             "_id" : "10",
  146.             "_score" : 1.0,
  147.             "_source" : {
  148.               "searchkey" : "小米笔记本",
  149.               "name" : "小米(MI)"
  150.             }
  151.           },
  152.           {
  153.             "text" : "小米耳环",
  154.             "_index" : "product_completion_index",
  155.             "_type" : "_doc",
  156.             "_id" : "7",
  157.             "_score" : 1.0,
  158.             "_source" : {
  159.               "searchkey" : "小米耳环",
  160.               "name" : "小米(MI)"
  161.             }
  162.           },
  163.           {
  164.             "text" : "小米路由器",
  165.             "_index" : "product_completion_index",
  166.             "_type" : "_doc",
  167.             "_id" : "4",
  168.             "_score" : 1.0,
  169.             "_source" : {
  170.               "searchkey" : "小米路由器",
  171.               "name" : "小米(MI)"
  172.             }
  173.           }
  174.         ]
  175.       }
  176.     ]
  177.   }
  178. }
  1. #拼音自动补全
  2. GET product_completion_index/_search
  3. {
  4. "from": 0,
  5. "size": 100,
  6. "suggest": {
  7.  "czbk-suggest": {
  8.   "prefix": "xiaomi",
  9.   "completion": {
  10.    "field": "searchkey",
  11.    "size": 20,
  12.    "skip_duplicates": true
  13.   }
  14.  }
  15. }
  16. }
  17.  
  18. // 结果
  19. {
  20.   "took" : 1,
  21.   "timed_out" : false,
  22.   "_shards" : {
  23.     "total" : 1,
  24.     "successful" : 1,
  25.     "skipped" : 0,
  26.     "failed" : 0
  27.   },
  28.   "hits" : {
  29.     "total" : {
  30.       "value" : 0,
  31.       "relation" : "eq"
  32.     },
  33.     "max_score" : null,
  34.     "hits" : [ ]
  35.   },
  36.   "suggest" : {
  37.     "czbk-suggest" : [
  38.       {
  39.         "text" : "xiaomi",
  40.         "offset" : 0,
  41.         "length" : 6,
  42.         "options" : [
  43.           {
  44.             "text" : "小米10",
  45.             "_index" : "product_completion_index",
  46.             "_type" : "_doc",
  47.             "_id" : "2",
  48.             "_score" : 1.0,
  49.             "_source" : {
  50.               "searchkey" : "小米10",
  51.               "name" : "小米(MI)"
  52.             }
  53.           },
  54.           {
  55.             "text" : "小米10Pro",
  56.             "_index" : "product_completion_index",
  57.             "_type" : "_doc",
  58.             "_id" : "9",
  59.             "_score" : 1.0,
  60.             "_source" : {
  61.               "searchkey" : "小米10Pro",
  62.               "name" : "小米(MI)"
  63.             }
  64.           },
  65.           {
  66.             "text" : "小米8",
  67.             "_index" : "product_completion_index",
  68.             "_type" : "_doc",
  69.             "_id" : "8",
  70.             "_score" : 1.0,
  71.             "_source" : {
  72.               "searchkey" : "小米8",
  73.               "name" : "小米(MI)"
  74.             }
  75.           },
  76.           {
  77.             "text" : "小米9",
  78.             "_index" : "product_completion_index",
  79.             "_type" : "_doc",
  80.             "_id" : "5",
  81.             "_score" : 1.0,
  82.             "_source" : {
  83.               "searchkey" : "小米9",
  84.               "name" : "小米(MI)"
  85.             }
  86.           },
  87.           {
  88.             "text" : "小米充电宝",
  89.             "_index" : "product_completion_index",
  90.             "_type" : "_doc",
  91.             "_id" : "13",
  92.             "_score" : 1.0,
  93.             "_source" : {
  94.               "searchkey" : "小米充电宝",
  95.               "name" : "小米(MI)"
  96.             }
  97.           },
  98.           {
  99.             "text" : "小米手机",
  100.             "_index" : "product_completion_index",
  101.             "_type" : "_doc",
  102.             "_id" : "1",
  103.             "_score" : 1.0,
  104.             "_source" : {
  105.               "name" : "小米(MI)",
  106.               "searchkey" : "小米手机"
  107.             }
  108.           },
  109.           {
  110.             "text" : "小米摄像头",
  111.             "_index" : "product_completion_index",
  112.             "_type" : "_doc",
  113.             "_id" : "11",
  114.             "_score" : 1.0,
  115.             "_source" : {
  116.               "searchkey" : "小米摄像头",
  117.               "name" : "小米(MI)"
  118.             }
  119.           },
  120.           {
  121.             "text" : "小米电视",
  122.             "_index" : "product_completion_index",
  123.             "_type" : "_doc",
  124.             "_id" : "3",
  125.             "_score" : 1.0,
  126.             "_source" : {
  127.               "searchkey" : "小米电视",
  128.               "name" : "小米(MI)"
  129.             }
  130.           },
  131.           {
  132.             "text" : "小米电饭煲",
  133.             "_index" : "product_completion_index",
  134.             "_type" : "_doc",
  135.             "_id" : "12",
  136.             "_score" : 1.0,
  137.             "_source" : {
  138.               "searchkey" : "小米电饭煲",
  139.               "name" : "小米(MI)"
  140.             }
  141.           },
  142.           {
  143.             "text" : "小米笔记本",
  144.             "_index" : "product_completion_index",
  145.             "_type" : "_doc",
  146.             "_id" : "10",
  147.             "_score" : 1.0,
  148.             "_source" : {
  149.               "searchkey" : "小米笔记本",
  150.               "name" : "小米(MI)"
  151.             }
  152.           },
  153.           {
  154.             "text" : "小米耳环",
  155.             "_index" : "product_completion_index",
  156.             "_type" : "_doc",
  157.             "_id" : "7",
  158.             "_score" : 1.0,
  159.             "_source" : {
  160.               "searchkey" : "小米耳环",
  161.               "name" : "小米(MI)"
  162.             }
  163.           },
  164.           {
  165.             "text" : "小米路由器",
  166.             "_index" : "product_completion_index",
  167.             "_type" : "_doc",
  168.             "_id" : "4",
  169.             "_score" : 1.0,
  170.             "_source" : {
  171.               "searchkey" : "小米路由器",
  172.               "name" : "小米(MI)"
  173.             }
  174.           }
  175.         ]
  176.       }
  177.     ]
  178.   }
  179. }
  1. #首字母自动补全
  2. GET product_completion_index/_search
  3. {
  4. "from": 0,
  5. "size": 100,
  6. "suggest": {
  7.  "czbk-suggest": {
  8.   "prefix": "xm",
  9.   "completion": {
  10.    "field": "searchkey",
  11.    "size": 20,
  12.    "skip_duplicates": true
  13.   }
  14.  }
  15. }
  16. }
  17.  
  18. //结果
  19. {
  20.   "took" : 1,
  21.   "timed_out" : false,
  22.   "_shards" : {
  23.     "total" : 1,
  24.     "successful" : 1,
  25.     "skipped" : 0,
  26.     "failed" : 0
  27.   },
  28.   "hits" : {
  29.     "total" : {
  30.       "value" : 0,
  31.       "relation" : "eq"
  32.     },
  33.     "max_score" : null,
  34.     "hits" : [ ]
  35.   },
  36.   "suggest" : {
  37.     "czbk-suggest" : [
  38.       {
  39.         "text" : "xm",
  40.         "offset" : 0,
  41.         "length" : 2,
  42.         "options" : [
  43.           {
  44.             "text" : "小米10",
  45.             "_index" : "product_completion_index",
  46.             "_type" : "_doc",
  47.             "_id" : "2",
  48.             "_score" : 1.0,
  49.             "_source" : {
  50.               "searchkey" : "小米10",
  51.               "name" : "小米(MI)"
  52.             }
  53.           },
  54.           {
  55.             "text" : "小米10Pro",
  56.             "_index" : "product_completion_index",
  57.             "_type" : "_doc",
  58.             "_id" : "9",
  59.             "_score" : 1.0,
  60.             "_source" : {
  61.               "searchkey" : "小米10Pro",
  62.               "name" : "小米(MI)"
  63.             }
  64.           },
  65.           {
  66.             "text" : "小米8",
  67.             "_index" : "product_completion_index",
  68.             "_type" : "_doc",
  69.             "_id" : "8",
  70.             "_score" : 1.0,
  71.             "_source" : {
  72.               "searchkey" : "小米8",
  73.               "name" : "小米(MI)"
  74.             }
  75.           },
  76.           {
  77.             "text" : "小米9",
  78.             "_index" : "product_completion_index",
  79.             "_type" : "_doc",
  80.             "_id" : "5",
  81.             "_score" : 1.0,
  82.             "_source" : {
  83.               "searchkey" : "小米9",
  84.               "name" : "小米(MI)"
  85.             }
  86.           },
  87.           {
  88.             "text" : "小米充电宝",
  89.             "_index" : "product_completion_index",
  90.             "_type" : "_doc",
  91.             "_id" : "13",
  92.             "_score" : 1.0,
  93.             "_source" : {
  94.               "searchkey" : "小米充电宝",
  95.               "name" : "小米(MI)"
  96.             }
  97.           },
  98.           {
  99.             "text" : "小米手机",
  100.             "_index" : "product_completion_index",
  101.             "_type" : "_doc",
  102.             "_id" : "1",
  103.             "_score" : 1.0,
  104.             "_source" : {
  105.               "name" : "小米(MI)",
  106.               "searchkey" : "小米手机"
  107.             }
  108.           },
  109.           {
  110.             "text" : "小米摄像头",
  111.             "_index" : "product_completion_index",
  112.             "_type" : "_doc",
  113.             "_id" : "11",
  114.             "_score" : 1.0,
  115.             "_source" : {
  116.               "searchkey" : "小米摄像头",
  117.               "name" : "小米(MI)"
  118.             }
  119.           },
  120.           {
  121.             "text" : "小米电视",
  122.             "_index" : "product_completion_index",
  123.             "_type" : "_doc",
  124.             "_id" : "3",
  125.             "_score" : 1.0,
  126.             "_source" : {
  127.               "searchkey" : "小米电视",
  128.               "name" : "小米(MI)"
  129.             }
  130.           },
  131.           {
  132.             "text" : "小米电饭煲",
  133.             "_index" : "product_completion_index",
  134.             "_type" : "_doc",
  135.             "_id" : "12",
  136.             "_score" : 1.0,
  137.             "_source" : {
  138.               "searchkey" : "小米电饭煲",
  139.               "name" : "小米(MI)"
  140.             }
  141.           },
  142.           {
  143.             "text" : "小米笔记本",
  144.             "_index" : "product_completion_index",
  145.             "_type" : "_doc",
  146.             "_id" : "10",
  147.             "_score" : 1.0,
  148.             "_source" : {
  149.               "searchkey" : "小米笔记本",
  150.               "name" : "小米(MI)"
  151.             }
  152.           },
  153.           {
  154.             "text" : "小米耳环",
  155.             "_index" : "product_completion_index",
  156.             "_type" : "_doc",
  157.             "_id" : "7",
  158.             "_score" : 1.0,
  159.             "_source" : {
  160.               "searchkey" : "小米耳环",
  161.               "name" : "小米(MI)"
  162.             }
  163.           },
  164.           {
  165.             "text" : "小米路由器",
  166.             "_index" : "product_completion_index",
  167.             "_type" : "_doc",
  168.             "_id" : "4",
  169.             "_score" : 1.0,
  170.             "_source" : {
  171.               "searchkey" : "小米路由器",
  172.               "name" : "小米(MI)"
  173.             }
  174.           }
  175.         ]
  176.       }
  177.     ]
  178.   }
  179. }

（3）代码实现

返回数据封装

  1. package com.tangbb.elasticsearch.pojo;
  2.  
  3.  
  4. //import com.fasterxml.jackson.annotation.JsonInclude;
  5.  
  6. import com.alibaba.fastjson.annotation.JSONField;
  7. import com.alibaba.fastjson.serializer.SerializerFeature;
  8. import com.fasterxml.jackson.annotation.JsonInclude;
  9. import com.tangbb.elasticsearch.pojo.ResultEnum;
  10.  
  11.  
  12. import java.io.Serializable;
  13.  
  14. /**
  15.  * @Class: ResponseData
  16.  * @Package com.itheima.commons.result
  17.  * @Description: 数据返回封装类
  18.  * @Company: http://www.itheima.com/
  19.  */
  20. //如果加该注解的字段为null,那么就不序列化
  21. @JsonInclude(JsonInclude.Include.NON_NULL)
  22. public class ResponseData<T> implements Serializable {
  23.  
  24.     //返回码
  25.     private String code;
  26.     //返回信息
  27.     private String desc;
  28.     //返回的数据
  29.     private T data;
  30.     //返回数据总数
  31.     private Integer count;
  32.  
  33.     public Integer getCount() {
  34.         return count;
  35.     }
  36.  
  37.     public void setCount(Integer count) {
  38.         this.count = count;
  39.     }
  40.  
  41.     public String getCode() {
  42.         return code;
  43.     }
  44.  
  45.     public void setCode(String code) {
  46.         this.code = code;
  47.     }
  48.  
  49.     public String getDesc() {
  50.         return desc;
  51.     }
  52.  
  53.     public T getData() {
  54.         return data;
  55.     }
  56.  
  57.     public ResponseData(T data, ResultEnum resultEnum) {
  58.         this.code = resultEnum.getCode();
  59.         this.desc = resultEnum.getDecs();
  60.         this.data = data;
  61.     }
  62.  
  63.     public ResponseData(ResultEnum resultEnum) {
  64.         this.code = resultEnum.getCode();
  65.         this.desc = resultEnum.getDecs();
  66.     }
  67.  
  68.     public ResponseData(String code, String desc) {
  69.         this.code = code;
  70.         this.desc = desc;
  71.     }
  72.  
  73.     public ResponseData setResultEnum(ResultEnum result) {
  74.         this.code = result.getCode();
  75.         this.desc = result.getDecs();
  76.         return this;
  77.     }
  78.  
  79.     public ResponseData setResultEnum(T data, ResultEnum resultEnum, Integer count) {
  80.         this.code = resultEnum.getCode();
  81.         this.desc = resultEnum.getDecs();
  82.         this.data = data;
  83.         this.count = count;
  84.         return this;
  85.     }
  86.  
  87.     public ResponseData(T data, ResultEnum resultEnum, Integer count) {
  88.         this.code = resultEnum.getCode();
  89.         this.desc = resultEnum.getDecs();
  90.         this.data = data;
  91.         this.count = count;
  92.     }
  93.  
  94.     public ResponseData() {
  95.     }
  96.  
  97.  
  98.     public ResponseData setResultEnum(String code, String desc) {
  99.         this.code = code;
  100.         this.desc = desc;
  101.         return this;
  102.     }
  103. }

枚举类

  1. package com.tangbb.elasticsearch.pojo;
  2.  
  3. /**
  4.  * @Class: ResultEnum
  5.  * @Package com.itheima.commons.enums
  6.  * @Description: 操作提示枚举类
  7.  * @Company: http://www.itheima.com/
  8.  */
  9. public enum ResultEnum {
  10.     success("200", "操作成功！"),
  11.     param_isnull("-400", "参数为空"),
  12.     error("-402", "操作失败！"),
  13.     server_error("-500", "服务异常"),
  14.     data_existent("-504", "数据不存在"),
  15.     result_empty("-000", "查询内容为空"),
  16.  
  17.     NOT_SYSTEM_API("404", "不是系统指定api"),
  18.     REPEAT("666", "数据已存在"),
  19.     HTTP_ERROR("-405", "请求异常");
  20.     private String code;
  21.     private String decs;
  22.  
  23.     public String getCode() {
  24.         return code;
  25.     }
  26.  
  27.     public String getDecs() {
  28.         return decs;
  29.     }
  30.  
  31.     ResultEnum(String code, String decs) {
  32.         this.code = code;
  33.         this.decs = decs;
  34.     }
  35.  
  36. }

日志类

  1. package com.tangbb.elasticsearch.pojo;
  2.  
  3. /**
  4.  * @Class: ResultEnum
  5.  * @Package com.itheima.commons.enums
  6.  * @Description: 应用层操作提示
  7.  * @Company: http://www.itheima.com/
  8.  */
  9. public enum TipsEnum {
  10.     create_index_success("创建索引成功！"),
  11.     create_index_fail("创建索引失败！"),
  12.     delete_index_success("删除索引成功！"),
  13.     delete_index_fail("删除索引失败！"),
  14.     open_index_success("打开索引成功！"),
  15.     open_index_fail("打开索引失败！"),
  16.     close_index_success("关闭索引成功！"),
  17.     close_index_fail("关闭索引失败！"),
  18.     alias_index_success("索引别名设置成功！"),
  19.     alias_index_fail("索引别名设置失败！"),
  20.     exists_index_success("索引是否存在查询成功！"),
  21.     exists_index_fail("引是否存在查询失败！"),
  22.  
  23.  
  24.     create_doc_success("创建文档成功！"),
  25.     create_doc_fail("创建文档失败！"),
  26.     batch_create_doc_success("批量创建文档成功！"),
  27.     batch_create_doc_fail("批量创建文档失败！"),
  28.     update_doc_success("修改文档成功！"),
  29.     update_doc_fail("修改文档失败！"),
  30.     get_doc_success("查询文档成功！"),
  31.     batch_get_doc_fail("批量查询文档失败！"),
  32.     batch_get_doc_success("批量查询文档成功！"),
  33.     get_doc_fail("查询文档失败！"),
  34.     delete_doc_success("删除文档成功！"),
  35.     delete_doc_fail("删除文档失败！"),
  36.     csuggest_get_doc_fail("自动补全获取失败！"),
  37.     csuggest_get_doc_success("自动补全获取成功！"),
  38.  
  39.     psuggest_get_doc_fail("拼写纠错获取失败！"),
  40.     psuggest_get_doc_success("拼写纠错获取成功！"),
  41.  
  42.     tsuggest_get_doc_fail("搜索推荐获取失败！"),
  43.     tsuggest_get_doc_success("搜索推荐获取成功！"),
  44.  
  45.     hotwords_get_doc_fail("搜索热词获取失败！"),
  46.     hotwords_get_doc_success("搜索热词获取成功！"),
  47.     metricagg_get_doc_fail("指标聚合处理失败！"),
  48.     metricagg_get_doc_success("指标聚合处理成功！"),
  49.  
  50.     bucketagg_get_doc_fail("桶聚合处理失败！"),
  51.     bucketagg_get_doc_success("桶聚合处理成功！"),
  52.  
  53.  
  54.     index_default("索引创建失败！");
  55.  
  56.     private String message;
  57.  
  58.     public String getMessage() {
  59.         return message;
  60.     }
  61.  
  62.     TipsEnum(String message) {
  63.         this.message = message;
  64.  
  65.     }
  66.  
  67. }

实体类

  1. package com.tangbb.elasticsearch.pojo;
  2.  
  3. import com.fasterxml.jackson.annotation.JsonInclude;
  4.  
  5. import java.io.Serializable;
  6. import java.util.List;
  7. import java.util.Map;
  8.  
  9. /**
  10.  * @Class: CommonEntity
  11.  * @Package com.itheima.commons.pojo
  12.  * @Description: 公共实体类
  13.  * @Company: http://www.itheima.com/
  14.  */
  15. //如果加该注解的字段为null,那么就不序列化
  16. @JsonInclude(JsonInclude.Include.NON_NULL)
  17. public class CommonEntity implements Serializable {
  18.     //页码
  19.     private int pageNumber;
  20.     //每页数据条数
  21.     private int pageSize;
  22.     //索引名称
  23.     private String indexName;
  24.     //高亮列
  25.     private String highlight;
  26.     //排序 DESC  ASC
  27.     private String sortOrder;
  28.     //排序列
  29.     private String sortField;
  30.     //自动补全建议列
  31.     private String suggestFileld;
  32.     //自动补全建议值
  33.     private String suggestValue;
  34.     //自动补全返回个数
  35.     private Integer suggestCount;
  36.     //动态查询参数封装
  37.     Map<String, Object> map;
  38.     //批量增加list
  39.     private List<Map<String, Object>> list;
  40.  
  41.     public int getPageNumber() {
  42.         return pageNumber;
  43.     }
  44.  
  45.     public void setPageNumber(int pageNumber) {
  46.         this.pageNumber = pageNumber;
  47.     }
  48.  
  49.     public int getPageSize() {
  50.         return pageSize;
  51.     }
  52.  
  53.     public void setPageSize(int pageSize) {
  54.         this.pageSize = pageSize;
  55.     }
  56.  
  57.     public String getIndexName() {
  58.         return indexName;
  59.     }
  60.  
  61.     public void setIndexName(String indexName) {
  62.         this.indexName = indexName;
  63.     }
  64.  
  65.     public String getHighlight() {
  66.         return highlight;
  67.     }
  68.  
  69.     public void setHighlight(String highlight) {
  70.         this.highlight = highlight;
  71.     }
  72.  
  73.     public String getSortOrder() {
  74.         return sortOrder;
  75.     }
  76.  
  77.     public void setSortOrder(String sortOrder) {
  78.         this.sortOrder = sortOrder;
  79.     }
  80.  
  81.     public String getSortField() {
  82.         return sortField;
  83.     }
  84.  
  85.     public void setSortField(String sortField) {
  86.         this.sortField = sortField;
  87.     }
  88.  
  89.     public String getSuggestFileld() {
  90.         return suggestFileld;
  91.     }
  92.  
  93.     public void setSuggestFileld(String suggestFileld) {
  94.         this.suggestFileld = suggestFileld;
  95.     }
  96.  
  97.     public String getSuggestValue() {
  98.         return suggestValue;
  99.     }
  100.  
  101.     public void setSuggestValue(String suggestValue) {
  102.         this.suggestValue = suggestValue;
  103.     }
  104.  
  105.     public Integer getSuggestCount() {
  106.         return suggestCount;
  107.     }
  108.  
  109.     public void setSuggestCount(Integer suggestCount) {
  110.         this.suggestCount = suggestCount;
  111.     }
  112.  
  113.     public Map<String, Object> getMap() {
  114.         return map;
  115.     }
  116.  
  117.     public void setMap(Map<String, Object> map) {
  118.         this.map = map;
  119.     }
  120.  
  121.     public List<Map<String, Object>> getList() {
  122.         return list;
  123.     }
  124.  
  125.     public void setList(List<Map<String, Object>> list) {
  126.         this.list = list;
  127.     }
  128. }

控制层

  1.  /*
  2.      * @Description 自动补全
  3.      * @Method: suggester
  4.      * @Param: [commonEntity]
  5.      * @Update:
  6.      * @since: 1.0.0
  7.      * @Return: com.oldlu.commons.result.ResponseData
  8.      *
  9.      */
  10.     @GetMapping(value = "/csuggest")
  11.     public ResponseData cSuggest(@RequestBody CommonEntity commonEntity) {
  12.         // 构造返回数据
  13.         ResponseData rData = new ResponseData();
  14.         if (StringUtils.isEmpty(commonEntity.getIndexName()) ||
  15.                 StringUtils.isEmpty(commonEntity.getSuggestFileld()) ||
  16.                 StringUtils.isEmpty(commonEntity.getSuggestValue())) {
  17.             rData.setResultEnum(ResultEnum.param_isnull);
  18.             return rData;
  19.         }
  20.         //批量查询返回结果
  21.         List<String> result = null;
  22.         try {
  23.             //通过高阶API调用批量新增操作方法
  24.             result = contentService.cSuggest(commonEntity);
  25.             //通过类型推断自动装箱（多个参数取交集）
  26.             rData.setResultEnum(result, ResultEnum.success, result.size());
  27.             //日志记录
  28.             logger.info(TipsEnum.get_doc_success.getMessage());
  29.         } catch (Exception e) {
  30.             //日志记录
  31.             logger.error(TipsEnum.get_doc_fail.getMessage(), e);
  32.             //构建错误返回信息
  33.             rData.setResultEnum(ResultEnum.error);
  34.         }
  35.         return rData;
  36.     }

服务层

  1. //自动补全
  2.     /*
  3.      * @Description: 自动补全 根据用户的输入联想到可能的词或者短语
  4.      * @Method: suggester
  5.      * @Param: [commonEntity]
  6.      * @Update:
  7.      * @since: 1.0.0
  8.      * @Return: org.elasticsearch.action.search.SearchResponse
  9.      *
  10.      */
  11.     public List<String> cSuggest(CommonEntity commonEntity) throws Exception {
  12.         //定义返回
  13.         List<String> suggestList = new ArrayList<>();
  14.         //构建查询请求
  15.         SearchRequest searchRequest = new
  16.                 SearchRequest(commonEntity.getIndexName());
  17.         //通过查询构建器定义评分排序
  18.         SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
  19.         searchSourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
  20.         //构造搜索建议语句,搜索条件字段
  21.         CompletionSuggestionBuilder completionSuggestionBuilder =new
  22.                 CompletionSuggestionBuilder(commonEntity.getSuggestFileld());
  23.         //搜索关键字
  24.         completionSuggestionBuilder.prefix(commonEntity.getSuggestValue());
  25.         //去除重复
  26.         completionSuggestionBuilder.skipDuplicates(true);
  27.         //匹配数量
  28.         completionSuggestionBuilder.size(commonEntity.getSuggestCount());
  29.         searchSourceBuilder.suggest(new SuggestBuilder().addSuggestion("czbk-suggest", completionSuggestionBuilder));
  30.                 //czbk-suggest为返回的字段，所有返回将在czbk-suggest里面，可写死,sort按照评分排序
  31.                 searchRequest.source(searchSourceBuilder);
  32.         //定义查找响应
  33.         SearchResponse suggestResponse = restHighLevelClient.search(searchRequest,
  34.                 RequestOptions.DEFAULT);
  35.         //定义完成建议对象
  36.         CompletionSuggestion completionSuggestion =
  37.                 suggestResponse.getSuggest().getSuggestion("czbk-suggest");
  38.         List<CompletionSuggestion.Entry.Option> optionsList =
  39.                 completionSuggestion.getEntries().get(0).getOptions();
  40.         //从optionsList取出结果
  41.         if (!CollectionUtils.isEmpty(optionsList)) {
  42.             optionsList.forEach(item ->
  43.                     suggestList.add(item.getText().toString()));
  44.         }
  45.         return suggestList;
  46.     }

测试结果

http://localhost:9090/csuggest

四.自动纠错

1.场景描述

例如：错误输入"【adidaas官方旗舰店】 ”能够纠错为【adidas官方旗舰店】

2.DSL实现

  1. #自动纠错
  2. GET product_completion_index/_search
  3. {
  4. "suggest": {
  5.  "czbk-suggestion": {
  6.   "text": "adidaas官方旗舰店",
  7.   "phrase": {
  8.    "field": "name",
  9.    "size": 13
  10.   }
  11.  }
  12. }
  13. }

3.java实现

 控制层

  1. /*
  2.      * @Description: 拼写纠错
  3.      * @Method: suggester2
  4.      * @Param: [commonEntity]
  5.      * @Update:
  6.      * @since: 1.0.0
  7.      * @Return: com.oldlu.commons.result.ResponseData
  8.      *
  9.      */
  10.     @GetMapping(value = "/psuggest")
  11.     public ResponseData pSuggest(@RequestBody CommonEntity commonEntity) {
  12.         // 构造返回数据
  13.         ResponseData rData = new ResponseData();
  14.         if (StringUtils.isEmpty(commonEntity.getIndexName()) ||
  15.                 StringUtils.isEmpty(commonEntity.getSuggestFileld()) ||
  16.                 StringUtils.isEmpty(commonEntity.getSuggestValue())) {
  17.             rData.setResultEnum(ResultEnum.param_isnull);
  18.             return rData;
  19.         }
  20.         //批量查询返回结果
  21.         String result = null;
  22.         try {
  23.             //通过高阶API调用批量新增操作方法
  24.             result = contentService.pSuggest(commonEntity);
  25.             //通过类型推断自动装箱（多个参数取交集）
  26.             rData.setResultEnum(result, ResultEnum.success, null);
  27.             //日志记录
  28.             logger.info(TipsEnum.get_doc_success.getMessage());
  29.         } catch (Exception e) {
  30.             //日志记录
  31.             logger.error(TipsEnum.batch_get_doc_fail.getMessage(), e);
  32.             //构建错误返回信息
  33.             rData.setResultEnum(ResultEnum.error);
  34.         }
  35.         return rData;
  36.     }

服务层

  1. /*
  2.      * @Description: 拼写纠错
  3.      * @Method: psuggest
  4.      * @Param: [commonEntity]
  5.      * @Update:
  6.      * @since: 1.0.0
  7.      * @Return: java.util.List<java.lang.String>
  8.      *
  9.      */
  10.     public String pSuggest(CommonEntity commonEntity) throws Exception {
  11.         //定义返回
  12.         String pSuggestString = new String();
  13.         //定义查询请求
  14.         SearchRequest searchRequest = new
  15.                 SearchRequest(commonEntity.getIndexName());
  16.         //定义查询条件构建器
  17.         SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
  18.         //定义排序器
  19.         searchSourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
  20.         //构造短语建议器对象（参数为匹配列）
  21.         PhraseSuggestionBuilder pSuggestionBuilder = new
  22.                 PhraseSuggestionBuilder(commonEntity.getSuggestFileld());
  23.         //搜索关键字（被纠错的值）
  24.         pSuggestionBuilder.text(commonEntity.getSuggestValue());
  25.         //匹配数量
  26.         pSuggestionBuilder.size(1);
  27.         searchSourceBuilder.suggest(new SuggestBuilder().addSuggestion("czbk-suggest", pSuggestionBuilder));
  28.                 searchRequest.source(searchSourceBuilder);
  29.         //定义查找响应
  30.         SearchResponse suggestResponse = restHighLevelClient.search(searchRequest,
  31.                 RequestOptions.DEFAULT);
  32.         //定义短语建议对象
  33.         PhraseSuggestion phraseSuggestion =
  34.                 suggestResponse.getSuggest().getSuggestion("czbk-suggest");
  35.         //获取返回数据
  36.         List<PhraseSuggestion.Entry.Option> optionsList =
  37.                 phraseSuggestion.getEntries().get(0).getOptions();
  38.         //从optionsList取出结果
  39.         if (!CollectionUtils.isEmpty(optionsList)
  40.                 &&optionsList.get(0).getText()!=null) {
  41.             pSuggestString = optionsList.get(0).getText().string().replaceAll(" ","");
  42.         }
  43.         return pSuggestString;
  44.     }

 测试

http://localhost:9090/psuggest

五.仿京东实战

仿京东实战完整代码详见我的gitee，代码地址如下

https://gitee.com/EkkoBoy/elasticsearch.git

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/IT小白/article/detail/966713
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号