当前位置:   article > 正文

SpringBoot+elasticsearchTemplate实现es索引数据的创建查询删除_springboot es withsearchtemplate

springboot es withsearchtemplate

通过elasticsearch实现创建索引、创建mapping映射、插入数据、模糊查询索引、查询index所对应的mapping、数据的滚动条件查询、删除指定数据

目录

elasticsearchTemplate创建索引、创建mapping映射(默认创建类型为doc)

elasticsearchTemplate添加数据

模糊查询index(查询以xxx开头的所有索引)并查询其对应的mapping包含的字段

elasticsearchTemplate滚动条件查询

elasticsearchTemplate查询指定索引类型下的所有数据

elasticsearchTemplate删除前缀为channel_channelId索引下taskId为xx的所有数据


elasticsearchTemplate创建索引、创建mapping映射(默认创建类型为doc)

  1. // mapping对象转为json字符串
  2. String s = objectMapper.writeValueAsString(channelOutput.getMappings());
  3. // 拼接索引名称index
  4. String index = "channel_" + channelAdd.getChannelId() + "_" + channelOutput.getType();
  5. // 判断索引是否已存在
  6. boolean b = elasticsearchTemplate.indexExists(index);
  7. // es索引不存在創建索引,mapping
  8. if (!b){
  9. elasticsearchTemplate.createIndex(index);
  10. elasticsearchTemplate.putMapping(index, "doc", s);
  11. }

elasticsearchTemplate添加数据

  1. List<IndexQuery> list=new ArrayList<>();
  2. IndexQuery indexQuery = new IndexQuery();
  3. // 设置索引名称 indexQuery.setIndexName("channel_"+taskDataAmount.getChannelId()+"_"+taskDataAmount.getType());
  4. // 设置索引下的类型
  5. indexQuery.setType("doc");
  6. // 需要添加的数据 indexQuery.setSource(objectMapper.writeValueAsString(taskDataAmount.getData()));
  7. list.add(indexQuery);
  8. elasticsearchTemplate.bulkIndex(list);

 特别说明:添加的数据结构和所设置的mapping结构不一样也可以添加成功

例如:mapping如下

  1. {
  2. "task_log": {
  3. "mappings": {
  4. "data": {
  5. "properties": {
  6. "className": {
  7. "type": "keyword"
  8. },
  9. "dataTypeId": {
  10. "type": "keyword"
  11. },
  12. "engineId": {
  13. "type": "keyword"
  14. },
  15. "engineName": {
  16. "type": "keyword"
  17. },
  18. "error": {
  19. "type": "keyword"
  20. },
  21. "extendsInfo": {
  22. "type": "keyword"
  23. },
  24. "method": {
  25. "type": "keyword"
  26. },
  27. "taskId": {
  28. "type": "keyword"
  29. }
  30. }
  31. }
  32. }
  33. }
  34. }

添加数据时,除了className这些字段还可以添加一些mapping中没有的字段

模糊查询index(查询以xxx开头的所有索引)并查询其对应的mapping包含的字段

  1. // 封装类
  2. @Data
  3. public class FieldVo {
  4. private String indexName;
  5. private Set<String> fieldNames;
  6. }
  1. List<FieldVo> fieldVos=new ArrayList<>();
  2. Client client = elasticsearchTemplate.getClient();
  3. IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
  4. // 前缀+*
  5. indicesStatsRequest.indices("channel_"+taskLogs.getChannelId()+"*");
  6. Map<String,IndexStats>stats=client.admin().indices().stats(indicesStatsRequest).actionGet().getIndices();
  7. for(IndexStats stat:stats.values()) {
  8. FieldVo fieldVo=new FieldVo();
  9. String in = stat.getIndex();//获取索引值
  10. fieldVo.setIndexName(in);
  11. //获取制定index下的mapping
  12. Map<String,String> o = (Map<String, String>) elasticsearchTemplate.getMapping(in, "doc").get("properties");
  13. //获取所有的字段
  14. Set<String> fields = o.keySet();
  15. fieldVo.setFieldNames(fields);

 以上两张图分别是Map stats和Map o的数据结构

elasticsearchTemplate滚动条件查询

  1. public ResultModel getDataDetail(String index,String scrollId,Long taskLogId,Integer pageSize){
  2. if (StringUtils.isEmpty(index)){
  3. return ResultModel.fail("索引为空");
  4. }
  5. if (null==taskLogId){
  6. return ResultModel.fail("taskLogId不能为空");
  7. }
  8. if (null==pageSize){
  9. pageSize=20;
  10. }
  11. TaskLogs taskLogs = taskLogsMapper.selectByPrimaryKey(taskLogId);
  12. // 创建一个查询条件对象
  13. BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
  14. // 拼接查询条件
  15. queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
  16. queryBuilder.must(QueryBuilders.termQuery("count",taskLogs.getCount()));
  17. SearchQuery searchQuery = new NativeSearchQueryBuilder()
  18. .withIndices(index)//索引名
  19. .withTypes("doc")//类型名
  20. .withPageable(PageRequest.of(0, pageSize))//0页开始查,每页1000个结果
  21. .withQuery(queryBuilder)
  22. .build();
  23. Page<Map> maps =null;
  24. if (StringUtils.isEmpty(scrollId)){
  25. maps=elasticsearchTemplate.startScroll(300000L, searchQuery, Map.class);
  26. }else {
  27. maps =elasticsearchTemplate.continueScroll(scrollId, 300000L, Map.class);
  28. }
  29. return ResultModel.success(maps);
  30. }

第一次查询时设置的pageSize,后续的查询中不能进行修改,因为是用的一个query

300000L是设置的过期时间

第一次请求是会返回一个scrollId,后续往下查询都需要把这个作为参数去请求

请求结果示例:

  1. {
  2. "code": 200,
  3. "message": "ok",
  4. "data": {
  5. "content": [
  6. {
  7. "id": "324",
  8. "name": "testONe",
  9. "username": "test",
  10. "count": 3,
  11. "taskId": 10
  12. }
  13. ],
  14. "pageable": "INSTANCE",
  15. "facets": [],
  16. "aggregations": null,
  17. "scrollId": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAyiXFkdkbm91M0hYUU9xSG13R3A0UHdoWkEAAAAAAAMolhZHZG5vdTNIWFFPcUhtd0dwNFB3aFpBAAAAAAAG2RcWdDJxTUx6QllUbmVmbjM2b0UzSmlFdwAAAAAABtgcFkpvMlZ0X04yVGVxdFE2ZWNOWEZJOWcAAAAAAAbYHRZKbzJWdF9OMlRlcXRRNmVjTlhGSTln",
  18. "totalElements": 1,
  19. "totalPages": 1,
  20. "number": 0,
  21. "size": 0,
  22. "sort": {
  23. "sorted": false,
  24. "unsorted": true
  25. },
  26. "numberOfElements": 1,
  27. "first": true,
  28. "last": true
  29. }
  30. }

elasticsearchTemplate查询指定索引类型下的所有数据

  1. public List<Map<String, Object>> test(){
  2. Client client = elasticsearchTemplate.getClient();
  3. SearchRequestBuilder srb = client
  4. .prepareSearch("channel_weibo_follow")
  5. .setTypes("doc");
  6. SearchResponse sr = srb.setQuery(QueryBuilders.matchAllQuery()).execute().actionGet();
  7. SearchHits hits = sr.getHits();
  8. List<Map<String, Object>> list = new ArrayList<Map<String, Object>>();
  9. for (SearchHit hit : hits) {
  10. Map<String, Object> source = hit.getSourceAsMap();
  11. list.add(source);
  12. System.out.println(hit.getSourceAsString());
  13. }
  14. return list;
  15. }

 返回的数据:

  1. {
  2. "code": 200,
  3. "message": "ok",
  4. "data": [
  5. {
  6. "uid": 3800468188,
  7. "fansUid": 1730077315,
  8. "count": 2,
  9. "taskId": 1
  10. },
  11. {
  12. "uid": 2634488643,
  13. "fansUid": 1730077315,
  14. "count": 2,
  15. "taskId": 1
  16. },
  17. {
  18. "uid": 2003394681,
  19. "fansUid": 1730077315,
  20. "count": 2,
  21. "taskId": 1
  22. },
  23. {
  24. "uid": 1782432341,
  25. "fansUid": 1730077315,
  26. "count": 2,
  27. "taskId": 1
  28. },
  29. {
  30. "uid": 1821525001,
  31. "fansUid": 1730077315,
  32. "count": 2,
  33. "taskId": 1
  34. },
  35. {
  36. "uid": 1709157165,
  37. "fansUid": 1730077315,
  38. "count": 2,
  39. "taskId": 1
  40. }
  41. ]
  42. }

elasticsearchTemplate删除前缀为channel_channelId索引下taskId为xx的所有数据

  1. DeleteQuery deleteQuery=new DeleteQuery();
  2. BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
  3. // // 拼接查询条件
  4. queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
  5. // 查询出所有的索引
  6. IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
  7. indicesStatsRequest.indices("channel_"+channelId+"*");
  8. Map<String, IndexStats> stats=elasticsearchTemplate.getClient().admin().indices().stats(indicesStatsRequest).actionGet().getIndices();
  9. // 根据索引遍历删除
  10. for(IndexStats stat:stats.values()) {
  11. deleteQuery.setIndex(stat.getIndex());
  12. deleteQuery.setQuery(queryBuilder);
  13. deleteQuery.setType("doc");
  14. elasticsearchTemplate.delete(deleteQuery);
  15. }

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/花生_TL007/article/detail/68814
推荐阅读
相关标签
  

闽ICP备14008679号