赞
踩
通过elasticsearch实现创建索引、创建mapping映射、插入数据、模糊查询索引、查询index所对应的mapping、数据的滚动条件查询、删除指定数据
目录
elasticsearchTemplate创建索引、创建mapping映射(默认创建类型为doc)
模糊查询index(查询以xxx开头的所有索引)并查询其对应的mapping包含的字段
elasticsearchTemplate查询指定索引类型下的所有数据
elasticsearchTemplate删除前缀为channel_channelId索引下taskId为xx的所有数据
elasticsearchTemplate创建索引、创建mapping映射(默认创建类型为doc)
- // mapping对象转为json字符串
- String s = objectMapper.writeValueAsString(channelOutput.getMappings());
- // 拼接索引名称index
- String index = "channel_" + channelAdd.getChannelId() + "_" + channelOutput.getType();
- // 判断索引是否已存在
- boolean b = elasticsearchTemplate.indexExists(index);
- // es索引不存在創建索引,mapping
- if (!b){
- elasticsearchTemplate.createIndex(index);
- elasticsearchTemplate.putMapping(index, "doc", s);
- }
- List<IndexQuery> list=new ArrayList<>();
- IndexQuery indexQuery = new IndexQuery();
- // 设置索引名称 indexQuery.setIndexName("channel_"+taskDataAmount.getChannelId()+"_"+taskDataAmount.getType());
- // 设置索引下的类型
- indexQuery.setType("doc");
- // 需要添加的数据 indexQuery.setSource(objectMapper.writeValueAsString(taskDataAmount.getData()));
- list.add(indexQuery);
- elasticsearchTemplate.bulkIndex(list);
特别说明:添加的数据结构和所设置的mapping结构不一样也可以添加成功
例如:mapping如下
- {
- "task_log": {
- "mappings": {
- "data": {
- "properties": {
- "className": {
- "type": "keyword"
- },
- "dataTypeId": {
- "type": "keyword"
- },
- "engineId": {
- "type": "keyword"
- },
- "engineName": {
- "type": "keyword"
- },
- "error": {
- "type": "keyword"
- },
- "extendsInfo": {
- "type": "keyword"
- },
- "method": {
- "type": "keyword"
- },
- "taskId": {
- "type": "keyword"
- }
- }
- }
- }
- }
- }
添加数据时,除了className这些字段还可以添加一些mapping中没有的字段
- // 封装类
- @Data
- public class FieldVo {
- private String indexName;
-
- private Set<String> fieldNames;
- }
- List<FieldVo> fieldVos=new ArrayList<>();
- Client client = elasticsearchTemplate.getClient();
- IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
- // 前缀+*
- indicesStatsRequest.indices("channel_"+taskLogs.getChannelId()+"*");
-
- Map<String,IndexStats>stats=client.admin().indices().stats(indicesStatsRequest).actionGet().getIndices();
-
- for(IndexStats stat:stats.values()) {
- FieldVo fieldVo=new FieldVo();
- String in = stat.getIndex();//获取索引值
- fieldVo.setIndexName(in);
- //获取制定index下的mapping
- Map<String,String> o = (Map<String, String>) elasticsearchTemplate.getMapping(in, "doc").get("properties");
- //获取所有的字段
- Set<String> fields = o.keySet();
- fieldVo.setFieldNames(fields);
以上两张图分别是Map stats和Map o的数据结构
- public ResultModel getDataDetail(String index,String scrollId,Long taskLogId,Integer pageSize){
- if (StringUtils.isEmpty(index)){
- return ResultModel.fail("索引为空");
- }
- if (null==taskLogId){
- return ResultModel.fail("taskLogId不能为空");
- }
- if (null==pageSize){
- pageSize=20;
- }
- TaskLogs taskLogs = taskLogsMapper.selectByPrimaryKey(taskLogId);
- // 创建一个查询条件对象
- BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
- // 拼接查询条件
- queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
- queryBuilder.must(QueryBuilders.termQuery("count",taskLogs.getCount()));
- SearchQuery searchQuery = new NativeSearchQueryBuilder()
- .withIndices(index)//索引名
- .withTypes("doc")//类型名
- .withPageable(PageRequest.of(0, pageSize))//从0页开始查,每页1000个结果
- .withQuery(queryBuilder)
- .build();
- Page<Map> maps =null;
- if (StringUtils.isEmpty(scrollId)){
- maps=elasticsearchTemplate.startScroll(300000L, searchQuery, Map.class);
- }else {
- maps =elasticsearchTemplate.continueScroll(scrollId, 300000L, Map.class);
-
- }
- return ResultModel.success(maps);
- }
第一次查询时设置的pageSize,后续的查询中不能进行修改,因为是用的一个query
300000L是设置的过期时间
第一次请求是会返回一个scrollId,后续往下查询都需要把这个作为参数去请求
请求结果示例:
- {
- "code": 200,
- "message": "ok",
- "data": {
- "content": [
- {
- "id": "324",
- "name": "testONe",
- "username": "test",
- "count": 3,
- "taskId": 10
- }
- ],
- "pageable": "INSTANCE",
- "facets": [],
- "aggregations": null,
- "scrollId": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAyiXFkdkbm91M0hYUU9xSG13R3A0UHdoWkEAAAAAAAMolhZHZG5vdTNIWFFPcUhtd0dwNFB3aFpBAAAAAAAG2RcWdDJxTUx6QllUbmVmbjM2b0UzSmlFdwAAAAAABtgcFkpvMlZ0X04yVGVxdFE2ZWNOWEZJOWcAAAAAAAbYHRZKbzJWdF9OMlRlcXRRNmVjTlhGSTln",
- "totalElements": 1,
- "totalPages": 1,
- "number": 0,
- "size": 0,
- "sort": {
- "sorted": false,
- "unsorted": true
- },
- "numberOfElements": 1,
- "first": true,
- "last": true
- }
- }
- public List<Map<String, Object>> test(){
- Client client = elasticsearchTemplate.getClient();
- SearchRequestBuilder srb = client
- .prepareSearch("channel_weibo_follow")
- .setTypes("doc");
-
- SearchResponse sr = srb.setQuery(QueryBuilders.matchAllQuery()).execute().actionGet();
-
- SearchHits hits = sr.getHits();
- List<Map<String, Object>> list = new ArrayList<Map<String, Object>>();
- for (SearchHit hit : hits) {
- Map<String, Object> source = hit.getSourceAsMap();
- list.add(source);
- System.out.println(hit.getSourceAsString());
- }
- return list;
- }
返回的数据:
- {
- "code": 200,
- "message": "ok",
- "data": [
- {
- "uid": 3800468188,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- },
- {
- "uid": 2634488643,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- },
- {
- "uid": 2003394681,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- },
- {
- "uid": 1782432341,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- },
- {
- "uid": 1821525001,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- },
- {
- "uid": 1709157165,
- "fansUid": 1730077315,
- "count": 2,
- "taskId": 1
- }
- ]
- }
- DeleteQuery deleteQuery=new DeleteQuery();
- BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
- // // 拼接查询条件
- queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
- // 查询出所有的索引
- IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
- indicesStatsRequest.indices("channel_"+channelId+"*");
- Map<String, IndexStats> stats=elasticsearchTemplate.getClient().admin().indices().stats(indicesStatsRequest).actionGet().getIndices();
- // 根据索引遍历删除
- for(IndexStats stat:stats.values()) {
- deleteQuery.setIndex(stat.getIndex());
- deleteQuery.setQuery(queryBuilder);
- deleteQuery.setType("doc");
- elasticsearchTemplate.delete(deleteQuery);
- }
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。