devbox
小丑西瓜9
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

django集成es(django-elasticsearch-dsl-drf)_django-elasticsearch-dsl详解

作者：小丑西瓜9 | 2024-03-07 12:16:38

赞

踩

django-elasticsearch-dsl详解

前提介绍

在此版本下,es自动与mysql或其他数据库,进行数据同步(修改,删除.新增),update并不会自动同步

1. 准备模拟数据，安装json-server(不安装也可以.本文没有安装)

json-server的安装https://www.jianshu.com/p/920d73fc32de

  1. # 本文借鉴
  2. https://www.jianshu.com/p/920d73fc32de
  3. # 安装(windows上安装)，以管理员身份打开cmd
  4. npm  install  -g  json-server
  5. # 测试安装成功
  6. json-server -h
  7. # 在D盘新建一个文件夹JsonServer
  8. # 进入JsonServer文件夹，初始化
  9. npm init
  10. # 在JsonServer文件夹里面新建db.json,用于接口数据
  11. {
  12.   "news": [
  13.     {
  14.        "id":1,
  15.        "title":"Tesla's Autopilot, other driver assists are in a regulatory grey zone",
  16.        "content":"Filed under: Government/Legal,Tesla,Safety,Technology Continue reading Tesla's Autopilot, other driver assists are in a regulatory grey zone Tesla's Autopilot, other driver assists are in a regulatory grey zone originally appeared on Autoblog on Sat, 24 Apr"
  17.     },
  18.     {
  19.        "id":2,
  20.        "title":"‘A Bunch of People Will Probably Die’ at Onset of Mars Colonisation, Musk Cautions",
  21.        "content":"The Silicone Valley mogul has candidly outlined the pitfalls of potential trips to the red planet, as his brainchild SpaceX has been going to great lengths to make commercial tourism on Mars possible one day."
  22.     },
  23.     {
  24.        "id":3,
  25.        "title":"IRS says to do this one thing to get the biggest possible tax refund",
  26.        "content":"In a normal year, one in which the country wasn't dealing with the catastrophic public health and financial impacts of a global pandemic, tax season would..."
  27.     }
  28.   ]
  29. }
  30. # 安装依赖模块，安装完成会发现多了两个文件：
  31. npm install json-server --save
  32. # 修改配置文件
  33. 打开package.json文件，将scripts进行修改，如下:
  34.     "scripts": {
  35.     "json:server": "json-server --watch db.json"
  36. },
  37. # 完成之后，输入命令，运行：
  38. npm run json:server
  39. # 访问 
  40. http://localhost:3000/news

2.django集成es

需要环境(文章中有requirements，无需担心)

django-elasticsearch-dsl==7.2.0
django-elasticsearch-dsl-drf==0.22

  1. python3.6.8
  2. django==3.2
  3. djangorestframework==3.12.4
  4. elasticsearch-dsl==7.3.0
  5. django-elasticsearch-dsl==7.2.0
  6. django-elasticsearch-dsl-drf==0.22

环境安装

  1. # 创建一个虚拟环境
  2.  
  3. # 安装Djangg3.2
  4. pip3 install django==3.2 -i https://pypi.douban.com/simple/
  5.     
  6. # 创建一个app news
  7.  
  8. # 安装依赖环境
  9. pip3 install -r requirements.txt -i https://pypi.douban.com/simple

requirements

  1. asgiref==3.3.4
  2. certifi==2020.12.5
  3. chardet==4.0.0
  4. Django==3.2
  5. django-cors-headers==3.7.0
  6. django-debug-toolbar==3.2.1
  7. django-discover-runner==1.0
  8. django-elasticsearch-dsl==7.2.0
  9. django-elasticsearch-dsl-drf==0.22
  10. django-froala-editor==3.2.2
  11. django-ipware==3.0.2
  12. django-nine==0.2.4
  13. django-redis==4.12.1
  14. django-rest-elasticsearch==0.4.2
  15. django-role-permissions==3.1.1
  16. django-webpack-loader==0.7.0
  17. djangorestframework==3.12.4
  18. elasticsearch==7.12.0
  19. elasticsearch-dsl==7.3.0
  20. Faker==8.1.0
  21. idna==2.10
  22. ipaddress==1.0.23
  23. Pillow==8.2.0
  24. python-dateutil==2.8.1
  25. pytz==2021.1
  26. redis==3.5.3
  27. requests==2.25.1
  28. six==1.15.0
  29. sqlparse==0.4.1
  30. text-unidecode==1.3
  31. typing-extensions==3.7.4.3
  32. urllib3==1.26.4

# settings.py进行配置

  1. INSTALLED_APPS = [
  2.     'django.contrib.admin',
  3.     ...
  4.     'rest_framework',
  5.     'django_elasticsearch_dsl',
  6.     'django_elasticsearch_dsl_drf',    
  7. ]
  8.  
  9. ELASTICSEARCH_DSL = {
  10.     'default': {
  11.         'hosts': 'localhost:9200'
  12.     },
  13. }

# 在 news app 下的 models.py：

  1. from django.db import models
  2.  
  3. # Create your models here.
  4. class ElasticNews(models.Model):
  5.     title = models.CharField(max_length=100)
  6.     content = models.TextField()

# 在 news app 的 admin.py 中进行如下的修改

本文用法

  1. from django.contrib import admin
  2.  
  3. # Register your models here.
  4. from news.models import *
  5.  
  6.  
  7. admin.site.register(ElasticNews)

admin的第二种用法 

    list_display = []  显示跟添加字段无关
    list_filter = []  
    search_fields = [] 
    readonly_fields = []
    filter_horizontal = []

  1. from django.contrib import admin
  2.  
  3. from .models import *
  4.  
  5.  
  6. @admin.register(Book)
  7. class BookAdmin(admin.ModelAdmin):
  8.     """Book admin."""
  9.  
  10.     list_display = ('title', 'price', 'publication_date')
  11.     search_fields = ('title',)
  12.     # 多选字段
  13.     filter_horizontal = ('authors', 'tags',)
  14.  
  15.  
  16. @admin.register(Author)
  17. class AuthorAdmin(admin.ModelAdmin):
  18.     """Author admin."""
  19.  
  20.     list_display = ('name', 'email',)
  21.     search_fields = ('name',)
  22.  
  23.  
  24. @admin.register(Publisher)
  25. class PublisherAdmin(admin.ModelAdmin):
  26.     """Publisher admin."""
  27.  
  28.     list_display = ('name',)
  29.     search_fields = ('name',)
  30.  
  31.  
  32. @admin.register(Tag)
  33. class TagAdmin(admin.ModelAdmin):
  34.     """Tag admin."""
  35.  
  36.     list_display = ('title',)
  37.     search_fields = ('title',)

# news app 下创建一个叫做 documents.py 的文件：

  1. from django_elasticsearch_dsl import (
  2.     Document,
  3.     fields,
  4.     Index,
  5. )
  6.  
  7. from .models import ElasticNews
  8.  
  9. # 创建索引
  10. PUBLISHER_INDEX = Index('elastic_news')
  11.  
  12. # 索引配置
  13. PUBLISHER_INDEX.settings(
  14.     number_of_shards=1,
  15.     number_of_replicas=0
  16. )
  17.  
  18.  
  19. # 创建映射及字段类型
  20. @PUBLISHER_INDEX.doc_type
  21. class NewsDocument(Document):
  22.     id = fields.IntegerField(attr='id')
  23.     fielddata = True
  24.     title = fields.TextField(
  25.         fields={
  26.             'keyword': {
  27.                 'type': 'keyword',
  28.             }
  29.  
  30.         }
  31.     )
  32.     content = fields.TextField(
  33.         fields={
  34.             'keyword': {
  35.                 'type': 'keyword',
  36.  
  37.             }
  38.         },
  39.     )
  40.  
  41.     # 配置模型
  42.     class Django(object):
  43.         model = ElasticNews

# News app 下创建一个叫做 serializsers.py 的文件：

  1. import json
  2. from .models import ElasticNews
  3.  
  4. from django_elasticsearch_dsl_drf.serializers import DocumentSerializer
  5. from .documents import *
  6. from rest_framework import serializers
  7.  
  8.  
  9. # 创建序列化器
  10. class NewsDocumentSerializer(DocumentSerializer):
  11.     # 同样支持重写字段
  12.     title = serializers.SerializerMethodField()
  13.  
  14.     class Meta(object):
  15.         """Meta options."""
  16.         # 模型
  17.         model = ElasticNews
  18.         # 索引
  19.         document = NewsDocument
  20.         # 控制序列化的字段
  21.         fields = ('id', 'title', 'content')
  22.         # fields = '__all__'
  23.  
  24.     # 自定义返回数据
  25.     def get_title(self, obj):
  26.         # print('dddddddddddddd', obj.price)
  27.         return obj.price

# 在 News app 下的 views.py 文件，以便能在访问页面时生产相应的 Elasticsearch 文档

  1. from django.http import JsonResponse
  2. import requests
  3. import json
  4. from news.models import *
  5.  
  6. from .documents import *
  7. from .serializers import *
  8.  
  9. from django_elasticsearch_dsl_drf.filter_backends import (
  10.     FilteringFilterBackend,
  11.     CompoundSearchFilterBackend
  12. )
  13. from django_elasticsearch_dsl_drf.viewsets import DocumentViewSet
  14. from django_elasticsearch_dsl_drf.filter_backends import (
  15.     FilteringFilterBackend,
  16.     OrderingFilterBackend,
  17. )
  18.  
  19.  
  20. # 写入es的函数
  21. def generate_random_data():
  22.     # 安装JsonServer,
  23.     """
  24.     url = 'http://localhost:3000/news'
  25.     r = requests.get(url)
  26.     payload = json.loads(r.text)
  27.     count = 1
  29.     print("---------------------payload--------------", payload)
  31.     print("type of payload is: ", type(payload))
  32.     for data in payload:
  33.         # print("title: ", data['title'])
  34.         # print("content: ", data['content'])
  35.         ElasticNews.objects.create(
  36.             title=data['title'],
  37.             content=data['content']
  38.         )
  39.     """
  40.     # 不安装JsonServer,自己构建数据
  41.     payload = [
  42.         {'title': '洞c内', 'content': 'casac', 'price': '666'},
  43.         {'title': '传s说', 'content': 'cas', 'price': '99'},
  44.         {'title': '啊不错', 'content': 'casa', 'price': '966'}
  45.     ]
  46.     for data in payload:
  47.         ElasticNews.objects.create(
  48.             **data
  49.         )
  50.  
  51.  
  52. # 写入es
  53. def index(request):
  54.     # es与mysql自动同步数据
  55.     generate_random_data()
  56.     return JsonResponse({'status': 200})
  57.     # return HttpResponse("Hello, the world")
  58.  
  59.  
  60. # 用来读数据进行搜索并排序的一个 view
  61. class PublisherDocumentView(DocumentViewSet):
  62.     # mapping
  63.     document = NewsDocument
  64.     # 序列化器
  65.     serializer_class = NewsDocumentSerializer
  66.     lookup_field = 'id'
  67.     fielddata = True
  68.     filter_backends = [
  69.         FilteringFilterBackend,
  70.         OrderingFilterBackend,
  71.         CompoundSearchFilterBackend,
  72.     ]
  73.     # 在全文中搜索范围的字段配置(模糊匹配)
  74.     # Define filtering fields
  75.     filter_fields = {
  76.         'title': 'title',
  77.         'content': 'content',
  78.         # 'id': 'id',
  79.     }
  80.     # Define search fields
  81.     search_fields = (
  82.         # 'id',
  83.         'title',
  84.         'content',
  85.     )
  86.     # Define ordering fields
  87.     ordering_fields = {
  88.         'id': None,
  89.     }
  90.     # 指定默认顺序
  91.     ordering = ('id', 'content')
  92.  
  93.     # 支持多个参数
  94.     multi_match_search_fields = (
  95.         'title',
  96.         'content',
  97.     )
  98.  
  99.  
  100. # 测试
  101. def update_data(request):
  102.     # 群更新会触发
  103.     # query = ElasticNews.objects.all()
  104.     # for item in query:
  105.     #     item.title = 'uuuuuuuuuuu'
  106.     #     item.save()
  107.  
  108.     # 单更新会触发
  109.     # obj = ElasticNews.objects.filter(id=3).first()
  110.     # obj.title = '刘亚萍99999999999'
  111.     # obj.save()
  112.  
  113.     # create会触发
  114.     # ElasticNews.objects.create(title='cas', content='222', price='85')
  115.     # ElasticNews.objects.create(title='多对多', content='cas', price='85')
  116.     ElasticNews.objects.create(title='cas', content='csa', price='85')
  117.     # ElasticNews.objects.create(title='cas2', content='lcas', price='85')
  118.  
  119.     # 删除
  120.     # ElasticNews.objects.filter(id=4).delete()
  121.  
  122.     # update不会触发es与mysql同步,也不会触发信号post_save
  123.     # ElasticNews.objects.filter(id=1).update(title='圆梦去', content='222', price='85')
  124.     return JsonResponse({'status': 200})

# 修改 ulrpatterns

  1. from django.contrib import admin
  2. from django.urls import path
  3. from news.views import *
  4.  
  5. urlpatterns = [
  6.     path('admin/', admin.site.urls),
  7.     path('search/', PublisherDocumentView.as_view({'get': 'list'})),
  8.     path('add/', index),
  9.     path('update/', update_data),
  10. ]

# 我们使用如下的命令来创建 Elasticsearch 索引 mapping：

  1. python manage.py makemigrations
  2. python manage.py migrate
  3. # 执行这一句,如果存在索引,就删除重新构建,不执行也可以,保证当前配置的索引唯一即可
  4. python manage.py search_index --rebuild

 # 访问 

  1. http://127.0.0.1:8000/
  2. http://127.0.0.1:8000/search/

查询使用 

  1.     class Meta:
  2.         """Meta options."""
  3.         # 即模型对象返回的记录结果集是按照这个字段排序的。
  4.         ordering = ["id"]
  5.         
  6. # 使用过滤排序        
  7. # 所有字段中搜索cas
  8. http://127.0.0.1:8000/search/?search=cas
  9. # 要在特定字段 中搜索术“cas”
  10. # 例子:要在字段title中搜索“cas”
  11. http://127.0.0.1:8000/search/?search=title:cas
  12. # 搜索多个术语  二者或的关系
  13. http://127.0.0.1:8000/search/?search=cas&search=洞c内
  14. # 要在特定字段 搜索多个术语  二者或的关系
  15. http://127.0.0.1:8000/search/?search=title:cas&search=content:卡车
  16. # 按单个字段筛选文档
  17. http://127.0.0.1:8000/search/?title=cas
  18. # 或搜索(不支持中文)
  19. http://127.0.0.1:8000/search/?title__in=cas__cas2
  20. # and(不支持中文)
  21. http://127.0.0.1:8000/search/?title=cas&content=csa
  22. # 按单个字段的单词部分筛选文档(正则搜索)
  23. http://127.0.0.1:8000/search/?title__wildcard=*内
  24. # 按字段对文档进行排序（升序
  25. http://127.0.0.1:8000/search/?search=content:cas&ordering=id
  26. # 按字段对文档进行排序（降序）
  27. http://127.0.0.1:8000/search/?search=content:cas&ordering=-id
  28. # 按多个字段对文档进行排序
  29. 如果要按多个字段排序，请使用多个排序查询参数。在下面的示例中，文档将首先按字段排序（降序），然后按字段排序（升序）
  30. http://127.0.0.1:8080/search/publisher/?ordering=-country&ordering=city

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小丑西瓜9/article/detail/205581
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号