devbox
IT小白
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

刘焕勇QABasedOnMedicaKnowledgeGraph项目全过程

作者：IT小白 | 2024-05-03 00:12:40

赞

踩

刘焕勇

原项目地址：新建标签页 (github.com)

实体规模4.4万，实体关系规模30万。

一、首先安装

MongoDB-Windows-x86_64

mongodb-compass-1.36.1-win32-x64(安装的时候注意安装路径最好不要出现中文，不然会很麻烦)

neo4j-community-4.2.4-windows(这里我是去找了4开头的历史版本）

jdk-11.0.18_windows-x64_bin

开始项目时MongoDB、neo4j全程打开

  1. 连接MongoDB

(1)打开cmd，进入E:\mongodb\bin目录下，输入命令“mongod --dbpath E:\mongodb\data”即可开启MongoDB服务。

(2)浏览器进入http://127.0.0.1:27017，显示“It looks like you are trying to access MongoDB over HTTP on the native driver port.”则表示连接成功。

  1. 连接neo4j

(1)进入E:\知识图谱\neo4j\neo4j-community-4.2.4\bin，输入neo4j.bat console。

(2)浏览器进入http://localhost:7474/browser/。

二、数据处理部分

1.data_spider.py 爬取数据（数据的原网站是寻医问药网的疾病百科）

  1. import urllib.request
  2. import urllib.parse
  3. from lxml import etree
  4. import pymongo
  5. import re
  6.  
  7. class CrimeSpider:
  8.     def __init__(self):
  9.         self.conn = pymongo.MongoClient()#创建数据库连接
  10.         self.db = self.conn['medical']#获取数据库‘medical'
  11.         self.col = self.db['data']#获取文件'data'
  12.  
  13.     #根据url，请求html
  14.     def get_html(self, url):
  15.         headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) '
  16.                                  'Chrome/51.0.2704.63 Safari/537.36'}#UA使得服务器能够识别客户使用的操作系统及版本、CPU类型、浏览器及版本等，这里是要程序伪装成浏览器进行访问
  17.         req = urllib.request.Request(url=url, headers=headers)#发起带header头的请求
  18.         res = urllib.request.urlopen(req)#返回一个object对象
  19.         html = res.read().decode('gbk')#想要获取内容还需要进一步的解析
  20.         return html
  21.  
  22.  
  23.     def url_parser(self, content):
  24.         selector = etree.HTML(content)#将content转化为能被xpath匹配的格式
  25.         urls = ['http://jib.xywy.com' + i for i in  selector.xpath('//h2[@class="item-title"]/a/@href')]#抓取所有病例的页面
  26.         return urls
  27.  
  28.     #将内容写入数据库
  29.     def spider_main(self):
  30.         for page in range(1, 11000):
  31.             try:
  32.                 basic_url = 'http://jib.xywy.com/il_sii/gaishu/%s.htm'%page#疾病概述页面
  33.                 cause_url = 'http://jib.xywy.com/il_sii/cause/%s.htm'%page#病因页面
  34.                 prevent_url = 'http://jib.xywy.com/il_sii/prevent/%s.htm'%page#预防页面
  35.                 symptom_url = 'http://jib.xywy.com/il_sii/symptom/%s.htm'%page#症状页面
  36.                 inspect_url = 'http://jib.xywy.com/il_sii/inspect/%s.htm'%page#检查方法页面
  37.                 treat_url = 'http://jib.xywy.com/il_sii/treat/%s.htm'%page#治疗页面
  38.                 food_url = 'http://jib.xywy.com/il_sii/food/%s.htm'%page#食物页面
  39.                 drug_url = 'http://jib.xywy.com/il_sii/drug/%s.htm'%page#药品页面
  40.                 data = {}#将以下数据封装成字典
  41.                 data['url'] = basic_url#基本网址
  42.                 data['basic_info'] = self.basicinfo_spider(basic_url)#疾病基本信息。字典。'basic_info':{'category':疾病分类,'name':疾病名称,'desc':疾病简介,'attributes':[基本知识,治疗常识,温馨提示]}
  43.                 data['cause_info'] =  self.common_spider(cause_url)#病因。list。'cause_info':[第一段文字,第二段文字,...]
  44.                 data['prevent_info'] =  self.common_spider(prevent_url)#预防。list。'prevent_info':[第一段文字,第二段文字,...]
  45.                 data['symptom_info'] = self.symptom_spider(symptom_url)#症状(应该是并发症)。元组。'symptom_info'：（症状信息，[第一段症状，第二段症状，...]）
  46.                 data['inspect_info'] = self.inspect_spider(inspect_url)#检查方法的网址。list。'inspect_info'：[检查方法网址,.....]
  47.                 data['treat_info'] = self.treat_spider(treat_url)#治疗概述。list。 'treat_info':[就诊科室,治疗方式,治疗周期,治愈率,常用药品]
  48.                 data['food_info'] = self.food_spider(food_url)#食物,字典。'food_info':{'good':宜吃食物, 'bad':忌食物, 'recommand':宜食物推荐食物}
  49.                 data['drug_info'] = self.drug_spider(drug_url)#药品名称。str。'drug_info':药品名称
  50.                 print(page, basic_url)#打印出
  51.                 self.col.insert(data)#将上述数据封装成的字典,写入数据库
  52.  
  53.             except Exception as e:
  54.                 print(e, page)
  55.         return
  56.  
  57.     #基本信息解析
  58.     def basicinfo_spider(self, url):
  59.         html = self.get_html(url)
  60.         selector = etree.HTML(html)
  61.         title = selector.xpath('//title/text()')[0]#网页的标题，使用正则表达式提取内容中的某部分
  62.         category = selector.xpath('//div[@class="wrap mt10 nav-bar"]/a/text()')#分类
  63.         desc = selector.xpath('//div[@class="jib-articl-con jib-lh-articl"]/p/text()')#简介
  64.         ps = selector.xpath('//div[@class="mt20 articl-know"]/p')#[基本知识,治疗常识,温馨提示]
  65.         infobox = []
  66.         for p in ps:
  67.             info = p.xpath('string(.)').replace('\r','').replace('\n','').replace('\xa0', '').replace('   ', '').replace('\t','')#清理换行符等内容
  68.             infobox.append(info)
  69.         basic_data = {}
  70.         basic_data['category'] = category
  71.         basic_data['name'] = title.split('的简介')[0]
  72.         basic_data['desc'] = desc
  73.         basic_data['attributes'] = infobox
  74.         return basic_data
  75.  
  76.     #treat_infobox治疗解析
  77.     def treat_spider(self, url):
  78.         html = self.get_html(url)
  79.         selector = etree.HTML(html)
  80.         ps = selector.xpath('//div[starts-with(@class,"mt20 articl-know")]/p')
  81.         infobox = []
  82.         for p in ps:
  83.             info = p.xpath('string(.)').replace('\r','').replace('\n','').replace('\xa0', '').replace('   ', '').replace('\t','')
  84.             infobox.append(info)
  85.         return infobox
  86.  
  87.     #treat_infobox治疗解析
  88.     def drug_spider(self, url):
  89.         html = self.get_html(url)
  90.         selector = etree.HTML(html)
  91.         drugs = [i.replace('\n','').replace('\t', '').replace(' ','') for i in selector.xpath('//div[@class="fl drug-pic-rec mr30"]/p/a/text()')]
  92.         return drugs
  93.  
  94.     #food治疗解析
  95.     def food_spider(self, url):
  96.         html = self.get_html(url)
  97.         selector = etree.HTML(html)
  98.         divs = selector.xpath('//div[@class="diet-img clearfix mt20"]')
  99.         try:#异常处理
  100.             food_data = {}
  101.             food_data['good'] = divs[0].xpath('./div/p/text()')
  102.             food_data['bad'] = divs[1].xpath('./div/p/text()')
  103.             food_data['recommand'] = divs[2].xpath('./div/p/text()')
  104.         except:
  105.             return {}
  106.  
  107.         return food_data
  108.  
  109.     #症状信息解析
  110.     def symptom_spider(self, url):
  111.         html = self.get_html(url)
  112.         selector = etree.HTML(html)
  113.         symptoms = selector.xpath('//a[@class="gre" ]/text()')
  114.         ps = selector.xpath('//p')
  115.         detail = []
  116.         for p in ps:
  117.             info = p.xpath('string(.)').replace('\r','').replace('\n','').replace('\xa0', '').replace('   ', '').replace('\t','')
  118.             detail.append(info)
  119.         symptoms_data = {}
  120.         symptoms_data['symptoms'] = symptoms
  121.         symptoms_data['symptoms_detail'] = detail
  122.         return symptoms, detail
  123.  
  124.     #检查信息解析
  125.     def inspect_spider(self, url):
  126.         html = self.get_html(url)
  127.         selector = etree.HTML(html)
  128.         inspects = selector.xpath('//li[@class="check-item"]/a/@href')
  129.         return inspects
  130.  
  131.     #通用解析模块
  132.     def common_spider(self, url):
  133.         html = self.get_html(url)
  134.         selector = etree.HTML(html)
  135.         ps = selector.xpath('//p')
  136.         infobox = []
  137.         for p in ps:
  138.             info = p.xpath('string(.)').replace('\r', '').replace('\n', '').replace('\xa0', '').replace('   ','').replace('\t', '')
  139.             if info:
  140.                 infobox.append(info)
  141.         return '\n'.join(infobox)
  142.  
  143.     #检查项抓取模块
  144.     def inspect_crawl(self):
  145.         for page in range(1, 3685):
  146.             try:
  147.                 url = 'http://jck.xywy.com/jc_%s.html'%page
  148.                 html = self.get_html(url)
  149.                 data = {}#检查的字典数据
  150.                 data['url']= url#检查项目网页的网址
  151.                 data['html'] = html#检查项目网页的源码
  152.                 self.db['jc'].insert(data)#放到jc这个数据库里
  153.                 print(url)
  154.             except Exception as e:
  155.                 print(e)
  156.  
  157.  
  158. handler = CrimeSpider()
  159. handler.inspect_crawl()
  160. handler.spider_main()

2.max_cut.py 基于词典的最大前向/后向匹配。

  1. class CutWords:
  2.     def __init__(self):
  3.         dict_path = './disease.txt'#加载保存疾病名称的文档
  4.         self.word_dict, self.max_wordlen = self.load_words(dict_path)
  5.  
  6.     #加载词典
  7.     def load_words(self, dict_path):
  8.         words = list()#list()方法用于将元组或字符串转换为列表，所以这里是想要构建一个空列表。然后定义最大字长变量，在字典遍历的过程中，不断将疾病名称保存到words中，同时更新最大字长。
  9.         max_len = 0
  10.         for line in open(dict_path):
  11.             wd = line.strip()
  12.             if not wd:
  13.                 continue
  14.             if len(wd) > max_len:
  15.                 max_len = len(wd)
  16.             words.append(wd)
  17.         return words, max_len
  18.  
  19.     #最大向前匹配
  20.     def max_forward_cut(self, sent):
  21.         # 1.从左向右取待切分汉语句的m个字符作为匹配字段，m为大机器词典中最长词条个数。
  22.         # 2.查找大机器词典并进行匹配。若匹配成功，则将这个匹配字段作为一个词切分出来。如果没有则字长减一继续匹配，直到匹配到为止，如果到最后也没有匹配成功的话，就返回该字段第一个值。
  23.         cutlist = []
  24.         index = 0
  25.         while index < len(sent):
  26.             matched = False
  27.             for i in range(self.max_wordlen, 0, -1):
  28.                 cand_word = sent[index: index + i]
  29.                 if cand_word in self.word_dict:
  30.                     cutlist.append(cand_word)
  31.                     matched = True
  32.                     break
  33.  
  34.             # 如果没有匹配上，则按字符切分
  35.             if not matched:
  36.                 i = 1
  37.                 cutlist.append(sent[index])
  38.             index += i
  39.         return cutlist
  40.  
  41.     # 最大向后匹配
  42.     def max_backward_cut(self, sent):
  43.         # 1.从右向左取待切分汉语句的m个字符作为匹配字段，m为大机器词典中最长词条个数。
  44.         # 2.查找大机器词典并进行匹配。若匹配成功，则将这个匹配字段作为一个词切分出来。和前面的最大向前匹配类似，只不过在该函数中如果最后匹配失败则返回字段最后一个值。
  45.         cutlist = []
  46.         index = len(sent)
  47.         max_wordlen = 5
  48.         while index > 0:
  49.             matched = False
  50.             for i in range(self.max_wordlen, 0, -1):
  51.                 tmp = (i + 1)
  52.                 cand_word = sent[index - tmp: index]
  53.                 # 如果匹配上，则将字典中的字符加入到切分字符中
  54.                 if cand_word in self.word_dict:
  55.                     cutlist.append(cand_word)
  56.                     matched = True
  57.                     break
  58.             # 如果没有匹配上，则按字符切分
  59.             if not matched:
  60.                 tmp = 1
  61.                 cutlist.append(sent[index - 1])
  62.  
  63.             index -= tmp
  64.  
  65.         return cutlist[::-1]
  66.  
  67.     # 双向最大向前匹配
  68.     def max_biward_cut(self, sent):
  69.         # 双向最大匹配法是将正向最大匹配法得到的分词结果和逆向最大匹配法的到的结果进行比较，从而决定正确的分词方法。
  70.         # 启发式规则：
  71.         # 1.如果正反向分词结果词数不同，则取分词数量较少的那个。
  72.         # 2.如果分词结果词数相同 a.分词结果相同，就说明没有歧义，可返回任意一个。 b.分词结果不同，返回其中单字较少的那个。
  73.         forward_cutlist = self.max_forward_cut(sent)
  74.         backward_cutlist = self.max_backward_cut(sent)
  75.         count_forward = len(forward_cutlist)
  76.         count_backward = len(backward_cutlist)
  77.  
  78.         def compute_single(word_list):
  79.             num = 0
  80.             for word in word_list:
  81.                 if len(word) == 1:
  82.                     num += 1
  83.             return num
  84.  
  85.         if count_forward == count_backward:
  86.             if compute_single(forward_cutlist) > compute_single(backward_cutlist):
  87.                 return backward_cutlist
  88.             else:
  89.                 return forward_cutlist
  90.  
  91.         elif count_backward > count_forward:
  92.             return forward_cutlist
  93.  
  94.         else:
  95.             return backward_cutlist
  96.

3.build_data.py 将爬虫爬取的数据进行规整

  1. import pymongo
  2. from lxml import etree
  3. import os
  4. from max_cut import *
  5.  
  6. class MedicalGraph:
  7.     def __init__(self):
  8.         self.conn = pymongo.MongoClient()#建立无用户密码连接
  9.         cur_dir = '/'.join(os.path.abspath(__file__).split('/')[:-1])#当前文件夹地址
  10.         self.db = self.conn['medical']
  11.         self.col = self.db['data']
  12.         first_words = [i.strip() for i in open(os.path.join(cur_dir, 'first_name.txt'))]
  13.         alphabets = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y', 'z']
  14.         nums = ['1','2','3','4','5','6','7','8','9','0']
  15.         self.stop_words = first_words + alphabets + nums
  16.         self.key_dict = {
  17.             '医保疾病' : 'yibao_status',
  18.             "患病比例" : "get_prob",
  19.             "易感人群" : "easy_get",
  20.             "传染方式" : "get_way",
  21.             "就诊科室" : "cure_department",
  22.             "治疗方式" : "cure_way",
  23.             "治疗周期" : "cure_lasttime",
  24.             "治愈率" : "cured_prob",
  25.             '药品明细': 'drug_detail',
  26.             '药品推荐': 'recommand_drug',
  27.             '推荐': 'recommand_eat',
  28.             '忌食': 'not_eat',
  29.             '宜食': 'do_eat',
  30.             '症状': 'symptom',
  31.             '检查': 'check',
  32.             '成因': 'cause',
  33.             '预防措施': 'prevent',
  34.             '所属类别': 'category',
  35.             '简介': 'desc',
  36.             '名称': 'name',
  37.             '常用药品' : 'common_drug',
  38.             '治疗费用': 'cost_money',
  39.             '并发症': 'acompany'
  40.         }
  41.         self.cuter = CutWords()#创建最大前向/后向匹配类的实例
  42.  
  43.     def collect_medical(self):
  44.         cates = []
  45.         inspects = []
  46.         count = 0
  47.         for item in self.col.find():
  48.             data = {}
  49.             basic_info = item['basic_info']#遍历
  50.             name = basic_info['name']
  51.             if not name:
  52.                 continue
  53.             # 基本信息
  54.             data['名称'] = name
  55.             data['简介'] = '\n'.join(basic_info['desc']).replace('\r\n\t', '').replace('\r\n\n\n','').replace(' ','').replace('\r\n','\n')
  56.             category = basic_info['category']
  57.             data['所属类别'] = category
  58.             cates += category
  59.             inspect = item['inspect_info']
  60.             inspects += inspect
  61.             attributes = basic_info['attributes']
  62.             # 成因及预防
  63.             data['预防措施'] = item['prevent_info']
  64.             data['成因'] = item['cause_info']
  65.             # 并发症
  66.             data['症状'] = list(set([i for i in item["symptom_info"][0] if i[0] not in self.stop_words]))
  67.             for attr in attributes:
  68.                 attr_pair = attr.split('：')
  69.                 if len(attr_pair) == 2:
  70.                     key = attr_pair[0]
  71.                     value = attr_pair[1]
  72.                     data[key] = value
  73.  
  74.             # 检查
  75.             inspects = item['inspect_info']
  76.             jcs = []
  77.             for inspect in inspects:
  78.                 jc_name = self.get_inspect(inspect)
  79.                 if jc_name:
  80.                     jcs.append(jc_name)
  81.             data['检查'] = jcs
  82.  
  83.             # 食物
  84.             food_info = item['food_info']
  85.             if food_info:
  86.                 data['宜食'] = food_info['good']
  87.                 data['忌食'] = food_info['bad']
  88.                 data['推荐'] = food_info['recommand']
  89.  
  90.             # 药品
  91.             drug_info = item['drug_info']
  92.             data['药品推荐'] = list(set([i.split('(')[-1].replace(')','') for i in drug_info]))
  93.             data['药品明细'] = drug_info
  94.             data_modify = {}
  95.             for attr, value in data.items():
  96.                 attr_en = self.key_dict.get(attr)#'name'
  97.                 if attr_en:
  98.                     data_modify[attr_en] = value#把内容放到data里面，如  'name':白百咳
  99.                 if attr_en in ['yibao_status', 'get_prob', 'easy_get', 'get_way', "cure_lasttime", "cured_prob"]:
  100.                     data_modify[attr_en] = value.replace(' ','').replace('\t','')
  101.                 elif attr_en in ['cure_department', 'cure_way', 'common_drug']:
  102.                     data_modify[attr_en] = [i for i in value.split(' ') if i]
  103.                 elif attr_en in ['acompany']:
  104.                     acompany = [i for i in self.cuter.max_biward_cut(data_modify[attr_en]) if len(i) > 1]
  105.                     data_modify[attr_en] = acompany
  106.  
  107.             try:
  108.                 self.db['medical'].insert(data_modify)#插入字典
  109.                 count += 1
  110.                 print(count)
  111.             except Exception as e:
  112.                 print(e)
  113.  
  114.         return
  115.  
  116.  
  117.     def get_inspect(self, url):
  118.         res = self.db['jc'].find_one({'url':url})
  119.         if not res:
  120.             return ''
  121.         else:
  122.             return res['name']
  123.  
  124.     def modify_jc(self):
  125.         for item in self.db['jc'].find():
  126.             url = item['url']
  127.             content = item['html']
  128.             selector = etree.HTML(content)
  129.             name = selector.xpath('//title/text()')[0].split('结果分析')[0]
  130.             desc = selector.xpath('//meta[@name="description"]/@content')[0].replace('\r\n\t','')
  131.             self.db['jc'].update({'url':url}, {'$set':{'name':name, 'desc':desc}})
  132.  
  133.  
  134.  
  135. if __name__ == '__main__':
  136.     handler = MedicalGraph()
  137.     handler.modify_jc()
  138.     handler.collect_medical()
三、构建知识图谱

  1. 运行build_medicalgraph.py 将结构化JSON数据导入neo4j

在PyCharm终端安装py2neo：pip install py2neo==4.3.0 -i https://pypi.douban.com/simple

（1）知识图谱实体类型

（2）知识图谱实体关系类型

（3）知识图谱属性类型

（4）支持问答的类型

  1. import os
  2. import json
  3. from py2neo import Graph,Node
  4.  
  5. class MedicalGraph:
  6.     def __init__(self):
  7.         cur_dir = '/'.join(os.path.abspath(__file__).split('/')[:-1])
  8.         self.data_path = os.path.join(cur_dir, 'medical.json')#json地址
  9.         self.g = Graph('http://localhost:7474', auth = ('neo4j', '072766'))#连接数据库
  10.  
  11.     #读取文件
  12.     def read_nodes(self):
  13.         # 共７类节点
  14.         drugs = [] # 药品
  15.         foods = [] # 食物
  16.         checks = [] # 检查
  17.         departments = [] # 科室
  18.         producers = [] # 药品大类
  19.         diseases = [] # 疾病
  20.         symptoms = [] # 症状
  21.  
  22.         disease_infos = []##疾病信息，里面是字典格式[{'name':...,'desc':...,...,...},{}],里面是所有的实体节点
  23.  
  24.         # 构建节点实体关系
  25.         rels_department = [] #　科室－科室关系
  26.         rels_noteat = [] # 疾病－忌吃食物关系
  27.         rels_doeat = [] # 疾病－宜吃食物关系
  28.         rels_recommandeat = [] # 疾病－推荐吃食物关系
  29.         rels_commonddrug = [] # 疾病－通用药品关系
  30.         rels_recommanddrug = [] # 疾病－热门药品关系
  31.         rels_check = [] # 疾病－检查关系
  32.         rels_drug_producer = [] # 厂商－药物关系
  33.  
  34.         rels_symptom = [] # 疾病症状关系
  35.         rels_acompany = [] # 疾病并发关系
  36.         rels_category = [] #　疾病与科室之间的关系
  37.  
  38.  
  39.         count = 0
  40.         for data in open(self.data_path,encoding='utf-8'):
  41.             disease_dict = {}
  42.             count += 1
  43.             print(count)
  44.             data_json = json.loads(data)
  45.             disease = data_json['name']
  46.             disease_dict['name'] = disease
  47.             diseases.append(disease)
  48.             disease_dict['desc'] = ''
  49.             disease_dict['prevent'] = ''
  50.             disease_dict['cause'] = ''
  51.             disease_dict['easy_get'] = ''
  52.             disease_dict['cure_department'] = ''
  53.             disease_dict['cure_way'] = ''
  54.             disease_dict['cure_lasttime'] = ''
  55.             disease_dict['symptom'] = ''
  56.             disease_dict['cured_prob'] = ''
  57.  
  58.             if 'symptom' in data_json:
  59.                 symptoms += data_json['symptom']
  60.                 for symptom in data_json['symptom']:
  61.                     rels_symptom.append([disease, symptom])
  62.  
  63.             if 'acompany' in data_json:
  64.                 for acompany in data_json['acompany']:
  65.                     rels_acompany.append([disease, acompany])
  66.  
  67.             if 'desc' in data_json:
  68.                 disease_dict['desc'] = data_json['desc']
  69.  
  70.             if 'prevent' in data_json:
  71.                 disease_dict['prevent'] = data_json['prevent']
  72.  
  73.             if 'cause' in data_json:
  74.                 disease_dict['cause'] = data_json['cause']
  75.  
  76.             if 'get_prob' in data_json:
  77.                 disease_dict['get_prob'] = data_json['get_prob']
  78.  
  79.             if 'easy_get' in data_json:
  80.                 disease_dict['easy_get'] = data_json['easy_get']
  81.  
  82.             if 'cure_department' in data_json:
  83.                 cure_department = data_json['cure_department']
  84.                 if len(cure_department) == 1:
  85.                      rels_category.append([disease, cure_department[0]])
  86.                 if len(cure_department) == 2:#如果有两个科室，则会有一个大科室下面包含一个小科室
  87.                     big = cure_department[0]
  88.                     small = cure_department[1]
  89.                     rels_department.append([small, big])#将大科室与小科室关联
  90.                     rels_category.append([disease, small])#再将该疾病与小科室关联
  91.  
  92.                 disease_dict['cure_department'] = cure_department#然后将其保存到disease_dict字典和departments列表中
  93.                 departments += cure_department
  94.  
  95.             if 'cure_way' in data_json:
  96.                 disease_dict['cure_way'] = data_json['cure_way']
  97.  
  98.             if  'cure_lasttime' in data_json:
  99.                 disease_dict['cure_lasttime'] = data_json['cure_lasttime']
  100.  
  101.             if 'cured_prob' in data_json:
  102.                 disease_dict['cured_prob'] = data_json['cured_prob']
  103.  
  104.             if 'common_drug' in data_json:
  105.                 common_drug = data_json['common_drug']
  106.                 for drug in common_drug:
  107.                     rels_commonddrug.append([disease, drug])
  108.                 drugs += common_drug
  109.  
  110.             if 'recommand_drug' in data_json:
  111.                 recommand_drug = data_json['recommand_drug']
  112.                 drugs += recommand_drug
  113.                 for drug in recommand_drug:
  114.                     rels_recommanddrug.append([disease, drug])
  115.  
  116.             if 'not_eat' in data_json:
  117.                 not_eat = data_json['not_eat']
  118.                 for _not in not_eat:
  119.                     rels_noteat.append([disease, _not])
  120.  
  121.                 foods += not_eat
  122.                 do_eat = data_json['do_eat']
  123.                 for _do in do_eat:
  124.                     rels_doeat.append([disease, _do])
  125.  
  126.                 foods += do_eat
  127.                 recommand_eat = data_json['recommand_eat']
  128.  
  129.                 for _recommand in recommand_eat:
  130.                     rels_recommandeat.append([disease, _recommand])
  131.                 foods += recommand_eat
  132.  
  133.             if 'check' in data_json:
  134.                 check = data_json['check']
  135.                 for _check in check:
  136.                     rels_check.append([disease, _check])
  137.                 checks += check
  138.             if 'drug_detail' in data_json:
  139.                 drug_detail = data_json['drug_detail']
  140.                 producer = [i.split('(')[0] for i in drug_detail]
  141.                 rels_drug_producer += [[i.split('(')[0], i.split('(')[-1].replace(')', '')] for i in drug_detail]
  142.                 producers += producer
  143.             disease_infos.append(disease_dict)#最后将建立好的该疾病字典保存到disease_infos列表当中
  144.         return set(drugs), set(foods), set(checks), set(departments), set(producers), set(symptoms), set(diseases), disease_infos,\
  145.                rels_check, rels_recommandeat, rels_noteat, rels_doeat, rels_department, rels_commonddrug, rels_drug_producer, rels_recommanddrug,\
  146.                rels_symptom, rels_acompany, rels_category#将该函数中保存好的各项列表值作为返回值返回。
  147.  
  148.     #建立节点
  149.     def create_node(self, label, nodes):
  150.         count = 0
  151.         for node_name in nodes:#新建节点，每个节点有一个name属性
  152.             node = Node(label, name=node_name)
  153.             self.g.create(node)#新建数据库节点
  154.             count += 1
  155.             print(count, len(nodes))
  156.         return
  157.  
  158.     #创建知识图谱中心疾病的节点
  159.     def create_diseases_nodes(self, disease_infos):
  160.         count = 0
  161.         for disease_dict in disease_infos:
  162.             node = Node("Disease", name=disease_dict['name'], desc=disease_dict['desc'],
  163.                         prevent=disease_dict['prevent'] ,cause=disease_dict['cause'],
  164.                         easy_get=disease_dict['easy_get'],cure_lasttime=disease_dict['cure_lasttime'],
  165.                         cure_department=disease_dict['cure_department']
  166.                         ,cure_way=disease_dict['cure_way'] , cured_prob=disease_dict['cured_prob'])
  167.             self.g.create(node)
  168.             count += 1
  169.             print(count)
  170.         return
  171.  
  172.     #创建知识图谱实体节点类型schema
  173.     def create_graphnodes(self):
  174.         Drugs, Foods, Checks, Departments, Producers, Symptoms, Diseases, disease_infos,rels_check, rels_recommandeat, rels_noteat, rels_doeat, rels_department, rels_commonddrug, rels_drug_producer, rels_recommanddrug,rels_symptom, rels_acompany, rels_category = self.read_nodes()
  175.         self.create_diseases_nodes(disease_infos)#为每一个疾病都创建了一个节点,每个节点有9个属性
  176.         self.create_node('Drug', Drugs)#为每个药品创建一个节点,有一个属性name = 药品名称
  177.         print(len(Drugs))
  178.         self.create_node('Food', Foods)
  179.         print(len(Foods))
  180.         self.create_node('Check', Checks)
  181.         print(len(Checks))
  182.         self.create_node('Department', Departments)
  183.         print(len(Departments))
  184.         self.create_node('Producer', Producers)
  185.         print(len(Producers))
  186.         self.create_node('Symptom', Symptoms)
  187.         return
  188.  
  189.  
  190.     #创建实体关系边
  191.     def create_graphrels(self):
  192.         Drugs, Foods, Checks, Departments, Producers, Symptoms, Diseases, disease_infos, rels_check, rels_recommandeat, rels_noteat, rels_doeat, rels_department, rels_commonddrug, rels_drug_producer, rels_recommanddrug,rels_symptom, rels_acompany, rels_category = self.read_nodes()
  193.         self.create_relationship('Disease', 'Food', rels_recommandeat, 'recommand_eat', '推荐食谱')
  194.         self.create_relationship('Disease', 'Food', rels_noteat, 'no_eat', '忌吃')
  195.         self.create_relationship('Disease', 'Food', rels_doeat, 'do_eat', '宜吃')
  196.         self.create_relationship('Department', 'Department', rels_department, 'belongs_to', '属于')
  197.         self.create_relationship('Disease', 'Drug', rels_commonddrug, 'common_drug', '常用药品')
  198.         self.create_relationship('Producer', 'Drug', rels_drug_producer, 'drugs_of', '生产药品')
  199.         self.create_relationship('Disease', 'Drug', rels_recommanddrug, 'recommand_drug', '好评药品')
  200.         self.create_relationship('Disease', 'Check', rels_check, 'need_check', '诊断检查')
  201.         self.create_relationship('Disease', 'Symptom', rels_symptom, 'has_symptom', '症状')
  202.         self.create_relationship('Disease', 'Disease', rels_acompany, 'acompany_with', '并发症')
  203.         self.create_relationship('Disease', 'Department', rels_category, 'belongs_to', '所属科室')
  204.  
  205.     #创建实体关联边
  206.     def create_relationship(self, start_node, end_node, edges, rel_type, rel_name):
  207.         # start_node是边的起点名称， end_node是边的终点如'Food', edges是边的关系(实际关联项) ，p是起始节点的名称，q是到达节点的名称，rel_type是关系的类型，具体字段如‘do_eat’；rel_name是关系的名称，具体字段如‘宜吃’
  208.         count = 0
  209.         # 去重处理,实体关系为形如[[“a”,“b”],[“c”,“d”]]的嵌套list，无法直接用set去重，所以先将嵌套内层的list转为字符串，再用set
  210.         set_edges = []
  211.         for edge in edges:
  212.             set_edges.append('###'.join(edge))#这就转化为字符串了
  213.         all = len(set(set_edges))
  214.         for edge in set(set_edges):#去重
  215.             edge = edge.split('###')
  216.             p = edge[0]
  217.             q = edge[1]
  218.             query = "match(p:%s),(q:%s) where p.name='%s'and q.name='%s' create (p)-[rel:%s{name:'%s'}]->(q)" % (
  219.                 start_node, end_node, p, q, rel_type, rel_name)
  220.             try:
  221.                 self.g.run(query)#run()的用法大概是这样的,比如start_node, end_node这两种节点已经提前存进去了，比如label分别是Disease和Food,现在需要在他们间建立关系，括号里面是neo4j的查询语句cql，语法结构类似sql语句。关系就是rel_typ，还带了一个属性{name:'rel_name'}
  222.                 count += 1
  223.                 print(rel_type, count, all)
  224.             except Exception as e:
  225.                 print(e)
  226.         return
  227.  
  228.     #导出数据
  229.     def export_data(self):
  230.         Drugs, Foods, Checks, Departments, Producers, Symptoms, Diseases, disease_infos, rels_check, rels_recommandeat, rels_noteat, rels_doeat, rels_department, rels_commonddrug, rels_drug_producer, rels_recommanddrug, rels_symptom, rels_acompany, rels_category = self.read_nodes()
  231.         f_drug = open('drug.txt', 'w+')
  232.         f_food = open('food.txt', 'w+')
  233.         f_check = open('check.txt', 'w+')
  234.         f_department = open('department.txt', 'w+')
  235.         f_producer = open('producer.txt', 'w+')
  236.         f_symptom = open('symptoms.txt', 'w+')
  237.         f_disease = open('disease.txt', 'w+')
  238.  
  239.         f_drug.write('\n'.join(list(Drugs)))
  240.         f_food.write('\n'.join(list(Foods)))
  241.         f_check.write('\n'.join(list(Checks)))
  242.         f_department.write('\n'.join(list(Departments)))
  243.         f_producer.write('\n'.join(list(Producers)))
  244.         f_symptom.write('\n'.join(list(Symptoms)))
  245.         f_disease.write('\n'.join(list(Diseases)))
  246.  
  247.         f_drug.close()
  248.         f_food.close()
  249.         f_check.close()
  250.         f_department.close()
  251.         f_producer.close()
  252.         f_symptom.close()
  253.         f_disease.close()
  254.  
  255.         return
  256.  
  257.  
  258. if __name__ == '__main__':
  259.     handler = MedicalGraph()
  260.     print("step1:导入图谱节点中")
  261.     handler.create_graphnodes()
  262.     print("step2:导入图谱边中")      
  263.     handler.create_graphrels()
四、问答部分

  1. question_classifier.py

  1. import os
  2. import ahocorasick#ahocorasick实现快速的关键字匹配
  3.  
  4. class QuestionClassifier:
  5.     def __init__(self):
  6.         cur_dir = '/'.join(os.path.abspath(__file__).split('/')[:-1])
  7.         #　特征词路径
  8.         self.disease_path = os.path.join(cur_dir, 'disease.txt')
  9.         self.department_path = os.path.join(cur_dir, 'department.txt')
  10.         self.check_path = os.path.join(cur_dir, 'check.txt')
  11.         self.drug_path = os.path.join(cur_dir, 'drug.txt')
  12.         self.food_path = os.path.join(cur_dir, 'food.txt')
  13.         self.producer_path = os.path.join(cur_dir, 'producer.txt')
  14.         self.symptom_path = os.path.join(cur_dir, 'symptom.txt')
  15.         self.deny_path = os.path.join(cur_dir, 'deny.txt')
  16.         # 加载特征词
  17.         self.disease_wds= [i.strip() for i in open(self.disease_path,encoding='utf-8') if i.strip()]
  18.         self.department_wds= [i.strip() for i in open(self.department_path,encoding='utf-8') if i.strip()]
  19.         self.check_wds= [i.strip() for i in open(self.check_path,encoding='utf-8') if i.strip()]
  20.         self.drug_wds= [i.strip() for i in open(self.drug_path,encoding='utf-8') if i.strip()]
  21.         self.food_wds= [i.strip() for i in open(self.food_path,encoding='utf-8') if i.strip()]
  22.         self.producer_wds= [i.strip() for i in open(self.producer_path,encoding='utf-8') if i.strip()]
  23.         self.symptom_wds= [i.strip() for i in open(self.symptom_path,encoding='utf-8') if i.strip()]
  24.         self.region_words = set(self.department_wds + self.disease_wds + self.check_wds + self.drug_wds + self.food_wds + self.producer_wds + self.symptom_wds)
  25.         self.deny_words = [i.strip() for i in open(self.deny_path,encoding='utf-8') if i.strip()]
  26.         # 构造领域actree
  27.         self.region_tree = self.build_actree(list(self.region_words))
  28.         # 构建词典
  29.         self.wdtype_dict = self.build_wdtype_dict()#词的类型的字典，{'百日咳':'disease',....}
  30.         # 问句疑问词
  31.         self.symptom_qwds = ['症状', '表征', '现象', '症候', '表现']
  32.         self.cause_qwds = ['原因','成因', '为什么', '怎么会', '怎样才', '咋样才', '怎样会', '如何会', '为啥', '为何', '如何才会', '怎么才会', '会导致', '会造成']
  33.         self.acompany_qwds = ['并发症', '并发', '一起发生', '一并发生', '一起出现', '一并出现', '一同发生', '一同出现', '伴随发生', '伴随', '共现']
  34.         self.food_qwds = ['饮食', '饮用', '吃', '食', '伙食', '膳食', '喝', '菜' ,'忌口', '补品', '保健品', '食谱', '菜谱', '食用', '食物','补品']
  35.         self.drug_qwds = ['药', '药品', '用药', '胶囊', '口服液', '炎片']
  36.         self.prevent_qwds = ['预防', '防范', '抵制', '抵御', '防止','躲避','逃避','避开','免得','逃开','避开','避掉','躲开','躲掉','绕开',
  37.                              '怎样才能不', '怎么才能不', '咋样才能不','咋才能不', '如何才能不',
  38.                              '怎样才不', '怎么才不', '咋样才不','咋才不', '如何才不',
  39.                              '怎样才可以不', '怎么才可以不', '咋样才可以不', '咋才可以不', '如何可以不',
  40.                              '怎样才可不', '怎么才可不', '咋样才可不', '咋才可不', '如何可不']
  41.         self.lasttime_qwds = ['周期', '多久', '多长时间', '多少时间', '几天', '几年', '多少天', '多少小时', '几个小时', '多少年']
  42.         self.cureway_qwds = ['怎么治疗', '如何医治', '怎么医治', '怎么治', '怎么医', '如何治', '医治方式', '疗法', '咋治', '怎么办', '咋办', '咋治']
  43.         self.cureprob_qwds = ['多大概率能治好', '多大几率能治好', '治好希望大么', '几率', '几成', '比例', '可能性', '能治', '可治', '可以治', '可以医']
  44.         self.easyget_qwds = ['易感人群', '容易感染', '易发人群', '什么人', '哪些人', '感染', '染上', '得上']
  45.         self.check_qwds = ['检查', '检查项目', '查出', '检查', '测出', '试出']
  46.         self.belong_qwds = ['属于什么科', '属于', '什么科', '科室']
  47.         self.cure_qwds = ['治疗什么', '治啥', '治疗啥', '医治啥', '治愈啥', '主治啥', '主治什么', '有什么用', '有何用', '用处', '用途',
  48.                           '有什么好处', '有什么益处', '有何益处', '用来', '用来做啥', '用来作甚', '需要', '要']
  49.  
  50.         print('model init finished ......')
  51.  
  52.         return
  53.  
  54.     #分类主函数
  55.     def classify(self, question):
  56.         data = {}
  57.         medical_dict = self.check_medical(question)#从问句中匹配出的，{'百日咳':'disease',....}
  58.         if not medical_dict:
  59.             return {}
  60.         data['args'] = medical_dict
  61.         #收集问句当中所涉及到的实体类型
  62.         types = []
  63.         for type_ in medical_dict.values():
  64.             types += type_
  65.         question_type = 'others'
  66.  
  67.         question_types = []
  68.  
  69.         # 症状
  70.         if self.check_words(self.symptom_qwds, question) and ('disease' in types):
  71.             question_type = 'disease_symptom'#问句类型为：已知疾病询问症状
  72.             question_types.append(question_type)
  73.  
  74.         if self.check_words(self.symptom_qwds, question) and ('symptom' in types):
  75.             question_type = 'symptom_disease'#已知症状问疾病
  76.             question_types.append(question_type)
  77.  
  78.         # 原因
  79.         if self.check_words(self.cause_qwds, question) and ('disease' in types):
  80.             question_type = 'disease_cause'
  81.             question_types.append(question_type)
  82.         # 并发症
  83.         if self.check_words(self.acompany_qwds, question) and ('disease' in types):
  84.             question_type = 'disease_acompany'
  85.             question_types.append(question_type)
  86.  
  87.         # 推荐食品
  88.         if self.check_words(self.food_qwds, question) and 'disease' in types:
  89.             deny_status = self.check_words(self.deny_words, question)
  90.             if deny_status:
  91.                 question_type = 'disease_not_food'
  92.             else:
  93.                 question_type = 'disease_do_food'
  94.             question_types.append(question_type)
  95.  
  96.         #已知食物找疾病
  97.         if self.check_words(self.food_qwds+self.cure_qwds, question) and 'food' in types:
  98.             deny_status = self.check_words(self.deny_words, question)
  99.             if deny_status:
  100.                 question_type = 'food_not_disease'
  101.             else:
  102.                 question_type = 'food_do_disease'
  103.             question_types.append(question_type)
  104.  
  105.         # 推荐药品
  106.         if self.check_words(self.drug_qwds, question) and 'disease' in types:
  107.             question_type = 'disease_drug'
  108.             question_types.append(question_type)
  109.  
  110.         # 药品治啥病
  111.         if self.check_words(self.cure_qwds, question) and 'drug' in types:
  112.             question_type = 'drug_disease'
  113.             question_types.append(question_type)
  114.  
  115.         # 疾病接受检查项目
  116.         if self.check_words(self.check_qwds, question) and 'disease' in types:
  117.             question_type = 'disease_check'
  118.             question_types.append(question_type)
  119.  
  120.         # 已知检查项目查相应疾病
  121.         if self.check_words(self.check_qwds+self.cure_qwds, question) and 'check' in types:
  122.             question_type = 'check_disease'
  123.             question_types.append(question_type)
  124.  
  125.         #　症状防御
  126.         if self.check_words(self.prevent_qwds, question) and 'disease' in types:
  127.             question_type = 'disease_prevent'
  128.             question_types.append(question_type)
  129.  
  130.         # 疾病医疗周期
  131.         if self.check_words(self.lasttime_qwds, question) and 'disease' in types:
  132.             question_type = 'disease_lasttime'
  133.             question_types.append(question_type)
  134.  
  135.         # 疾病治疗方式
  136.         if self.check_words(self.cureway_qwds, question) and 'disease' in types:
  137.             question_type = 'disease_cureway'
  138.             question_types.append(question_type)
  139.  
  140.         # 疾病治愈可能性
  141.         if self.check_words(self.cureprob_qwds, question) and 'disease' in types:
  142.             question_type = 'disease_cureprob'
  143.             question_types.append(question_type)
  144.  
  145.         # 疾病易感染人群
  146.         if self.check_words(self.easyget_qwds, question) and 'disease' in types :
  147.             question_type = 'disease_easyget'
  148.             question_types.append(question_type)
  149.  
  150.         # 若没有查到相关的外部查询信息，那么则将该疾病的描述信息返回
  151.         if question_types == [] and 'disease' in types:
  152.             question_types = ['disease_desc']
  153.  
  154.         # 若没有查到相关的外部查询信息，那么则将该疾病的描述信息返回
  155.         if question_types == [] and 'symptom' in types:
  156.             question_types = ['symptom_disease']
  157.  
  158.         # 将多个分类结果进行合并处理，组装成一个字典
  159.         data['question_types'] = question_types
  160.  
  161.         return data
  162.  
  163.     #构造词对应的类型
  164.     def build_wdtype_dict(self):
  165.         wd_dict = dict()#创建一个空字典
  166.         for wd in self.region_words:#所有的关键字
  167.             wd_dict[wd] = []
  168.             if wd in self.disease_wds:#如果该关键字属于 疾病，则wd_dict[wd] = ['disease']
  169.                 wd_dict[wd].append('disease')
  170.             if wd in self.department_wds:
  171.                 wd_dict[wd].append('department')
  172.             if wd in self.check_wds:
  173.                 wd_dict[wd].append('check')
  174.             if wd in self.drug_wds:
  175.                 wd_dict[wd].append('drug')
  176.             if wd in self.food_wds:
  177.                 wd_dict[wd].append('food')
  178.             if wd in self.symptom_wds:
  179.                 wd_dict[wd].append('symptom')
  180.             if wd in self.producer_wds:
  181.                 wd_dict[wd].append('producer')
  182.         return wd_dict#返回了一个字典，里面的内容是{'百日咳':'disease',....}
  183.  
  184.     #构造actree，加速过滤，输入list，输出一个AC树
  185.     def build_actree(self, wordlist):
  186.         actree = ahocorasick.Automaton()
  187.         for index, word in enumerate(wordlist):
  188.             actree.add_word(word, (index, word))
  189.         actree.make_automaton()
  190.         return actree
  191.  
  192.     #问句过滤
  193.     def check_medical(self, question):
  194.         region_wds = []
  195.         for i in self.region_tree.iter(question):
  196.             # 快速匹配，匹配的结果是一个双重元组序列，形如('百日咳',(3324,'百日咳'))，就是前面的actree.add_word(word, (index, word))给加进来成tree的。这个index是在前面所有关键字加在一起之后的region_words中的序列号
  197.             wd = i[1][1]#把'百日咳'这个关键字给挑出来了
  198.             region_wds.append(wd)#把所有的关键字列入列表region_wds
  199.         stop_wds = []
  200.         for wd1 in region_wds:#1.如果一个问句里面挑出了两个词
  201.             for wd2 in region_wds:
  202.                 if wd1 in wd2 and wd1 != wd2:#2.如果其中一个词包含另一个词
  203.                     stop_wds.append(wd1)#3.则把短的词写入stop_wds
  204.         final_wds = [i for i in region_wds if i not in stop_wds]#4.final_wds中是把region_wds去掉stop_wds。就是说如果问句中一个词包含另外一个词，则以长词为准。
  205.         final_dict = {i:self.wdtype_dict.get(i) for i in final_wds}#类似于{'百日咳':'disease',....}，其中'百日咳'在问句中，且不在stop_wds中。
  206.  
  207.         return final_dict
  208.  
  209.     #基于特征词进行分类
  210.     def check_words(self, wds, sent):
  211.         for wd in wds:
  212.             if wd in sent:
  213.                 return True
  214.         return False
  215.  
  216.  
  217. if __name__ == '__main__':
  218.     handler = QuestionClassifier()
  219.     while 1:
  220.         question = input('input an question:')
  221.         data = handler.classify(question)
  222.         print(data)

2.question_parser.py

  1. class QuestionPaser:
  2.  
  3.     #构建实体节点
  4.     def build_entitydict(self, args):
  5.         entity_dict = {}
  6.         for arg, types in args.items():
  7.             for type in types:
  8.                 if type not in entity_dict:
  9.                     entity_dict[type] = [arg]
  10.                 else:
  11.                     entity_dict[type].append(arg)
  12.  
  13.         return entity_dict#构建一个字典，将所有同类型的实体放在一个值里，键是类型。如{'disease':['百日咳',...],....}
  14.  
  15.     #解析主函数
  16.     def parser_main(self, res_classify):
  17.         args = res_classify['args']#{'百日咳':'disease',....}
  18.         entity_dict = self.build_entitydict(args)#{'disease':['百日咳',...],....}
  19.         question_types = res_classify['question_types']#问题类型
  20.         sqls = []
  21.         for question_type in question_types:
  22.             sql_ = {}
  23.             sql_['question_type'] = question_type
  24.             sql = []
  25.             if question_type == 'disease_symptom':
  26.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  27.  
  28.             elif question_type == 'symptom_disease':
  29.                 sql = self.sql_transfer(question_type, entity_dict.get('symptom'))
  30.  
  31.             elif question_type == 'disease_cause':
  32.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  33.  
  34.             elif question_type == 'disease_acompany':
  35.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  36.  
  37.             elif question_type == 'disease_not_food':
  38.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  39.  
  40.             elif question_type == 'disease_do_food':
  41.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  42.  
  43.             elif question_type == 'food_not_disease':
  44.                 sql = self.sql_transfer(question_type, entity_dict.get('food'))
  45.  
  46.             elif question_type == 'food_do_disease':
  47.                 sql = self.sql_transfer(question_type, entity_dict.get('food'))
  48.  
  49.             elif question_type == 'disease_drug':
  50.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  51.  
  52.             elif question_type == 'drug_disease':
  53.                 sql = self.sql_transfer(question_type, entity_dict.get('drug'))
  54.  
  55.             elif question_type == 'disease_check':
  56.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  57.  
  58.             elif question_type == 'check_disease':
  59.                 sql = self.sql_transfer(question_type, entity_dict.get('check'))
  60.  
  61.             elif question_type == 'disease_prevent':
  62.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  63.  
  64.             elif question_type == 'disease_lasttime':
  65.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  66.  
  67.             elif question_type == 'disease_cureway':
  68.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  69.  
  70.             elif question_type == 'disease_cureprob':
  71.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  72.  
  73.             elif question_type == 'disease_easyget':
  74.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  75.  
  76.             elif question_type == 'disease_desc':
  77.                 sql = self.sql_transfer(question_type, entity_dict.get('disease'))
  78.  
  79.             if sql:
  80.                 sql_['sql'] = sql
  81.  
  82.                 sqls.append(sql_)
  83.  
  84.         return sqls
  85.  
  86.     #针对不同的问题，分开进行处理
  87.     def sql_transfer(self, question_type, entities):#question_type是'disease_do_food'等问题类型，entities是['百日咳',...]等实体
  88.         if not entities:
  89.             return []
  90.  
  91.         # 查询语句
  92.         sql = []
  93.         # 查询疾病的原因
  94.         if question_type == 'disease_cause':
  95.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.cause".format(i) for i in entities]
  96.             # 查询节点m，类型为Disease，当m的属性name为entities，返回m的那么，cause
  97.  
  98.         # 查询疾病的防御措施
  99.         elif question_type == 'disease_prevent':
  100.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.prevent".format(i) for i in entities]
  101.  
  102.         # 查询疾病的持续时间
  103.         elif question_type == 'disease_lasttime':
  104.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.cure_lasttime".format(i) for i in entities]
  105.  
  106.         # 查询疾病的治愈概率
  107.         elif question_type == 'disease_cureprob':
  108.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.cured_prob".format(i) for i in entities]
  109.  
  110.         # 查询疾病的治疗方式
  111.         elif question_type == 'disease_cureway':
  112.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.cure_way".format(i) for i in entities]
  113.  
  114.         # 查询疾病的易发人群
  115.         elif question_type == 'disease_easyget':
  116.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.easy_get".format(i) for i in entities]
  117.  
  118.         # 查询疾病的相关介绍
  119.         elif question_type == 'disease_desc':
  120.             sql = ["MATCH (m:Disease) where m.name = '{0}' return m.name, m.desc".format(i) for i in entities]
  121.  
  122.         # 查询疾病有哪些症状
  123.         elif question_type == 'disease_symptom':
  124.             sql = ["MATCH (m:Disease)-[r:has_symptom]->(n:Symptom) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  125.  
  126.         # 查询症状会导致哪些疾病
  127.         elif question_type == 'symptom_disease':
  128.             sql = ["MATCH (m:Disease)-[r:has_symptom]->(n:Symptom) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  129.  
  130.         # 查询疾病的并发症
  131.         elif question_type == 'disease_acompany':
  132.             sql1 = ["MATCH (m:Disease)-[r:acompany_with]->(n:Disease) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  133.             sql2 = ["MATCH (m:Disease)-[r:acompany_with]->(n:Disease) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  134.             sql = sql1 + sql2
  135.         # 查询疾病的忌口
  136.         elif question_type == 'disease_not_food':
  137.             sql = ["MATCH (m:Disease)-[r:no_eat]->(n:Food) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  138.  
  139.         # 查询疾病建议吃的东西
  140.         elif question_type == 'disease_do_food':
  141.             sql1 = ["MATCH (m:Disease)-[r:do_eat]->(n:Food) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  142.             sql2 = ["MATCH (m:Disease)-[r:recommand_eat]->(n:Food) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  143.             sql = sql1 + sql2
  144.  
  145.         # 已知忌口查疾病
  146.         elif question_type == 'food_not_disease':
  147.             sql = ["MATCH (m:Disease)-[r:no_eat]->(n:Food) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  148.  
  149.         # 已知推荐查疾病
  150.         elif question_type == 'food_do_disease':
  151.             sql1 = ["MATCH (m:Disease)-[r:do_eat]->(n:Food) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  152.             sql2 = ["MATCH (m:Disease)-[r:recommand_eat]->(n:Food) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  153.             sql = sql1 + sql2
  154.  
  155.         # 查询疾病常用药品－药品别名记得扩充
  156.         elif question_type == 'disease_drug':
  157.             sql1 = ["MATCH (m:Disease)-[r:common_drug]->(n:Drug) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  158.             sql2 = ["MATCH (m:Disease)-[r:recommand_drug]->(n:Drug) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  159.             sql = sql1 + sql2
  160.  
  161.         # 已知药品查询能够治疗的疾病
  162.         elif question_type == 'drug_disease':
  163.             sql1 = ["MATCH (m:Disease)-[r:common_drug]->(n:Drug) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  164.             sql2 = ["MATCH (m:Disease)-[r:recommand_drug]->(n:Drug) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  165.             sql = sql1 + sql2
  166.         # 查询疾病应该进行的检查
  167.         elif question_type == 'disease_check':
  168.             sql = ["MATCH (m:Disease)-[r:need_check]->(n:Check) where m.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  169.  
  170.         # 已知检查查询疾病
  171.         elif question_type == 'check_disease':
  172.             sql = ["MATCH (m:Disease)-[r:need_check]->(n:Check) where n.name = '{0}' return m.name, r.name, n.name".format(i) for i in entities]
  173.  
  174.         return sql
  175.  
  176.  
  177.  
  178. if __name__ == '__main__':
  179.     handler = QuestionPaser()

3.answer_search.py

  1. from py2neo import Graph#python操作图数据库
  2.  
  3. class AnswerSearcher:
  4.     def __init__(self):
  5.         self.g = Graph('http://localhost:7474', auth=('neo4j', '072766'))
  6.         self.num_limit = 20#答案的实体数量上限。比如一个疾病可以吃什么食物，最多回答20个
  7.  
  8.     #执行cypher查询，并返回相应结果'''
  9.     def search_main(self, sqls):
  10.         final_answers = []
  11.         for sql_ in sqls:
  12.             question_type = sql_['question_type']
  13.             queries = sql_['sql']
  14.             answers = []
  15.             for query in queries:
  16.                 ress = self.g.run(query).data()#返回查询结果
  17.                 answers += ress
  18.             final_answer = self.answer_prettify(question_type, answers)#再根据[‘question_type’]的不同调用answer_prettify函数将查询结果和答案话术结合起来。
  19.             if final_answer:
  20.                 final_answers.append(final_answer)
  21.         return final_answers
  22.  
  23.     #根据对应的qustion_type，调用相应的回复模板
  24.     def answer_prettify(self, question_type, answers):
  25.         final_answer = []
  26.         if not answers:
  27.             return ''
  28.         if question_type == 'disease_symptom':
  29.             desc = [i['n.name'] for i in answers]
  30.             subject = answers[0]['m.name']
  31.             final_answer = '{0}的症状包括：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  32.  
  33.         elif question_type == 'symptom_disease':
  34.             desc = [i['m.name'] for i in answers]
  35.             subject = answers[0]['n.name']
  36.             final_answer = '症状{0}可能染上的疾病有：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  37.  
  38.         elif question_type == 'disease_cause':
  39.             desc = [i['m.cause'] for i in answers]
  40.             subject = answers[0]['m.name']
  41.             final_answer = '{0}可能的成因有：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  42.  
  43.         elif question_type == 'disease_prevent':
  44.             desc = [i['m.prevent'] for i in answers]
  45.             subject = answers[0]['m.name']
  46.             final_answer = '{0}的预防措施包括：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  47.  
  48.         elif question_type == 'disease_lasttime':
  49.             desc = [i['m.cure_lasttime'] for i in answers]
  50.             subject = answers[0]['m.name']
  51.             final_answer = '{0}治疗可能持续的周期为：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  52.  
  53.         elif question_type == 'disease_cureway':
  54.             desc = [';'.join(i['m.cure_way']) for i in answers]
  55.             subject = answers[0]['m.name']
  56.             final_answer = '{0}可以尝试如下治疗：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  57.  
  58.         elif question_type == 'disease_cureprob':
  59.             desc = [i['m.cured_prob'] for i in answers]
  60.             subject = answers[0]['m.name']
  61.             final_answer = '{0}治愈的概率为（仅供参考）：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  62.  
  63.         elif question_type == 'disease_easyget':
  64.             desc = [i['m.easy_get'] for i in answers]
  65.             subject = answers[0]['m.name']
  66.  
  67.             final_answer = '{0}的易感人群包括：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  68.  
  69.         elif question_type == 'disease_desc':
  70.             desc = [i['m.desc'] for i in answers]
  71.             subject = answers[0]['m.name']
  72.             final_answer = '{0},熟悉一下：{1}'.format(subject,  '；'.join(list(set(desc))[:self.num_limit]))
  73.  
  74.         elif question_type == 'disease_acompany':
  75.             desc1 = [i['n.name'] for i in answers]
  76.             desc2 = [i['m.name'] for i in answers]
  77.             subject = answers[0]['m.name']
  78.             desc = [i for i in desc1 + desc2 if i != subject]
  79.             final_answer = '{0}的症状包括：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  80.  
  81.         elif question_type == 'disease_not_food':
  82.             desc = [i['n.name'] for i in answers]
  83.             subject = answers[0]['m.name']
  84.             final_answer = '{0}忌食的食物包括有：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  85.  
  86.         elif question_type == 'disease_do_food':
  87.             do_desc = [i['n.name'] for i in answers if i['r.name'] == '宜吃']
  88.             recommand_desc = [i['n.name'] for i in answers if i['r.name'] == '推荐食谱']
  89.             subject = answers[0]['m.name']
  90.             final_answer = '{0}宜食的食物包括有：{1}\n推荐食谱包括有：{2}'.format(subject, ';'.join(list(set(do_desc))[:self.num_limit]), ';'.join(list(set(recommand_desc))[:self.num_limit]))
  91.  
  92.         elif question_type == 'food_not_disease':
  93.             desc = [i['m.name'] for i in answers]
  94.             subject = answers[0]['n.name']
  95.             final_answer = '患有{0}的人最好不要吃{1}'.format('；'.join(list(set(desc))[:self.num_limit]), subject)
  96.  
  97.         elif question_type == 'food_do_disease':
  98.             desc = [i['m.name'] for i in answers]
  99.             subject = answers[0]['n.name']
  100.             final_answer = '患有{0}的人建议多试试{1}'.format('；'.join(list(set(desc))[:self.num_limit]), subject)
  101.  
  102.         elif question_type == 'disease_drug':
  103.             desc = [i['n.name'] for i in answers]
  104.             subject = answers[0]['m.name']
  105.             final_answer = '{0}通常的使用的药品包括：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  106.  
  107.         elif question_type == 'drug_disease':
  108.             desc = [i['m.name'] for i in answers]
  109.             subject = answers[0]['n.name']
  110.             final_answer = '{0}主治的疾病有{1},可以试试'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  111.  
  112.         elif question_type == 'disease_check':
  113.             desc = [i['n.name'] for i in answers]
  114.             subject = answers[0]['m.name']
  115.             final_answer = '{0}通常可以通过以下方式检查出来：{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  116.  
  117.         elif question_type == 'check_disease':
  118.             desc = [i['m.name'] for i in answers]
  119.             subject = answers[0]['n.name']
  120.             final_answer = '通常可以通过{0}检查出来的疾病有{1}'.format(subject, '；'.join(list(set(desc))[:self.num_limit]))
  121.  
  122.         return final_answer
  123.  
  124.  
  125. if __name__ == '__main__':
  126.     searcher = AnswerSearcher()

4.chatbot_graph.py

  1. from question_classifier import *
  2. from question_parser import *
  3. from answer_search import *
  4.  
  5. #问答类
  6. class ChatBotGraph:
  7.     def __init__(self):
  8.         self.classifier = QuestionClassifier()
  9.         self.parser = QuestionPaser()
  10.         self.searcher = AnswerSearcher()
  11.  
  12.     def chat_main(self, sent):
  13.         answer = '您好，我是小鱼医药智能助理，希望可以帮到您。祝您身体棒棒！'
  14.         res_classify = self.classifier.classify(sent)
  15.         if not res_classify:
  16.             return answer#如果没有找到关键词，则返回固定答案
  17.         res_sql = self.parser.parser_main(res_classify)#查询语句
  18.         final_answers = self.searcher.search_main(res_sql)#根据查询语句返回答案
  19.         if not final_answers:
  20.             return answer
  21.         else:
  22.             return '\n'.join(final_answers)
  23.  
  24. if __name__ == '__main__':
  25.     handler = ChatBotGraph()
  26.     while 1:
  27.         question = input('用户:')
  28.         answer = handler.chat_main(question)
  29.         print('小鱼:', answer)

实现结果：

五、参考博客

(60条消息) 知识图谱的节点和关系实现（python）_python 知识图谱_chen_nnn的博客-CSDN博客

(60条消息) 菜哥学知识图谱（通过“基于医疗知识图谱的问答系统”）（四）（代码分析2）_weixin_40539807的博客-CSDN博客

(60条消息) 基于医疗知识图谱的问答系统源码详解_vivian_ll的博客-CSDN博客

本项目还有不足：关于疾病的起因、预防等，实际返回的是一大段文字，这里其实可以引入事件抽取的概念，进一步将原因结构化表示出来。这个可以后面进行尝试。

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/IT小白/article/detail/526803
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号