devbox
繁依Fanyi0
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

spacy简单使用

作者：繁依Fanyi0 | 2024-03-31 06:52:30

赞

踩

spacy

spacy官方: Install spaCy · spaCy Usage Documentation

目录

简介:

一、安装

1. 训练模型

二、功能

1. 分句 (sentencizer)

2.分词 (Tokenization)

3.词性标注 (Part-of-speech tagging)

4.识别停用词 (Stop words)

5.命名实体识别 (Named Entity Recognization)

6.依存分析 (Dependency Parsing)

7.词性还原 (Lemmatization)

8.提取名词短语 (Noun Chunks)

9.指代消解 (Coreference Resolution)

三、可视化

简介:

spacy 可以用于进行分词，命名实体识别，词性识别等等

一、安装

pip install spacy

1. 训练模型

安装之后还要下载官方的训练模型, 不同的语言有不同的训练模型,这里只用对应中文的模型演示:

python -m spacy download zh_core_web_sm

代码中使用:

  1. import spacy
  2. nlp = spacy.load("zh_core_web_sm")

模型官方文档:

Trained Models & Pipelines · spaCy Models Documentation

每种语言也会有几种不同的模型,例如中文的模型除了刚才下载的 zh_core_web_sm 外,还有zh_core_web_trf、zh_core_web_md 等,它们的区别在于准确度和体积大小, zh_core_web_sm 体积小,准确度相比zh_core_web_trf差,zh_core_web_trf相对就体积大。这样可以适应不同场景。

这里以模型 zh_core_web_sm 做一个介绍  

Trained Models & Pipelines · spaCy Models Documentation

  • LANGUAGE : 那种语言的模型
  • SIZE : 模型体积的大小
  • COMPONENTS(组件): 模型具备的功能

                tok2vec: 分词

                tagger: 词性标注

                parser: 依存分析

                senter: 分句

                ner: 命名实体识别

                attribute_ruler: 更改属性映射(没有具体了解)

  • PIPELINE (管道) :  管道组件, 模型会按照管道组件去执行

        

模型会中指明包含哪些词性、依存分析、实体种类:

 这是一些词性名称的解释:

IP：简单从句 
NP：名词短语 
VP：动词短语 
PU：断句符，通常是句号、问号、感叹号等标点符号 
LCP：方位词短语 
PP：介词短语 
CP：由‘的’构成的表示修饰性关系的短语 
DNP：由‘的’构成的表示所属关系的短语 
ADVP：副词短语 
ADJP：形容词短语 
DP：限定词短语 
QP：量词短语 
NN：常用名词 
NR：固有名词 
NT：时间名词 
PN：代词 
VV：动词 
VC：是 
CC：表示连词 
VE：有 
VA：表语形容词 
AS：内容标记（如：了） 
VRD：动补复合词 
CD: 表示基数词 
DT: determiner 表示限定词 
EX: existential there 存在句 
FW: foreign word 外来词 
IN: preposition or conjunction, subordinating 介词或从属连词 
JJ: adjective or numeral, ordinal 形容词或序数词 
JJR: adjective, comparative 形容词比较级 
JJS: adjective, superlative 形容词最高级 
LS: list item marker 列表标识 
MD: modal auxiliary 情态助动词 
PDT: pre-determiner 前位限定词 
POS: genitive marker 所有格标记 
PRP: pronoun, personal 人称代词 
RB: adverb 副词 
RBR: adverb, comparative 副词比较级 
RBS: adverb, superlative 副词最高级 
RP: particle 小品词 
SYM: symbol 符号 
TO:”to” as preposition or infinitive marker 作为介词或不定式标记 
WDT: WH-determiner WH限定词 
WP: WH-pronoun WH代词 
WP$: WH-pronoun, possessive WH所有格代词 
WRB:Wh-adverb WH副词

官方关于词性、依存关系、实体的名词解释:

  1. def explain(term):
  2.     """Get a description for a given POS tag, dependency label or entity type.
  3.     term (str): The term to explain.
  4.     RETURNS (str): The explanation, or `None` if not found in the glossary.
  5.     EXAMPLE:
  6.         >>> spacy.explain(u'NORP')
  7.         >>> doc = nlp(u'Hello world')
  8.         >>> print([w.text, w.tag_, spacy.explain(w.tag_) for w in doc])
  9.     """
  10.     if term in GLOSSARY:
  11.         return GLOSSARY[term]
  12.  
  13.  
  14. GLOSSARY = {
  15.     # POS tags
  16.     # Universal POS Tags
  17.     # http://universaldependencies.org/u/pos/
  18.     "ADJ": "adjective",
  19.     "ADP": "adposition",
  20.     "ADV": "adverb",
  21.     "AUX": "auxiliary",
  22.     "CONJ": "conjunction",
  23.     "CCONJ": "coordinating conjunction",
  24.     "DET": "determiner",
  25.     "INTJ": "interjection",
  26.     "NOUN": "noun",
  27.     "NUM": "numeral",
  28.     "PART": "particle",
  29.     "PRON": "pronoun",
  30.     "PROPN": "proper noun",
  31.     "PUNCT": "punctuation",
  32.     "SCONJ": "subordinating conjunction",
  33.     "SYM": "symbol",
  34.     "VERB": "verb",
  35.     "X": "other",
  36.     "EOL": "end of line",
  37.     "SPACE": "space",
  38.     # POS tags (English)
  39.     # OntoNotes 5 / Penn Treebank
  40.     # https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html
  41.     ".": "punctuation mark, sentence closer",
  42.     ",": "punctuation mark, comma",
  43.     "-LRB-": "left round bracket",
  44.     "-RRB-": "right round bracket",
  45.     "``": "opening quotation mark",
  46.     '""': "closing quotation mark",
  47.     "''": "closing quotation mark",
  48.     ":": "punctuation mark, colon or ellipsis",
  49.     "$": "symbol, currency",
  50.     "#": "symbol, number sign",
  51.     "AFX": "affix",
  52.     "CC": "conjunction, coordinating",
  53.     "CD": "cardinal number",
  54.     "DT": "determiner",
  55.     "EX": "existential there",
  56.     "FW": "foreign word",
  57.     "HYPH": "punctuation mark, hyphen",
  58.     "IN": "conjunction, subordinating or preposition",
  59.     "JJ": "adjective (English), other noun-modifier (Chinese)",
  60.     "JJR": "adjective, comparative",
  61.     "JJS": "adjective, superlative",
  62.     "LS": "list item marker",
  63.     "MD": "verb, modal auxiliary",
  64.     "NIL": "missing tag",
  65.     "NN": "noun, singular or mass",
  66.     "NNP": "noun, proper singular",
  67.     "NNPS": "noun, proper plural",
  68.     "NNS": "noun, plural",
  69.     "PDT": "predeterminer",
  70.     "POS": "possessive ending",
  71.     "PRP": "pronoun, personal",
  72.     "PRP$": "pronoun, possessive",
  73.     "RB": "adverb",
  74.     "RBR": "adverb, comparative",
  75.     "RBS": "adverb, superlative",
  76.     "RP": "adverb, particle",
  77.     "TO": 'infinitival "to"',
  78.     "UH": "interjection",
  79.     "VB": "verb, base form",
  80.     "VBD": "verb, past tense",
  81.     "VBG": "verb, gerund or present participle",
  82.     "VBN": "verb, past participle",
  83.     "VBP": "verb, non-3rd person singular present",
  84.     "VBZ": "verb, 3rd person singular present",
  85.     "WDT": "wh-determiner",
  86.     "WP": "wh-pronoun, personal",
  87.     "WP$": "wh-pronoun, possessive",
  88.     "WRB": "wh-adverb",
  89.     "SP": "space (English), sentence-final particle (Chinese)",
  90.     "ADD": "email",
  91.     "NFP": "superfluous punctuation",
  92.     "GW": "additional word in multi-word expression",
  93.     "XX": "unknown",
  94.     "BES": 'auxiliary "be"',
  95.     "HVS": 'forms of "have"',
  96.     "_SP": "whitespace",
  97.     # POS Tags (German)
  98.     # TIGER Treebank
  99.     # http://www.ims.uni-stuttgart.de/forschung/ressourcen/korpora/TIGERCorpus/annotation/tiger_introduction.pdf
  100.     "$(": "other sentence-internal punctuation mark",
  101.     "$,": "comma",
  102.     "$.": "sentence-final punctuation mark",
  103.     "ADJA": "adjective, attributive",
  104.     "ADJD": "adjective, adverbial or predicative",
  105.     "APPO": "postposition",
  106.     "APPR": "preposition; circumposition left",
  107.     "APPRART": "preposition with article",
  108.     "APZR": "circumposition right",
  109.     "ART": "definite or indefinite article",
  110.     "CARD": "cardinal number",
  111.     "FM": "foreign language material",
  112.     "ITJ": "interjection",
  113.     "KOKOM": "comparative conjunction",
  114.     "KON": "coordinate conjunction",
  115.     "KOUI": 'subordinate conjunction with "zu" and infinitive',
  116.     "KOUS": "subordinate conjunction with sentence",
  117.     "NE": "proper noun",
  118.     "NNE": "proper noun",
  119.     "PAV": "pronominal adverb",
  120.     "PROAV": "pronominal adverb",
  121.     "PDAT": "attributive demonstrative pronoun",
  122.     "PDS": "substituting demonstrative pronoun",
  123.     "PIAT": "attributive indefinite pronoun without determiner",
  124.     "PIDAT": "attributive indefinite pronoun with determiner",
  125.     "PIS": "substituting indefinite pronoun",
  126.     "PPER": "non-reflexive personal pronoun",
  127.     "PPOSAT": "attributive possessive pronoun",
  128.     "PPOSS": "substituting possessive pronoun",
  129.     "PRELAT": "attributive relative pronoun",
  130.     "PRELS": "substituting relative pronoun",
  131.     "PRF": "reflexive personal pronoun",
  132.     "PTKA": "particle with adjective or adverb",
  133.     "PTKANT": "answer particle",
  134.     "PTKNEG": "negative particle",
  135.     "PTKVZ": "separable verbal particle",
  136.     "PTKZU": '"zu" before infinitive',
  137.     "PWAT": "attributive interrogative pronoun",
  138.     "PWAV": "adverbial interrogative or relative pronoun",
  139.     "PWS": "substituting interrogative pronoun",
  140.     "TRUNC": "word remnant",
  141.     "VAFIN": "finite verb, auxiliary",
  142.     "VAIMP": "imperative, auxiliary",
  143.     "VAINF": "infinitive, auxiliary",
  144.     "VAPP": "perfect participle, auxiliary",
  145.     "VMFIN": "finite verb, modal",
  146.     "VMINF": "infinitive, modal",
  147.     "VMPP": "perfect participle, modal",
  148.     "VVFIN": "finite verb, full",
  149.     "VVIMP": "imperative, full",
  150.     "VVINF": "infinitive, full",
  151.     "VVIZU": 'infinitive with "zu", full',
  152.     "VVPP": "perfect participle, full",
  153.     "XY": "non-word containing non-letter",
  154.     # POS Tags (Chinese)
  155.     # OntoNotes / Chinese Penn Treebank
  156.     # https://repository.upenn.edu/cgi/viewcontent.cgi?article=1039&context=ircs_reports
  157.     "AD": "adverb",
  158.     "AS": "aspect marker",
  159.     "BA": "把 in ba-construction",
  160.     # "CD": "cardinal number",
  161.     "CS": "subordinating conjunction",
  162.     "DEC": "的 in a relative clause",
  163.     "DEG": "associative 的",
  164.     "DER": "得 in V-de const. and V-de-R",
  165.     "DEV": "地 before VP",
  166.     "ETC": "for words 等, 等等",
  167.     # "FW": "foreign words"
  168.     "IJ": "interjection",
  169.     # "JJ": "other noun-modifier",
  170.     "LB": "被 in long bei-const",
  171.     "LC": "localizer",
  172.     "M": "measure word",
  173.     "MSP": "other particle",
  174.     # "NN": "common noun",
  175.     "NR": "proper noun",
  176.     "NT": "temporal noun",
  177.     "OD": "ordinal number",
  178.     "ON": "onomatopoeia",
  179.     "P": "preposition excluding 把 and 被",
  180.     "PN": "pronoun",
  181.     "PU": "punctuation",
  182.     "SB": "被 in short bei-const",
  183.     # "SP": "sentence-final particle",
  184.     "VA": "predicative adjective",
  185.     "VC": "是 (copula)",
  186.     "VE": "有 as the main verb",
  187.     "VV": "other verb",
  188.     # Noun chunks
  189.     "NP": "noun phrase",
  190.     "PP": "prepositional phrase",
  191.     "VP": "verb phrase",
  192.     "ADVP": "adverb phrase",
  193.     "ADJP": "adjective phrase",
  194.     "SBAR": "subordinating conjunction",
  195.     "PRT": "particle",
  196.     "PNP": "prepositional noun phrase",
  197.     # Dependency Labels (English)
  198.     # ClearNLP / Universal Dependencies
  199.     # https://github.com/clir/clearnlp-guidelines/blob/master/md/specifications/dependency_labels.md
  200.     "acl": "clausal modifier of noun (adjectival clause)",
  201.     "acomp": "adjectival complement",
  202.     "advcl": "adverbial clause modifier",
  203.     "advmod": "adverbial modifier",
  204.     "agent": "agent",
  205.     "amod": "adjectival modifier",
  206.     "appos": "appositional modifier",
  207.     "attr": "attribute",
  208.     "aux": "auxiliary",
  209.     "auxpass": "auxiliary (passive)",
  210.     "case": "case marking",
  211.     "cc": "coordinating conjunction",
  212.     "ccomp": "clausal complement",
  213.     "clf": "classifier",
  214.     "complm": "complementizer",
  215.     "compound": "compound",
  216.     "conj": "conjunct",
  217.     "cop": "copula",
  218.     "csubj": "clausal subject",
  219.     "csubjpass": "clausal subject (passive)",
  220.     "dative": "dative",
  221.     "dep": "unclassified dependent",
  222.     "det": "determiner",
  223.     "discourse": "discourse element",
  224.     "dislocated": "dislocated elements",
  225.     "dobj": "direct object",
  226.     "expl": "expletive",
  227.     "fixed": "fixed multiword expression",
  228.     "flat": "flat multiword expression",
  229.     "goeswith": "goes with",
  230.     "hmod": "modifier in hyphenation",
  231.     "hyph": "hyphen",
  232.     "infmod": "infinitival modifier",
  233.     "intj": "interjection",
  234.     "iobj": "indirect object",
  235.     "list": "list",
  236.     "mark": "marker",
  237.     "meta": "meta modifier",
  238.     "neg": "negation modifier",
  239.     "nmod": "modifier of nominal",
  240.     "nn": "noun compound modifier",
  241.     "npadvmod": "noun phrase as adverbial modifier",
  242.     "nsubj": "nominal subject",
  243.     "nsubjpass": "nominal subject (passive)",
  244.     "nounmod": "modifier of nominal",
  245.     "npmod": "noun phrase as adverbial modifier",
  246.     "num": "number modifier",
  247.     "number": "number compound modifier",
  248.     "nummod": "numeric modifier",
  249.     "oprd": "object predicate",
  250.     "obj": "object",
  251.     "obl": "oblique nominal",
  252.     "orphan": "orphan",
  253.     "parataxis": "parataxis",
  254.     "partmod": "participal modifier",
  255.     "pcomp": "complement of preposition",
  256.     "pobj": "object of preposition",
  257.     "poss": "possession modifier",
  258.     "possessive": "possessive modifier",
  259.     "preconj": "pre-correlative conjunction",
  260.     "prep": "prepositional modifier",
  261.     "prt": "particle",
  262.     "punct": "punctuation",
  263.     "quantmod": "modifier of quantifier",
  264.     "rcmod": "relative clause modifier",
  265.     "relcl": "relative clause modifier",
  266.     "reparandum": "overridden disfluency",
  267.     "root": "root",
  268.     "vocative": "vocative",
  269.     "xcomp": "open clausal complement",
  270.     # Dependency labels (German)
  271.     # TIGER Treebank
  272.     # http://www.ims.uni-stuttgart.de/forschung/ressourcen/korpora/TIGERCorpus/annotation/tiger_introduction.pdf
  273.     # currently missing: 'cc' (comparative complement) because of conflict
  274.     # with English labels
  275.     "ac": "adpositional case marker",
  276.     "adc": "adjective component",
  277.     "ag": "genitive attribute",
  278.     "ams": "measure argument of adjective",
  279.     "app": "apposition",
  280.     "avc": "adverbial phrase component",
  281.     "cd": "coordinating conjunction",
  282.     "cj": "conjunct",
  283.     "cm": "comparative conjunction",
  284.     "cp": "complementizer",
  285.     "cvc": "collocational verb construction",
  286.     "da": "dative",
  287.     "dh": "discourse-level head",
  288.     "dm": "discourse marker",
  289.     "ep": "expletive es",
  290.     "hd": "head",
  291.     "ju": "junctor",
  292.     "mnr": "postnominal modifier",
  293.     "mo": "modifier",
  294.     "ng": "negation",
  295.     "nk": "noun kernel element",
  296.     "nmc": "numerical component",
  297.     "oa": "accusative object",
  298.     "oc": "clausal object",
  299.     "og": "genitive object",
  300.     "op": "prepositional object",
  301.     "par": "parenthetical element",
  302.     "pd": "predicate",
  303.     "pg": "phrasal genitive",
  304.     "ph": "placeholder",
  305.     "pm": "morphological particle",
  306.     "pnc": "proper noun component",
  307.     "rc": "relative clause",
  308.     "re": "repeated element",
  309.     "rs": "reported speech",
  310.     "sb": "subject",
  311.     "sbp": "passivized subject (PP)",
  312.     "sp": "subject or predicate",
  313.     "svp": "separable verb prefix",
  314.     "uc": "unit component",
  315.     "vo": "vocative",
  316.     # Named Entity Recognition
  317.     # OntoNotes 5
  318.     # https://catalog.ldc.upenn.edu/docs/LDC2013T19/OntoNotes-Release-5.0.pdf
  319.     "PERSON": "People, including fictional",
  320.     "NORP": "Nationalities or religious or political groups",
  321.     "FACILITY": "Buildings, airports, highways, bridges, etc.",
  322.     "FAC": "Buildings, airports, highways, bridges, etc.",
  323.     "ORG": "Companies, agencies, institutions, etc.",
  324.     "GPE": "Countries, cities, states",
  325.     "LOC": "Non-GPE locations, mountain ranges, bodies of water",
  326.     "PRODUCT": "Objects, vehicles, foods, etc. (not services)",
  327.     "EVENT": "Named hurricanes, battles, wars, sports events, etc.",
  328.     "WORK_OF_ART": "Titles of books, songs, etc.",
  329.     "LAW": "Named documents made into laws.",
  330.     "LANGUAGE": "Any named language",
  331.     "DATE": "Absolute or relative dates or periods",
  332.     "TIME": "Times smaller than a day",
  333.     "PERCENT": 'Percentage, including "%"',
  334.     "MONEY": "Monetary values, including unit",
  335.     "QUANTITY": "Measurements, as of weight or distance",
  336.     "ORDINAL": '"first", "second", etc.',
  337.     "CARDINAL": "Numerals that do not fall under another type",
  338.     # Named Entity Recognition
  339.     # Wikipedia
  340.     # http://www.sciencedirect.com/science/article/pii/S0004370212000276
  341.     # https://pdfs.semanticscholar.org/5744/578cc243d92287f47448870bb426c66cc941.pdf
  342.     "PER": "Named person or family.",
  343.     "MISC": "Miscellaneous entities, e.g. events, nationalities, products or works of art",
  344.     # https://github.com/ltgoslo/norne
  345.     "EVT": "Festivals, cultural events, sports events, weather phenomena, wars, etc.",
  346.     "PROD": "Product, i.e. artificially produced entities including speeches, radio shows, programming languages, contracts, laws and ideas",
  347.     "DRV": "Words (and phrases?) that are dervied from a name, but not a name in themselves, e.g. 'Oslo-mannen' ('the man from Oslo')",
  348.     "GPE_LOC": "Geo-political entity, with a locative sense, e.g. 'John lives in Spain'",
  349.     "GPE_ORG": "Geo-political entity, with an organisation sense, e.g. 'Spain declined to meet with Belgium'",
  350. }

url: https://github.com/explosion/spaCy/blob/master/spacy/glossary.py

二、功能

  1. import spacy
  2.  
  3.  
  4. s = "小米董事长叶凡决定投资华为。在2002年，他还创作了<遮天>。"
  5.  
  6. nlp = spacy.load("zh_core_web_sm")
  7. doc = nlp(s)
  8.

1. 分句 (sentencizer)

  1. # 1. 分句 (sentencizer)
  2. for i in doc.sents:
  3.     print(i)
  4.  
  5.  
  6.  
  7. """
  8. 小米董事长叶凡决定投资华为。
  9. 在2002年，他还创作了<遮天>。
  11. """

2.分词 (Tokenization)

  1. # 2. 分词 (Tokenization)
  2. print([w.text for w in doc])
  3.  
  4.  
  5. """
  6. ['小米', '董事长', '叶凡', '决定', '投资', '华为', '。', '在', '2002年', '，', '他', '还', '创作', '了', '<遮天>', '。']
  7. """

3.词性标注 (Part-of-speech tagging)

细粒度

  1. print([(w.text, w.tag_) for w in doc])
  2.  
  3.  
  4. """
  5. [('小米', 'NR'), ('董事长', 'NN'), ('叶凡', 'NR'), ('决定', 'VV'), ('投资', 'VV'), ('华为', 'NR'), ('。', 'PU'), ('在', 'P'), ('2002年', 'NT'), ('，', 'PU'), ('他', 'PN'), ('还', 'AD'), ('创作', 'VV'), ('了', 'AS'), ('<遮天>', 'NN'), ('。', 'PU')]
  6. """

粗粒度

  1. print([(w.text, w.pos_) for w in doc])
  2.  
  3. """
  4. [('小米', 'PROPN'), ('董事长', 'NOUN'), ('叶凡', 'PROPN'), ('决定', 'VERB'), ('投资', 'VERB'), ('华为', 'PROPN'), ('。', 'PUNCT'), ('在', 'ADP'), ('2002年', 'NOUN'), ('，', 'PUNCT'), ('他', 'PRON'), ('还', 'ADV'), ('创作', 'VERB'), ('了', 'PART'), ('<遮天>', 'NOUN'), ('。', 'PUNCT')]
  6. """

4.识别停用词 (Stop words)

  1. print([(w.text, w.is_stop) for w in doc])
  2.  
  3.  
  4. """
  5. [('小米', False), ('董事长', False), ('叶凡', False), ('决定', True), ('投资', False), ('华为', False), ('。', True), ('在', True), ('2002年', False), ('，', True), ('他', True), ('还', True), ('创作', False), ('了', True), ('<遮天>', False), ('。', True)]
  6. """

5.命名实体识别 (Named Entity Recognization)

  1. # 命名实体识别 (Named Entity Recognization)
  2. print([(e.text, e.label_) for e in doc.ents])
  3.  
  4.  
  5.  
  6. """
  7. [('小米', 'PERSON'), ('叶凡', 'PERSON'), ('2002年', 'DATE')]
  8. """

6.依存分析 (Dependency Parsing)

  1. print([(w.text, w.dep_) for w in doc])
  2.  
  3.  
  4. """
  5. [('小米', 'nmod:assmod'), ('董事长', 'appos'), ('叶凡', 'nsubj'), ('决定', 'ROOT'), ('投资', 'ccomp'), ('华为', 'dobj'), ('。', 'punct'), ('在', 'case'), ('2002年', 'nmod:prep'), ('，', 'punct'), ('他', 'nsubj'), ('还', 'advmod'), ('创作', 'ROOT'), ('了', 'aux:asp'), ('<遮天>', 'dobj'), ('。', 'punct')]
  6. """

7.词性还原 (Lemmatization)

这个模型没有这个功能,用英文模型演示下

找到单词的原型，即词性还原，将am, is, are, have been 还原成be，复数还原成单数(cats -> cat)，过去时态还原成现在时态 (had -> have)。

  1. import spacy
  2. nlp = spacy.load('en_core_web_sm')
  3.  
  4. txt = "A magnetic monopole is a hypothetical elementary particle."
  5. doc = nlp(txt)
  6.  
  7. lem = [token.lemma_ for token in doc]
  8. print(lem)
  9.  
  10.  
  11. """
  12. ['a', 'magnetic', 'monopole', 'be', 'a', 'hypothetical', 'elementary', 'particle', '.']
  13. """

8.提取名词短语 (Noun Chunks)

这个模型没有这个功能,用英文模型演示下

  1. noun_chunks = [nc for nc in doc.noun_chunks]
  2. print(noun_chunks)
  3.  
  4. """
  5. [A magnetic monopole, a hypothetical elementary particle]
  6. """

9.指代消解 (Coreference Resolution)

指代消解 ，寻找句子中代词 he，she，it 所对应的实体。为了使用这个模块，需要使用神经网络预训练的指代消解系数，如果前面没有安装，可运行命令：pip install neuralcoref

这个模型没有这个功能,用英文模型演示下

  1. txt = "My sister has a son and she loves him."
  2.  
  3. # 将预训练的神经网络指代消解加入到spacy的管道中
  4. import neuralcoref
  5. neuralcoref.add_to_pipe(nlp)
  6.  
  7. doc = nlp(txt)
  8. doc._.coref_clusters
  9.  
  10. """
  11. [My sister: [My sister, she], a son: [a son, him]]
  12. """

三、可视化

  1. from spacy import displacy
  2.  
  3.  
  4.  
  5. # 可视化依存关系
  6. html_str = displacy.render(doc, style="dep")
  7.  
  8.  
  9. #可视化命名名称实体
  10. # html_str = displacy.render(doc, style="ent")
  11.  
  12.  
  13. with open("D:\\data\\ss.html", "w", encoding="utf8") as f:
  14.     f.write(html_str)
  15.  
  16.

html_str  是一个html格式的字符串, 保存到本地 ss.html文件,浏览器打开效果:

依存关系

命名实体

 

官方还有一个可视化的库:  spacy-streamlit , 专门用于spacy相关的nlp可视化。

streamlit 也是一个专门可视化的库。

spacy-streamlit 有一个使用demo:


https://share.streamlit.io/ines/spacy-streamlit-demo/app.py

demo对应githup

GitHub - ines/spacy-streamlit-demo

声明：本文内容由网友自发贡献，转载请注明出处：【wpsshop】
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号