《Python 自然语言处理》学习笔记--第一章：语言处理与python_tokenwrap

NLTK入门与函数

本文将记录书中所述函数，并对报错处做出修正。

安装

从NLTK库中下载文章资料并查看。

>>> import nltk
>>> nltk.download()
1
2

>>> from nltk.book import *
*** Introductory Examples for the NLTK Book ***
Loading text1, ..., text9 and sent1, ..., sent9
Type the name of the text or sentence to view it.
Type: 'texts()' or 'sents()' to list the materials.
text1: Moby Dick by Herman Melville 1851
text2: Sense and Sensibility by Jane Austen 1811
text3: The Book of Genesis
text4: Inaugural Address Corpus
text5: Chat Corpus
text6: Monty Python and the Holy Grail
text7: Wall Street Journal
text8: Personals Corpus
text9: The Man Who Was Thursday by G . K . Chesterton 1908
1
2
3
4
5
6
7
8
9
10
11
12
13
14

>>> text1
<Text: Moby Dick by Herman Melville 1851>
1
2

基础函数

词语索引

>>> text1.concordance("monstrous")
Displaying 11 of 11 matches:
ong the former , one was of a most monstrous size . ... This came towards us ,
ON OF THE PSALMS . " Touching that monstrous bulk of the whale or ork we have r
ll over with a heathenish array of monstrous clubs and spears . Some were thick
d as you gazed , and wondered what monstrous cannibal and savage could ever hav
that has survived the flood ; most monstrous and most mountainous ! That Himmal
they might scout at Moby Dick as a monstrous fable , or still worse and more de
th of Radney .'" CHAPTER 55 Of the Monstrous Pictures of Whales . I shall ere l
ing Scenes . In connexion with the monstrous pictures of whales , I am strongly
ere to enter upon those still more monstrous stories of them which are to be fo
ght have been rummaged out of this monstrous cabinet there is no telling . But
of Whale - Bones ; for Whales of a monstrous size are oftentimes cast up dead u
1
2
3
4
5
6
7
8
9
10
11
12
13

查看那些词出现在类似的上下文

>>> text1.similar("monstrous")
true contemptible christian abundant few part mean careful puzzled
mystifying passing curious loving wise doleful gamesome singular
delightfully perilous fearless
1
2
3
4

研究两个及以上的词共同的上下文

>>> text2.common_contexts(["monstrous","very"])
a_pretty am_glad a_lucky is_pretty be_glad
1
2

词汇分布离散图

>>> text4.dispersion_plot(["citizens","democracy","freedom","duties","America"])
1

产生随机文本（报错）

>>> text3.generate