NLP04-pyLDAvis可视化主题_pyldavis官网

摘要：演示一个pyLDAvis例子，作为入门，结合gensim来进行文本主题分类提供可视化的入门。

1. 安装pyLDAvis

官网：http://pyldavis.readthedocs.io/en/latest/
安装：pip install pyLDAvis
这个安装如果网速不好，会比较慢，这里要安装比较多的依赖包。

2. 例子

这个例子的数据采用前面文章的，http://blog.csdn.net/ld326/article/details/78353338，这里只是重点突出一下主题模型的可视化。

import pyLDAvis.gensim
from gensim import corpora
from gensim.models import LdaModel

def get_corpus_dictionary():
    documents = ["Human machine interface for lab abc computer applications",
                 "A survey of user opinion of computer system response time",
                 "The EPS user interface management system",
                 "System and human system engineering testing of EPS",
                 "Relation of user perceived response time to error measurement",
                 "The generation of random binary unordered trees",
                 "The intersection graph of paths in trees",
                 "Graph minors IV Widths of trees and well quasi ordering",
                 "Graph minors A survey"]
    stoplist = set('for a of the and to in'.split())
    texts = [[word for word in document.lower().split() if word not in stoplist]
             for document in documents]

    from collections import defaultdict
    frequency = defaultdict(int)
    for text in texts:
        for token in text:
            frequency[token] += 1

    texts = [[token for token in text if frequency[token] > 1]
             for text in texts]
    dictionary = corpora.Dictionary(texts)
    corpus = [dictionary.doc2bow(text) for text in texts]

    return corpus, dictionary

def test_lda():
    corpus, dictionary = get_corpus_dictionary()
    lda = LdaModel(corpus=corpus,num_topics=2)
    data = pyLDAvis.gensim.prepare(lda, corpus, dictionary)
    pyLDAvis.show(data,open_browser=False)

if __name__ == "__main__":
    test_lda()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

3. 结果显示

打开web浏览器，
输入：http://127.0.0.1:8888/
至于这个IP与端口是可以修改的，在pyLDAvis.show()方法中可以修改了。
点击显示的内容查看各主题情况：

【作者：happyprince, http://blog.csdn.net/ld326/article/details/78370495】