| 不同特征对文本聚类效果的比较研究——以新闻文本为例 |
| |
| 引用本文： | 张旭,孙玉伟,成颖.不同特征对文本聚类效果的比较研究——以新闻文本为例[J].情报理论与实践,2020,43(1):169-176. |
| |
| 作者姓名： | 张旭 孙玉伟 成颖 |
| |
| 作者单位： | 南京大学信息管理学院,江苏 南京 210023;南京大学信息管理学院,江苏 南京 210023;山东师范大学图书馆,山东 济南 250014;南京大学信息管理学院,江苏 南京 210023;山东师范大学文学院,山东 济南250014 |
| |
| 基金项目： | 国家社会科学基金重大招标项目“中国近现代文学期刊全文数据库建设与研究(1872—1949)”的成果之一,项目编号:17ZDA276 |
| |
| 摘 要： | 目的/意义]通过实验分析不同特征提取算法对新闻文本聚类效果的影响。方法/过程]选取搜狗实验室的搜狐新闻语料库以及澳大利亚广播公司2003-2017年间的新闻标题语料库,对TF-IDF、Word2vec以及Doc2vec三种单一特征,TF-IDF+Word2vec、TF-IDF+Doc2vec、Word2vec+Doc2vec以及TF-IDF+Word2vec+Doc2vec四种组合特征在K-means、凝聚以及DBSCAN算法上分别进行聚类分析,通过Purity以及NMI两个评测指标对聚类效果进行评价。结果/结论]单类特征中三个特征的聚类质量呈Word2vec> TF-IDF> Doc2vec关系;组合特征中TF-IDF+Word2vec的效果最优。Word2vec在单一特征中的表现最优,其也是不同组合特征间差异的主要因素,特征组合是否可以提升聚类性能需基于多因素进行综合判定。
|
| 关 键 词： | TF-IDF Word2Vec Doc2Vec 文本聚类 比较研究 聚类分析 |
Comparative Study on the Effect of Different Features on Text Clustering: Take News Text as an Example |
| |
| Abstract: | Purpose/significance] This paper compares and analyzes the effects of different feature extraction algorithms on the clustering results of news texts. Method/process] We select the Sohu news corpus and a million news headlines from the reputable Australian news source ABC( Australian Broadcasting Corp) during 2003-2017 for experiment. Three single features of TFIDF,Word2 vec and Doc2 vec and four combined features of TF-IDF + Word2 vec,TF-IDF + Doc2 vec,Word2 vec + Doc2 vec and TF-IDF + Word2 vec + Doc2 vec were used for this study. Then,we used K-means,Agglomerative and DBSCAN for clustering analysis,and the clustering effects were evaluated by Purity and NMI. Result/conclusion] The clustering quality of the three single features is: Word2 vec > TF-IDF > Doc2 vec. The effect of TF-IDF + Word2 vec in the combination features is the best. This means that Word2 vec performs best when using a single feature,which is also the main factor of the difference between different combinations of features. Whether the feature combination can improve the clustering performance needs to be verified based on multiple factors. |
| |
| Keywords: | TF-IDF Word2Vec Doc2Vec text clustering comparative study cluster analysis |
| 本文献已被 维普 万方数据 等数据库收录！ |
|
