| 数据挖掘中Web文档转换算法的设计与实现 |
| |
| 引用本文： | 赵小龙,佘东.数据挖掘中Web文档转换算法的设计与实现[J].巢湖学院学报,2011(6):34-38. |
| |
| 作者姓名： | 赵小龙 佘东 |
| |
| 作者单位： | 安徽工业经济学院,安徽合肥,230051 |
| |
| 基金项目： | 安徽省高校优秀人才基金项目，巢湖学院一般项目，安徽工业经济学院《学院科研管理信息系统开发研究》自然科学基金项目支持 |
| |
| 摘 要： | Web文本挖掘是数据挖掘技术在网络信息处理中的一个重要应用,如何将web文档转换成数据挖掘所要求的格式,即web文档预处理是一项很重要的研究课题.本文的方法是:从Internet网上下载了大量的网页文件,将网页文件转换成文本文件,然后通过算法对这些文本文件中的数据进行词频统计,删除非用词,去掉高频词,对单词进行词根处理,建立用词词表,从而抽取用词,按字母排序生成词频索引,和字典文件进行对照,获取单词的ID,最后生成Reuters-21578的Database数据格式.这样就将web文档数据转换成标准的数据集,以便为数据挖掘中分类、聚类作好准备.
|
| 关 键 词： | Web文档 数据挖掘 预处理 |
DESIGN AND IMPLEMENTATION OF WEB DOCUMENTS CONVERSION ALGORITHM IN DATA MINING |
| |
| ZHAO Xiao-long SHE Dong.DESIGN AND IMPLEMENTATION OF WEB DOCUMENTS CONVERSION ALGORITHM IN DATA MINING[J].Chaohu College Journal,2011(6):34-38. |
| |
| Authors: | ZHAO Xiao-long SHE Dong |
| |
| Institution: | ZHAO Xiao-long SHE Dong (Anhui Institute of Industrial Economy,Hefei Anhui 230051) |
| |
| Abstract: | Web text information mining is one of the important applications of applying data mining technologies into informa- tion analysis and processing, how to transform web documents into data mining to the required format, i.e. web document pre- processing becomes a significant research task. In this paper the method is ： from Internet to download a large number of web- page files, webpage files are converted into a text files, and then through the algorithm to word frequency statistics the data of the text files, delete non-using words, remove high frequency words, process etyma of substantive words, extract stems, elimi- nate redundant words and establish word lis4 thus extraction word list, alphabetical index to generate word frequency index, and the dictionary file comparison, get the word ID, the last generation of Reuters-21578 Database data format. This web docu ment data converted into standard data sets for classification and clustering to prepare in data mining. |
| |
| Keywords: | web documents data mining preprocessing |
| 本文献已被 CNKI 万方数据 等数据库收录！ |
|
