共查询到20条相似文献，搜索用时 921 毫秒
1.
2.
以分布式数据库文本数据为研究对象,提出基于分类特征的改进共享最近邻方法对数据实现去重备份处理。根据文本数据内容先划分到预设定类别中,利用特征词条到实数的映射即特征选择函数进行特征选择,按照一定准则从初始特征中选取具有强分类能力的特征,通过计算某训练集中各个词条特征选择函数参数值,获取参数值低于阈值的词条。利用哈希思想将文本空间相邻2个数据点实现指纹空间变换,转换空间后保持数据点相近,通过共享最近邻方法对相近指纹文本数据聚类,对聚类后数据进行迭代增删处理。实验证明,运用文中方法可对文本数据实现快速去重备份,有效解决数据占用空间问题。 相似文献
3.
4.
渔业文本分类是充分利用渔业信息资源的有效途径。针对中文文献资料的结构特点,提出一种结合特征词权值和支持向量机(Support Vector Machine,SVM)的渔业文本分类方法,利用向量空间模型(Vector Space Model,VSM)构建文本向量空间,并结合特征词权值计算文本特征向量中的各特征项,将构建的文本向量送入SVM进行渔业文本分类。采用中国知网下载的标准文档进行了实验测试,并考察了准确率和召回率两个指标,实验结果表明,文章提出的渔业文本分类方法具有较好的分类效果。 相似文献
5.
6.
提出一种基于云理论和神经网络构造决策树的文本分类方法。运用云神经网络学习变量间的云映射关系,从中生成云决策树。这种方法结合了神经网络的学习算法和决策树的推理方法,具有神经网络的学习能力,并且应用了云发生器对处理不确定性的能力。更符合人类的思维方式,从而进一步提高了文本分类的效率、准确性和可靠性。 相似文献
7.
一种改进的SVM决策树文本分类算法 总被引：1，自引：0，他引：1
将SVM和二叉决策树结合起来构成SVM决策树的方法能够较好地解决多类文本分类问题,在此基础上引入了一种基于支持向量数据描述(SVDD)的类间可分性度量方法,对SVM决策树分类器进行改进,实验表明,该方法有效地提高了SVM决策树多类分类器的分类精度和速度. 相似文献
8.
9.
文本提出了一种基于模糊向量空间模型和径向基函数网络的分类方法。该方法在特征提取时充分考虑了特征项在文档中的位置信息,构造出模糊特征向量,使自动分类更接近手工分类方法。以中国期刊网全文数据库部分文档数据为例验证了该方法的有效性。 相似文献
10.
基于模糊向量空间的文本分类方法 总被引：1，自引：0，他引：1
本文针对文本自动分类问题，提出了一种基于模糊向量空间模型和径向基函数网络的分类方法。网络由输入层、隐层和输出层组成。输入层完成分类样本的输入，隐层提取输入样本所隐含的模式特征，将分类结果在输出层表现出来。该方法在特征提取时充分考虑了特征项在文档中的位置信息，构造出模糊特征向量，使自动分类更接近手工分类方法。以中国期刊网全文数据库部分文档数据为例验证了该方法的有效性。 相似文献
11.
基于改进KNN的文本分类方法 总被引：8，自引：0，他引：8
本文针对VSM (向量空间模型)中KNN (K最近邻算法)在文本处理环境下的不足,根据SOM (自组织映射神经网络)理论、特征选取和模式聚合理论,提出了一种改进的KNN文本分类方法。应用特征选取和模式聚合理论以降低特征空间维数。传统的VSM模型各维相同的权重并不适应于文本处理的环境,本文提出应用SOM神经网络进行VSM模型各维权重的计算。结合两种改进,有效地降低了向量空间的维数,提高了文本分类的精度和速度。 相似文献
12.
Jieming Yang Yuanning Liu Xiaodong Zhu Zhen Liu Xiaoxu Zhang 《Information processing & management》2012
The feature selection, which can reduce the dimensionality of vector space without sacrificing the performance of the classifier, is widely used in text categorization. In this paper, we proposed a new feature selection algorithm, named CMFS, which comprehensively measures the significance of a term both in inter-category and intra-category. We evaluated CMFS on three benchmark document collections, 20-Newsgroups, Reuters-21578 and WebKB, using two classification algorithms, Naïve Bayes (NB) and Support Vector Machines (SVMs). The experimental results, comparing CMFS with six well-known feature selection algorithms, show that the proposed method CMFS is significantly superior to Information Gain (IG), Chi statistic (CHI), Document Frequency (DF), Orthogonal Centroid Feature Selection (OCFS) and DIA association factor (DIA) when Naïve Bayes classifier is used and significantly outperforms IG, DF, OCFS and DIA when Support Vector Machines are used. 相似文献
13.
Most previous works of feature selection emphasized only the reduction of high dimensionality of the feature space. But in cases where many features are highly redundant with each other, we must utilize other means, for example, more complex dependence models such as Bayesian network classifiers. In this paper, we introduce a new information gain and divergence-based feature selection method for statistical machine learning-based text categorization without relying on more complex dependence models. Our feature selection method strives to reduce redundancy between features while maintaining information gain in selecting appropriate features for text categorization. Empirical results are given on a number of dataset, showing that our feature selection method is more effective than Koller and Sahami’s method [Koller, D., & Sahami, M. (1996). Toward optimal feature selection. In Proceedings of ICML-96, 13th international conference on machine learning], which is one of greedy feature selection methods, and conventional information gain which is commonly used in feature selection for text categorization. Moreover, our feature selection method sometimes produces more improvements of conventional machine learning algorithms over support vector machines which are known to give the best classification accuracy. 相似文献
14.
15.
16.
17.
SOM聚类算法在文本分类上的应用 总被引：2，自引：0，他引：2
随着网络信息指数级的增长，如何高效地组织海量的文本信息成为众多终端信息查询的基本要求。本文利用神经网络的联想记忆原理，提出一种改进自组织映射（SOM）神经网络聚类算法来对这些信息进行索引和分类。改进SOM聚类算法通过文本的预处理和词汇权值的计算，SOM网络的训练过程以及多次聚类来细化各文本类别，最终产生概念空间。试验结果表明该算法对文本有很好的分类管理功能，便于文本检索。 相似文献
18.
In text categorization, it is quite often that the numbers of documents in different categories are different, i.e., the class distribution is imbalanced. We propose a unique approach to improve text categorization under class imbalance by exploiting the semantic context in text documents. Specifically, we generate new samples of rare classes (categories with relatively small amount of training data) by using global semantic information of classes represented by probabilistic topic models. In this way, the numbers of samples in different categories can become more balanced and the performance of text categorization can be improved using this transformed data set. Indeed, the proposed method is different from traditional re-sampling methods, which try to balance the number of documents in different classes by re-sampling the documents in rare classes. Such re-sampling methods can cause overfitting. Another benefit of our approach is the effective handling of noisy samples. Since all the new samples are generated by topic models, the impact of noisy samples is dramatically reduced. Finally, as demonstrated by the experimental results, the proposed methods can achieve better performance under class imbalance and is more tolerant to noisy samples. 相似文献
19.
Wanpeng Song Liu Wenyin Naijie Gu Xiaojun Quan Tianyong Hao 《Information processing & management》2011
Question categorization, which suggests one of a set of predefined categories to a user’s question according to the question’s topic or content, is a useful technique in user-interactive question answering systems. In this paper, we propose an automatic method for question categorization in a user-interactive question answering system. This method includes four steps: feature space construction, topic-wise words identification and weighting, semantic mapping, and similarity calculation. We firstly construct the feature space based on all accumulated questions and calculate the feature vector of each predefined category which contains certain accumulated questions. When a new question is posted, the semantic pattern of the question is used to identify and weigh the important words of the question. After that, the question is semantically mapped into the constructed feature space to enrich its representation. Finally, the similarity between the question and each category is calculated based on their feature vectors. The category with the highest similarity is assigned to the question. The experimental results show that our proposed method achieves good categorization precision and outperforms the traditional categorization methods on the selected test questions. 相似文献
20.
Cross-lingual text categorization: Conquering language boundaries in globalized environments 总被引：1，自引：0，他引：1
Text categorization pertains to the automatic learning of a text categorization model from a training set of preclassified documents on the basis of their contents and the subsequent assignment of unclassified documents to appropriate categories. Most existing text categorization techniques deal with monolingual documents (i.e., written in the same language) during the learning of the text categorization model and category assignment (or prediction) for unclassified documents. However, with the globalization of business environments and advances in Internet technology, an organization or individual may generate and organize into categories documents in one language and subsequently archive documents in different languages into existing categories, which necessitate cross-lingual text categorization (CLTC). Specifically, cross-lingual text categorization deals with learning a text categorization model from a set of training documents written in one language (e.g., L1) and then classifying new documents in a different language (e.g., L2). Motivated by the significance of this demand, this study aims to design a CLTC technique with two different category assignment methods, namely, individual- and cluster-based. Using monolingual text categorization as a performance reference, our empirical evaluation results demonstrate the cross-lingual capability of the proposed CLTC technique. Moreover, the classification accuracy achieved by the cluster-based category assignment method is statistically significantly higher than that attained by the individual-based method. 相似文献