共查询到20条相似文献，搜索用时 31 毫秒
1.
［目的/意义］在对MNCS和百分位数两种指标机制进行阐述、对比和分析的基础上,对百分位数指标的计算框架进行改进,提出一种动态权重的百分位数指标用于学术影响力的评价。［方法/过程］以ESI学科为评价对象,分别选取同一研究实体下的不同学科和不同研究实体下的同一学科作为两个实例进行实证研究。［结果/结论］实证结果表明这种动态权重的百分位数指标与MNCS和百分位数指标相比更能展现评价对象学术影响力的细节。 相似文献
2.
3.
In the field of scientometrics, impact indicators and ranking algorithms are frequently evaluated using unlabelled test data comprising relevant entities (e.g., papers, authors, or institutions) that are considered important. The rationale is that the higher some algorithm ranks these entities, the better its performance. To compute a performance score for an algorithm, an evaluation measure is required to translate the rank distribution of the relevant entities into a single-value performance score. Until recently, it was simply assumed that taking the average rank (of the relevant entities) is an appropriate evaluation measure when comparing ranking algorithms or fine-tuning algorithm parameters.With this paper we propose a framework for evaluating the evaluation measures themselves. Using this framework the following questions can now be answered: (1) which evaluation measure should be chosen for an experiment, and (2) given an evaluation measure and corresponding performance scores for the algorithms under investigation, how significant are the observed performance differences?Using two publication databases and four test data sets we demonstrate the functionality of the framework and analyse the stability and discriminative power of the most common information retrieval evaluation measures. We find that there is no clear winner and that the performance of the evaluation measures is highly dependent on the underlying data. Our results show that the average rank is indeed an adequate and stable measure. However, we also show that relatively large performance differences are required to confidently determine if one ranking algorithm is significantly superior to another. Lastly, we list alternative measures that also yield stable results and highlight measures that should not be used in this context. 相似文献
4.
5.
Fusion Via a Linear Combination of Scores 总被引：9，自引：2，他引：7
We present a thorough analysis of the capabilities of the linear combination (LC) model for fusion of information retrieval systems. The LC model combines the results lists of multiple IR systems by scoring each document using a weighted sum of the scores from each of the component systems. We first present both empirical and analytical justification for the hypotheses that such a model should only be used when the systems involved have high performance, a large overlap of relevant documents, and a small overlap of nonrelevant documents. The empirical approach allows us to very accurately predict the performance of a combined system. We also derive a formula for a theoretically optimal weighting scheme for combining 2 systems. We introduce d—the difference between the average score on relevant documents and the average score on nonrelevant documents—as a performance measure which not only allows mathematical reasoning about system performance, but also allows the selection of weights which generalize well to new documents. We describe a number of experiments involving large numbers of different IR systems which support these findings. 相似文献
6.
7.
Robin Aly Aiden Doherty Djoerd Hiemstra Franciska de Jong Alan F. Smeaton 《Information Retrieval》2013,16(5):557-583
Concept based video retrieval often relies on imperfect and uncertain concept detectors. We propose a general ranking framework to define effective and robust ranking functions, through explicitly addressing detector uncertainty. It can cope with multiple concept-based representations per video segment and it allows the re-use of effective text retrieval functions which are defined on similar representations. The final ranking status value is a weighted combination of two components: the expected score of the possible scores, which represents the risk-neutral choice, and the scores’ standard deviation, which represents the risk or opportunity that the score for the actual representation is higher. The framework consistently improves the search performance in the shot retrieval task and the segment retrieval task over several baselines in five TRECVid collections and two collections which use simulated detectors of varying performance. 相似文献
8.
基于内容的图像检索技术是对图像的物理内容为加工对象的检索技术之一,主要实现方式包括基于颜色、纹理、形状、空间位置和语义等。其中基于颜色的图像检索发展最为成熟,而基于语义的检索则尚处于探讨、研究阶段。基于内容检索和基于文本检索在数字图书馆中可以融合共同提供检索服务。Google为这一尝试提供了在后控阶段的有效案例,而真正的实现两者的融合是在预处理阶段。两者结合在数字图书馆中的应用是可行的,相信能够提供更好的图像检索服务。 相似文献
9.
Knowledge transfer for cross domain learning to rank 总被引：1，自引：1，他引：0
Depin Chen Yan Xiong Jun Yan Gui-Rong Xue Gang Wang Zheng Chen 《Information Retrieval》2010,13(3):236-253
Recently, learning to rank technology is attracting increasing attention from both academia and industry in the areas of machine
learning and information retrieval. A number of algorithms have been proposed to rank documents according to the user-given
query using a human-labeled training dataset. A basic assumption behind general learning to rank algorithms is that the training
and test data are drawn from the same data distribution. However, this assumption does not always hold true in real world
applications. For example, it can be violated when the labeled training data become outdated or originally come from another
domain different from its counterpart of test data. Such situations bring a new problem, which we define as cross domain learning
to rank. In this paper, we aim at improving the learning of a ranking model in target domain by leveraging knowledge from
the outdated or out-of-domain data (both are referred to as source domain data). We first give a formal definition of the
cross domain learning to rank problem. Following this, two novel methods are proposed to conduct knowledge transfer at feature
level and instance level, respectively. These two methods both utilize Ranking SVM as the basic learner. In the experiments,
we evaluate these two methods using data from benchmark datasets for document retrieval. The results show that the feature-level
transfer method performs better with steady improvements over baseline approaches across different datasets, while the instance-level
transfer method comes out with varying performance depending on the dataset used. 相似文献
10.
[目的/意义]运用心流理论设计量表对信息检索体验进行测量与分析,探索个体在信息检索活动中的情感体验规律。[方法/过程]采用体验抽样法在三所大学图书馆对正在进行信息检索的用户进行问卷调查,利用数据统计分析检验量表的质量。依据调查对象在技巧维度、挑战维度及技巧与挑战平衡维度的得分,把样本划分为心流、焦虑、冷漠和无趣四通道,并比较不同通道的体验质量。[结果/结论]结果显示,该量表是测量信息检索体验的有效工具,技巧与挑战水平及两者的匹配程度是影响信息检索体验的关键变量,技巧与挑战匹配且都处于高水平的心流通道体验最佳,冷漠通道体验最差,焦虑和无趣通道居中。 相似文献
11.
Credibility-inspired ranking for blog post retrieval 总被引：1，自引：0，他引：1
Credibility of information refers to its believability or the believability of its sources. We explore the impact of credibility-inspired indicators on the task of blog post retrieval, following the intuition that more credible blog posts are preferred by searchers. Based on a previously introduced credibility framework for blogs, we define several credibility indicators, and divide them into post-level (e.g., spelling, timeliness, document length) and blog-level (e.g., regularity, expertise, comments) indicators. The retrieval task at hand is precision-oriented, and we hypothesize that the use of credibility-inspired indicators will positively impact precision. We propose to use ideas from the credibility framework in a reranking approach to the blog post retrieval problem: We introduce two simple ways of reranking the top n of an initial run. The first approach, Credibility-inspired reranking, simply reranks the top n of a baseline based on the credibility-inspired score. The second approach, Combined reranking, multiplies the credibility-inspired score of the top n results by their retrieval score, and reranks based on this score. Results show that Credibility-inspired reranking leads to larger improvements over the baseline than Combined reranking, but both approaches are capable of improving over an already strong baseline. For Credibility-inspired reranking the best performance is achieved using a combination of all post-level indicators. Combined reranking works best using the post-level indicators combined with comments and pronouns. The blog-level indicators expertise, regularity, and coherence do not contribute positively to the performance, although analysis shows that they can be useful for certain topics. Additional analysis shows that a relative small value of n (15–25) leads to the best results, and that posts that move up the ranking due to the integration of reranking based on credibility-inspired indicators do indeed appear to be more credible than the ones that go down. 相似文献
12.
Query suggestions have become pervasive in modern web search, as a mechanism to guide users towards a better representation of their information need. In this article, we propose a ranking approach for producing effective query suggestions. In particular, we devise a structured representation of candidate suggestions mined from a query log that leverages evidence from other queries with a common session or a common click. This enriched representation not only helps overcome data sparsity for long-tail queries, but also leads to multiple ranking criteria, which we integrate as features for learning to rank query suggestions. To validate our approach, we build upon existing efforts for web search evaluation and propose a novel framework for the quantitative assessment of query suggestion effectiveness. Thorough experiments using publicly available data from the TREC Web track show that our approach provides effective suggestions for adhoc and diversity search. 相似文献
13.
如何有效的进行生物医学文献检索和信息挖掘,是计算机技术和生物信息技术研究领域中的一个经典课题。本文对生物医学文献中自然语言问题文档,片段,概念和RDF三元组,构建了高效的检索和问答系统。特别的,在文档检索中,我们搭建了基于顺序依赖模型,词向量,和伪相关反馈相结合的通用检索模型;同时,前k个文档被分离为句子和片段,并以此建立检索索引,并基于文档检索模型,完成片段检索;在概念挖掘中,提取生物医学的概念,列出相关的概念属于网络服务的五个数据库链接,通过得分排名得到最终的概念。在CLEF BioASQ几年的评测数据上,我们构造的检索系统都取得了不错的性能。 相似文献
14.
Recently direct optimization of information retrieval (IR) measures has become a new trend in learning to rank. In this paper,
we propose a general framework for direct optimization of IR measures, which enjoys several theoretical advantages. The general
framework, which can be used to optimize most IR measures, addresses the task by approximating the IR measures and optimizing
the approximated surrogate functions. Theoretical analysis shows that a high approximation accuracy can be achieved by the
framework. We take average precision (AP) and normalized discounted cumulated gains (NDCG) as examples to demonstrate how
to realize the proposed framework. Experiments on benchmark datasets show that the algorithms deduced from our framework are
very effective when compared to existing methods. The empirical results also agree well with the theoretical results obtained
in the paper. 相似文献
15.
16.
As retrieval systems become more complex, learning to rank approaches are being developed to automatically tune their parameters. Using online learning to rank, retrieval systems can learn directly from implicit feedback inferred from user interactions. In such an online setting, algorithms must obtain feedback for effective learning while simultaneously utilizing what has already been learned to produce high quality results. We formulate this challenge as an exploration–exploitation dilemma and propose two methods for addressing it. By adding mechanisms for balancing exploration and exploitation during learning, each method extends a state-of-the-art learning to rank method, one based on listwise learning and the other on pairwise learning. Using a recently developed simulation framework that allows assessment of online performance, we empirically evaluate both methods. Our results show that balancing exploration and exploitation can substantially and significantly improve the online retrieval performance of both listwise and pairwise approaches. In addition, the results demonstrate that such a balance affects the two approaches in different ways, especially when user feedback is noisy, yielding new insights relevant to making online learning to rank effective in practice. 相似文献
17.
Computational modelling of music similarity is an increasingly important part of personalisation and optimisation in music information retrieval and research in music perception and cognition. The use of relative similarity ratings is a new and promising approach to modelling similarity that avoids well known problems with absolute ratings. In this article, we use relative ratings from the MagnaTagATune dataset with new and existing variants of state-of-the-art algorithms and provide the first comprehensive and rigorous evaluation of this approach. We compare metric learning based on support vector machines (SVMs) and metric-learning-to-rank (MLR), including a diagonal and a novel weighted variant, and relative distance learning with neural networks (RDNN). We further evaluate the effectiveness of different high and low level audio features and genre data, as well as dimensionality reduction methods, weighting of similarity ratings, and different sampling methods. Our results show that music similarity measures learnt on relative ratings can be significantly better than a standard Euclidian metric, depending on the choice of learning algorithm, feature sets and application scenario. MLR and SVM outperform DMLR and RDNN, while MLR with weighted ratings leads to no further performance gain. Timbral and music-structural features are most effective, and all features jointly are significantly better than any other combination of feature sets. Sharing audio clips (but not the similarity ratings) between test and training sets improves performance, in particular for the SVM-based methods, which is useful for some applications scenarios. A testing framework has been implemented in Matlab and made publicly available http://mi.soi.city.ac.uk/datasets/ir2012framework so that these results are reproducible. 相似文献
18.
In many realistic settings of expert finding, the evidence for expertise often comes from heterogeneous knowledge sources.
As some sources tend to be more reliable and indicative than the others, different information sources need to receive different
weights to reflect their degrees of importance. However, most previous studies in expert finding did not differentiate data
sources, which may lead to unsatisfactory performance in the settings where the heterogeneity of data sources is present.
In this paper, we investigate how to merge and weight heterogeneous knowledge sources in the context of expert finding. A
relevance-based supervised learning framework is presented to learn the combination weights from training data. Beyond just
learning a fixed combination strategy for all the queries and experts, we propose a series of discriminative probabilistic
models which have increasing capability to associate the combination weights with specific experts and queries. In the last
(and also the most sophisticated) proposed model, the combination weights depend on both expert classes and query topics,
and these classes/topics are derived from expert and query features. Compared with expert and query independent combination
methods, the proposed combination strategy can better adjust to different types of experts and queries. In consequence, the
model yields much flexibility of combining data sources when dealing with a broad range of expertise areas and a large variation
in experts. To the best of our knowledge, this is the first work that designs discriminative learning models to rank experts.
Empirical studies on two real world faculty expertise testbeds demonstrate the effectiveness and robustness of the proposed
discriminative learning models. 相似文献
19.
When speaking of information retrieval, we often mean text retrieval. But there exist many other forms of information retrieval applications. A typical example is collaborative filtering that suggests interesting items to a user by taking into account other users’ preferences or tastes. Due to the uniqueness of the problem, it has been modeled and studied differently in the past, mainly drawing from the preference prediction and machine learning view point. A few attempts have yet been made to bring back collaborative filtering to information (text) retrieval modeling and subsequently new interesting collaborative filtering techniques have been thus derived. In this paper, we show that from the algorithmic view point, there is an even closer relationship between collaborative filtering and text retrieval. Specifically, major collaborative filtering algorithms, such as the memory-based, essentially calculate the dot product between the user vector (as the query vector in text retrieval) and the item rating vector (as the document vector in text retrieval). Thus, if we properly structure user preference data and employ the target user’s ratings as query input, major text retrieval algorithms and systems can be directly used without any modification. In this regard, we propose a unified formulation under a common notational framework for memory-based collaborative filtering, and a technique to use any text retrieval weighting function with collaborative filtering preference data. Besides confirming the rationale of the framework, our preliminary experimental results have also demonstrated the effectiveness of the approach in using text retrieval models and systems to perform item ranking tasks in collaborative filtering. 相似文献
20.
个性化信息检索中的相关反馈技术研究 总被引：3，自引：0，他引：3
简要介绍了相关反馈的研究现状及基本思想,在深入分析相关反馈的实现策略和在不同系统中设计的差别后,提出了相关反馈技术和个性化信息检索结合的模型,最后讨论引入数据融合的思想来进一步改善反馈效果。 相似文献