共查询到20条相似文献，搜索用时 46 毫秒
1.
Research evaluation, which is an increasingly pressing issue, invariably relies on citation counts. In this contribution we highlight two concerns that the research community needs to pay attention to. One, in the world of search engine facilitated research, factors such as ease of Web discovery, ease of access, and content relevance, rather than quality, influence what gets read and cited. Two, research evaluation based on citation counts works against many types of high-quality works. We also elaborate on the implications of these points by examining a recent nationwide evaluation of researchers performed in Italy. We focus on our discipline (computer science), but we believe that our observations have relevance for a broad audience. 相似文献
2.
Linda Schamber Michael B. Eisenberg Michael S. Nilan 《Information processing & management》1990,26(6)
Although relevance judgments are fundamental to the design and evaluation of all information retrieval systems, information scientists have not reached a consensus in defining the central concept of relevance. In this paper we ask two questions: What is the meaning of relevance? and What role does relevance play in information behavior? We attempt to address these questions by reviewing literature over the last 30 years that presents various views of relevance as topical, user-oriented, multidimensional, cognitive, and dynamic. We then discuss traditional assumptions on which most research in the field has been based and begin building a case for an approach to the problem of definition based on alternative assumptions. The dynamic, situational approach we suggest views the user — regardless of system — as the central and active determinant of the dimensions of relevance. We believe that relevance is a multidimensional concept; that it is dependent on both internal (cognitive) and external (situational) factors; that it is based on a dynamic human judgment process; and that it is a complex but systematic and measurable phenomenon. 相似文献
3.
Dag Elgesem 《Ethics and Information Technology》2002,4(3):195-203
In the analysis of the ethicalproblems of online research, there is much tobe learned from the work that has already beendone on research ethics in the socialsciences and the humanities. I discuss thestructure of norms in the Norwegian ethicalguidelines for research in the social scienceswith respect to their relevance for the ethicalissues of Internet research. A four-stepprocedure for the ethical evaluation ofresearch is suggested. I argue that eventhough, at one level, the problems of onlineresearch are very similar to those we find intraditional areas of social scientificresearch, there still are some issues that areunique to research online. A general model forthe analysis of privacy and data protection issuggested. This model is then used tocharacterize the special problems pertaining tothe protection of privacy in online contexts,and to argue that one cannot assume a simpledistinction between the private and the publicwhen researching in such contexts. 相似文献
4.
Some of the most popular measures to evaluate information filtering systems are usually independent of the users because they are based in relevance judgments obtained from experts. On the other hand, the user-centred evaluation allows showing the different impressions that the users have perceived about the system running. This work is focused on discussing the problem of user-centred versus system-centred evaluation of a Web content personalization system where the personalization is based on a user model that stores long term (section, categories and keywords) and short term interests (adapted from user provided feedback). The user-centred evaluation is based on questionnaires filled in by the users before and after using the system and the system-centred evaluation is based on the comparison between ranking of documents, obtained from the application of a multi-tier selection process, and binary relevance judgments collected previously from real users. The user-centred and system-centred evaluations performed with 106 users during 14 working days have provided valuable data concerning the behaviour of the users with respect to issues such as document relevance or the relative importance attributed to different ways of personalization. The results obtained shows general satisfaction on both the personalization processes (selection, adaptation and presentation) and the system as a whole. 相似文献
5.
[目的/意义]大数据时代下,移动图书馆对用户阅读方式和获取知识具有重要作用。在用户感知移动图书馆服务质量需求由分散转向关联,服务质量评价过程由精确转向模糊多元的情形下,文章提出了一种模糊Kano模型与犹豫模糊Taguchi方法为核心的综合评价方法。[方法/过程]首先,根据用户与决策者的偏好信息以模糊Kano修正指标重要度,分析用户的直接需求。其次,根据评价过程中的不确定性及指标的关联冲突,提出一种犹豫模糊Kano-Taguchi方法,该方法通过对主客观综合重要度的分析体现了用户与决策者对移动图书馆的偏好程度。最后,以五所云南高校为例进行算例分析,且通过对比分析验证本研究方法的科学性与合理性。[结果/结论]研究表明,文章提出的方法不仅简便明了,而且客观充足。 相似文献
6.
相关性基础理论及其在检索建模中的作用研究 总被引：1，自引：0，他引：1
本文是对信息检索的一次理论研究。在总结了已有相关性研究的基础上,对信息检索模型之中的相关性因素做了系统梳理,认为现有信息检索模型中的系统相关性因素体现得不十分明显,同时用户相关性的因素没有很好地纳入系统相关性和系统设计研究之中。与相关性有关的概念是相似,它存在于文本空间之中。与相关相比,相似具有更好的数学特征。查询是相关判断的过程载体,它同时也是信息检索研究的瓶颈。寻找更为恰当的相关性的隐喻则需要跳出文本的藩篱,从更为深入的模式相关切入,探索更为复杂的相关性因素。 相似文献
7.
《Information processing & management》2020,57(1):102132
The research field of crisis informatics examines, amongst others, the potentials and barriers of social media use during disasters and emergencies. Social media allow emergency services to receive valuable information (e.g., eyewitness reports, pictures, or videos) from social media. However, the vast amount of data generated during large-scale incidents can lead to issue of information overload. Research indicates that supervised machine learning techniques are suitable for identifying relevant messages and filter out irrelevant messages, thus mitigating information overload. Still, they require a considerable amount of labeled data, clear criteria for relevance classification, a usable interface to facilitate the labeling process and a mechanism to rapidly deploy retrained classifiers. To overcome these issues, we present (1) a system for social media monitoring, analysis and relevance classification, (2) abstract and precise criteria for relevance classification in social media during disasters and emergencies, (3) the evaluation of a well-performing Random Forest algorithm for relevance classification incorporating metadata from social media into a batch learning approach (e.g., 91.28%/89.19% accuracy, 98.3%/89.6% precision and 80.4%/87.5% recall with a fast training time with feature subset selection on the European floods/BASF SE incident datasets), as well as (4) an approach and preliminary evaluation for relevance classification including active, incremental and online learning to reduce the amount of required labeled data and to correct misclassifications of the algorithm by feedback classification. Using the latter approach, we achieved a well-performing classifier based on the European floods dataset by only requiring a quarter of labeled data compared to the traditional batch learning approach. Despite a lesser effect on the BASF SE incident dataset, still a substantial improvement could be determined. 相似文献
8.
Javier Parapar Alejandro Bellogín Pablo Castells Álvaro Barreiro 《Information processing & management》2013
Relevance-Based Language Models, commonly known as Relevance Models, are successful approaches to explicitly introduce the concept of relevance in the statistical Language Modelling framework of Information Retrieval. These models achieve state-of-the-art retrieval performance in the pseudo relevance feedback task. On the other hand, the field of recommender systems is a fertile research area where users are provided with personalised recommendations in several applications. In this paper, we propose an adaptation of the Relevance Modelling framework to effectively suggest recommendations to a user. We also propose a probabilistic clustering technique to perform the neighbour selection process as a way to achieve a better approximation of the set of relevant items in the pseudo relevance feedback process. These techniques, although well known in the Information Retrieval field, have not been applied yet to recommender systems, and, as the empirical evaluation results show, both proposals outperform individually several baseline methods. Furthermore, by combining both approaches even larger effectiveness improvements are achieved. 相似文献
9.
In this paper results from three studies examining 1295 relevance judgments by 36 information retrieval (IR) system end-users is reported. Both the region of the relevance judgments, from non-relevant to highly relevant, and the motivations or levels for the relevance judgments are examined. Three major findings are studied. First, the frequency distributions of relevance judgments by IR system end-users tend to take on a bi-modal shape with peaks at the extremes (non-relevant/relevant) with a flatter middle range. Second, the different type of scale (interval or ordinal) used in each study did not alter the shape of the relevance frequency distributions. And third, on an interval scale, the median point of relevance judgment distributions correlates with the point where relevant and partially relevant items begin to be retrieved. The median point of a distribution of relevance judgments may provide a measure of user/IR system interaction to supplement precision/recall measures. The implications of investigation for relevance theory and IR systems evaluation are discussed. 相似文献
10.
构建高效规范的科技评价体系是我国科技治理体系和治理能力现代化的重要标志.以中国知网数据库2017-2019年科技评价领域的期刊论文为对象,分析近年来的研究热点分布及演化特征、主要研究机构及合作关系.研究发现:我国科技评价领域的研究热点包括高质量科技成果评价、学科评估、绩效评价、多元化评价方法等,符合国家科技评价改革方向;领域宽泛但缺少统一的学术语言和共同的认知理念,给学术研究和评价实践带来挑战;对评价方法的探索已成为主要研究方向,定量分析、大数据分析以及针对"分类评价""破四唯"等改革导向的分析已成趋势;研究机构以高校为主,学术研究与实践工作存在一定脱节.对未来我国科技评价体系的改革提出四点建议,包括坚持政策引导、建立行业共识、加强方法探索、促进研用结合等. 相似文献
11.
随着中国制造型企业数字化转型升级的不断深入,衡量中国制造型企业数字化水平,寻找企业数字化能力提升关键因素愈加重要。从战略、运营技术、文化组织能力、生态圈4个维度构建制造型企业数字化成熟度评价模型,并深入阐述基于层次分析(AHP)-决策试验与评价实验室(DEMATEL)方法的某商用车企数字化成熟度评价的详细过程。研究表明,案例车企的数字化成熟度分值为1.00分,尚处于数字化升级者阶段,以实现客户价值为中心、与业务战略关联性、生产质量管理、劳动力效率、资产利用率是影响企业数字化能力提升的核心因素和未来的重点提升方向。 相似文献
12.
Ruihua Song Qingwei Guo Ruochi Zhang Guomao Xin Ji-Rong Wen Yong Yu Hsiao-Wuen Hon 《Information processing & management》2011
In the traditional evaluation of information retrieval systems, assessors are asked to determine the relevance of a document on a graded scale, independent of any other documents. Such judgments are absolute judgments. Learning to rank brings some new challenges to this traditional evaluation methodology, especially regarding absolute relevance judgments. Recently preferences judgments have been investigated as an alternative. Instead of assigning a relevance grade to a document, an assessor looks at a pair of pages and judges which one is better. In this paper, we generalize pairwise preference judgments to relative judgments. We formulate the problem of relative judgments in a formal way and then propose a new strategy called Select-the-Best-Ones to solve the problem. Through user studies, we compare our proposed method with a pairwise preference judgment method and an absolute judgment method. The results indicate that users can distinguish by about one more relevance degree when using relative methods than when using the absolute method. Consequently, the relative methods generate 15–30% more document pairs for learning to rank. Compared to the pairwise method, our proposed method increases the agreement among assessors from 95% to 99%, while halving the labeling time and the number of discordant pairs to experts’ judgments. 相似文献
13.
文章通过灰色系统理论中的递进时段灰色关联度分析技术,利用2009~2017年我国全球创新指数及7个二级影响指标相关数据,以5年时间为子时段长度计算全球创新指数序列与各影响指标序列之间的灰色关联度,并根据计算结果,从定量的角度分析不同指标对我国全球创新指数的影响程度,最后得出这些指标对全球创新指数的灰色关联度排序,结果表明人力资本和研究、基础设施和商业成熟度这三项指标灰色关联度综合排名最高,是影响中国全球创新指数排名的关键因素,应着重从这几方面推动我国创新驱动发展战略的全面实施。 相似文献
14.
15.
16.
Tadeusz Radecki 《Information processing & management》1976,12(5):319-326
The object of this paper is to present a new kind of approach to the problem of information system effectiveness evaluation as based on the theory of fuzzy sets. On the basis of this theory, the concepts of relevance and pertinence, which are the basic concepts used in determining the indices of information system effectiveness evaluation, have been defined. Assuming that in evaluating the effectiveness of information systems, one should consider separately the problem of quality evaluation of the transformation of the contents of documents and information requests into their search patterns and the problem of quality evaluation of the process of profile control of a document set of the information system, definitions have been given of parameters of quality evaluation of the transformation of the contents of documents and information requests into their search patterns with regard to a given information request as well as of parameters of quality evaluation of the process with regard to the whole set of information requests under examination. Besides, parameters of quality evaluation of the process of profile control of a document set of the information system have been defined. The parameters of effectiveness evaluation of information systems put forward in this paper take account of the fact that both evaluation of the relevance and evaluation of the pertinence of documents are of a continuous character. 相似文献
17.
随着科技的不断更新,图书馆的服务内容不断创新,高校的移动数字图书馆服务成为了人们关注的焦点。如何提高用户的服务质量满意度,更好的满足用户需求成为急需解决的问题。本文针对高校移动数字图书馆服务质量统计数据质量评估问题,综合若干国家统计部门与国际组织的统计质量,确立高校移动数字图书馆服务统计数据质量维度测评体系。基于该量表的问卷调查结果显示,本研究的测量问项基本可以通过信度与效度检验,并进行用户重要性认知的满意度检验和不同数据以来程度的用户满意度检验。本文构建的质量维度及测评量表的设计是较为合理的,调查得到的数据可以用于进一步的研究分析。 相似文献
18.
Cristina Lpez-Pujalte Vicente P. Guerrero Bote Flix de Moya Anegn 《Information processing & management》2002,38(6):971
There have been recent applications of genetic algorithms to information retrieval, mostly with respect to relevance feedback. Nevertheless, they are yet to be evaluated in a way that allows them to be compared with each other and with other relevance feedback techniques. We here implement the different genetic algorithms that have been applied in the literature together with some of our own variations, and evaluate them using the residual collection method described by Salton in 1990 for the evaluation of relevance feedback techniques. We compare the results with those of the Ide dec-hi method, which is one of the traditional methods that yields the best results. 相似文献
19.
《Information processing & management》2003,39(2):215-231
Recent studies suggest that significant improvement in information retrieval performance can be achieved by combining multiple representations of an information need. The paper presents a genetic approach that combines the results from multiple query evaluations. The genetic algorithm aims to optimise the overall relevance estimate by exploring different directions of the document space. We investigate ways to improve the effectiveness of the genetic exploration by combining appropriate techniques and heuristics known in genetic theory or in the IR field. Indeed, the approach uses a niching technique to solve the relevance multimodality problem, a relevance feedback technique to perform genetic transformations on query formulations and evolution heuristics in order to improve the convergence conditions of the genetic process. The effectiveness of the global approach is demonstrated by comparing the retrieval results obtained by both genetic multiple query evaluation and classical single query evaluation performed on a subset of TREC-4 using the Mercure IRS. Moreover, experimental results show the positive effect of the various techniques integrated to our genetic algorithm model. 相似文献
20.
Performance-based university research funding systems 总被引：1，自引：0，他引：1
Diana Hicks 《Research Policy》2012,41(2):251-261
The university research environment has been undergoing profound change in recent decades and performance-based research funding systems (PRFSs) are one of the many novelties introduced. This paper seeks to find general lessons in the accumulated experience with PRFSs that can serve to enrich our understanding of how research policy and innovation systems are evolving. The paper also links the PRFS experience with the public management literature, particularly new public management, and understanding of public sector performance evaluation systems. PRFSs were found to be complex, dynamic systems, balancing peer review and metrics, accommodating differences between fields, and involving lengthy consultation with the academic community and transparency in data and results. Although the importance of PRFSs seems based on their distribution of universities’ research funding, this is something of an illusion, and the literature agrees that it is the competition for prestige created by a PRSF that creates powerful incentives within university systems. The literature suggests that under the right circumstances a PRFS will enhance control by professional elites. PRFSs since they aim for excellence, may compromise other important values such as equity or diversity. They will not serve the goal of enhancing the economic relevance of research. 相似文献