共查询到20条相似文献，搜索用时 437 毫秒
1.
《Information processing & management》2022,59(5):103007
Re-using research resources is essential for advancing knowledge and developing repeatable, empirically solid experiments in scientific fields, including interactive information retrieval (IIR). Despite recent efforts on standardizing research re-use and documentation, how to quantitatively measure the reusability of IIR resources still remains an open challenge. Inspired by the reusability evaluations on Cranfield experiments, our work proactively explores the problem of measuring IIR test collection reusability and makes threefold contributions: (1) constructing a novel usefulness-oriented framework with specific analytical methods for evaluating the reusability of IIR test collections consisting of query sets, document/page sets, and sets of task-document usefulness (tuse); (2) explaining the potential impacts of varying IIR-specific factors (e.g. search tasks, sessions, user characteristics) on test collection reusability; (3) proposing actionable methods for building reusable test collections in IIR and thereby amortizing the true cost of user-oriented evaluations. The Cranfield-inspired reusability assessment framework serves as an initial step towards accurately evaluating the reusability of IIR research resources and measuring the reproducibility of IIR evaluation results. It also demonstrates an innovative approach to integrating the insights from individual heterogeneous user studies with the evaluation techniques developed in standardized ad hoc retrieval experiments, which will facilitate the maturation of IIR fields and eventually benefits both sides of research. 相似文献
2.
Charles Cole 《Information processing & management》1998,34(6):709-720
The research examines the notion that the principles underlying the procedure used by doctors to diagnose a patient's disease are useful in the design of “intelligent” IR systems because the task of the doctor is conceptually similar to the computer (or human) intermediary's task in “intelligent information retrieval”: to draw out, through interaction with the IR system, the user's query/information need. The research is reported in two parts. In Part II, an information retrieval tool is described which is based on “intelligent information retrieval” assumptions about the information user. In Part I, presented here, the theoretical framework for the tool is set out. This framework is borrowed from the diagnostic procedure currently used in medicine, called “differential diagnosis”. Because of the severe consequences that attend misdiagnosis, the operating principle in differential diagnosis is (1) to expand the uncertainty in the diagnosis situation so that all possible hypotheses and evidence are considered, then (2) to contract the uncertainty in a step by step fashion (from an examination of the patient's symptoms, through the patient's history and a physical (signs), to laboratory tests). The IR theories of Taylor, Kuhlthau and Belkin are used to demonstrate that these medical diagnosis procedures are already present in IR and that it is a viable model with which to design “intelligent” IR tools and systems. 相似文献
3.
One difficult problem in information retrieval (IR) is the proper interpretation of user queries. It is extremely hard for users to express their information needs in a specific yet exhaustive way. In an effort to alleviate this problem, two theoretical models have been proposed to utilize user characteristics maintained in the form of a user profile. Although the idea of integrating user profiles into an IR system is intuitively appealing, and the models seem viable, no research to date has established a foundation for the roles of user profiles in such a system. Aiming at the investigation of the roles of user profiles, therefore, this study first identifies and extends various query/profile interaction models to provide a ground upon which the investigation can be undertaken. From a continuum of models characterized on the basis of interaction types, metrics, and parameters, nearly 400 models are chosen to investigate the “model space.” New measures are developed based on the notion of user satisfaction/frustration. In addition, three different criteria are used to guide users in making judgments on the quality of retrieved items. Analysis of the data obtained from the experiments shows that, for a wide variety of criteria and metrics, there are always some query/profile interaction models that outperform the query alone model. In addition, preferable characteristics for different criteria are identified in terms of interaction types, parameters, and metrics. 相似文献
4.
Information retrieval systems consist of many complicated components. Research and development of such systems is often hampered by the difficulty in evaluating how each particular component would behave across multiple systems. We present a novel integrated information retrieval system—the Query, Cluster, Summarize (QCS) system—which is portable, modular, and permits experimentation with different instantiations of each of the constituent text analysis components. Most importantly, the combination of the three types of methods in the QCS design improves retrievals by providing users more focused information organized by topic.We demonstrate the improved performance by a series of experiments using standard test sets from the Document Understanding Conferences (DUC) as measured by the best known automatic metric for summarization system evaluation, ROUGE. Although the DUC data and evaluations were originally designed to test multidocument summarization, we developed a framework to extend it to the task of evaluation for each of the three components: query, clustering, and summarization. Under this framework, we then demonstrate that the QCS system (end-to-end) achieves performance as good as or better than the best summarization engines.Given a query, QCS retrieves relevant documents, separates the retrieved documents into topic clusters, and creates a single summary for each cluster. In the current implementation, Latent Semantic Indexing is used for retrieval, generalized spherical k-means is used for the document clustering, and a method coupling sentence “trimming” and a hidden Markov model, followed by a pivoted QR decomposition, is used to create a single extract summary for each cluster. The user interface is designed to provide access to detailed information in a compact and useful format.Our system demonstrates the feasibility of assembling an effective IR system from existing software libraries, the usefulness of the modularity of the design, and the value of this particular combination of modules. 相似文献
5.
[目的/意义]旨在为信息检索相关性研究提供参考。[方法/过程]以CNKI为数据源,采用定性方法,从信息检索的历史脉络和研究学派进行梳理总结,分析信息检索的影响因素和发展趋势。[结果/结论]信息检索相关性是用户、系统的相关性的综合体,任何一方都不能脱离。相关性应该是以用户为关键,系统为基础,研究用户与检索系统的交互、认知以及真实需求的描述与反馈。随着信息检索相关性研究的深入,系统观与用户观将会相互交融,检索技术与用户需求将会协调统一,共同推进检索相关性的发展。 相似文献
6.
ERLI was asked by the French TELECOM to develop a specific system to query the professional headings of the French Yellow Pages directory. Approximately 4 million end users now have access (via their “Minitel” terminals) to some 6 million professionals registered under 2500 different headings. (A second application has also been developed using a similar system: the Minitel Applications Directory, which gives information on all the available applications in the Minitel network.)Although the retrieval of a heading is a necessary step in accessing data, it is of no real interest to the user, who wishes only to retrieve the phone number of a given professional or tradesperson.The general aims of the Natural Language System (NLS) are to facilitate access to headings by intelligent query processing (or even to bypass completely the necessity of choosing between headings).This is done through: • The association of a specific knowledge base to the list of headings, • The construction of a “grammar” ensuring a consistent interpretation of the queries.ERLI's system is as an alternative to the existing one, which is based on a key-word indexing technique. The weaknesses and insufficiencies of such a technique are well known, especially in this context, where queries are expressed by unqualified users, who are unfamiliar with the data (i.e., the headings of the directory).Finally, it is important to note that the NLS was developed with regard to industrial considerations (in particular, the minimizing of the average processing time per query). The system is not a prototype. Extensive on-side testing is scheduled to begin in July 1988 and a complete installation will be carried out at the end of the year. 相似文献
7.
Peretz Shoval 《Information processing & management》1985,21(6):475-487
An expert system was developed in the area of information retrieval, with the objective of performing the job of an information specialist, who assists users in selecting the right vocabulary terms for a database search.The system is composed of two components: One is the knowledge base, represented as a semantic network, in which the nodes are words, concepts, phrases, comprising a vocabulary of the application area and the links express semantic relationships between those nodes. The second component is the rules, or procedures, which operate upon the knowledge-base, analogous to the decision rules or work patterns of the information specialist.Two major stages comprise the consulting process of the system: During the “search” stage relevant knowledge in the semantic network is activated, and search and evaluation rules are applied in order to find appropriate vocabulary terms to represent the user's problem. During the “suggest” stage those terms are further evaluated, dynamically rank-ordered according to relevancy, and suggested to the user. Explanations to the findings can be provided by the system and backtracking is possible in order to find alternatives in case some suggested term is rejected by the user.This article presents the principle, procedures and rules which are utilized in the expert system. 相似文献
8.
This paper investigates the influence of user characteristics (e.g. search experience and cognitive skills) on user effectiveness. A user study was conducted to investigate this effect, 56 participants completed searches for 56 topics using the TREC test collection. Results indicated that participants with search experience and high cognitive skills were more effective than those with less experience and slower perceptual abilities. However, all users rated themselves with the same level of satisfaction with the search results despite the fact they varied substantially in their effectiveness. Therefore, information retrieval evaluators should take these factors into consideration when investigating the impact of system effectiveness on user effectiveness. 相似文献
9.
Caryl McAllister 《Information processing & management》1975,11(8-12)
FACTS is an APL-based interactive on-line system used for retrieval of budget and accounting data. The system provides selective retrieval and manipulation of financial data for management in a development laboratory. The terms “teilnehmer” and “teilhaber” are defined and it is argued that use of a teilnehmer system, such as APL, can considerably reduce the programming and monitary investment for information science systems applications. A brief discussion of APL's text editing facilities is also included to introduce this relatively unknown language to information scientists. 相似文献
10.
The use of geometrical factors to locate information centers for a spatially distributed user population will be shown. The total amount of information for the community of users is considered to be predetermined. A proportion of that information is to be allocated to each information center created. An optimal user versus distance and contents of the center compromise will be obtained using standard mathematical programming techniques. An interesting theoretical situation results for those cases where the “satisfaction benefit” due to quantity of information increases more slowly than the quantity of information. For such cases, the optimal decentralization (or pluralization) is no decentralization at all—a single location results. A case study locating the Mathematics information of a University concludes the work. 相似文献
11.
The problem of content-based video retrieval continues to pose a challenge to the research community, the performance of video retrieval systems being low due to the semantic gap. In this paper we consider whether taking advantage of context can aid the video retrieval process by making the prediction of relevance easier, i.e. if it is easier for a classification system to predict the relevance of a video shot under a given context, then that context has potential in also improving retrieval, since the underlying features better differentiate relevant from non-relevant video shots. We use an operational definition of context, where datasets can be split into disjoint sub-collections which reflect a particular context. Contexts considered include task difficulty and user expertise, among others. In the classification process, four main types of features are used to represent video-shots: conventional low-level visual features representing physical properties of the video shots, behavioral features which are based on user interaction with the video shots, and two different bag-of-words features obtained from the Automatic Speech Recognition from the audio of the video. 相似文献
12.
13.
Recently, question series have become one focus of research in question answering. These series are comprised of individual factoid, list, and “other” questions organized around a central topic, and represent abstractions of user–system dialogs. Existing evaluation methodologies have yet to catch up with this richer task model, as they fail to take into account contextual dependencies and different user behaviors. This paper presents a novel simulation-based methodology for evaluating answers to question series that addresses some of these shortcomings. Using this methodology, we examine two different behavior models: a “QA-styled” user and an “IR-styled” user. Results suggest that an off-the-shelf document retrieval system is competitive with state-of-the-art QA systems in this task. Advantages and limitations of evaluations based on user simulations are also discussed. 相似文献
14.
Media sharing applications, such as Flickr and Panoramio, contain a large amount of pictures related to real life events. For this reason, the development of effective methods to retrieve these pictures is important, but still a challenging task. Recognizing this importance, and to improve the retrieval effectiveness of tag-based event retrieval systems, we propose a new method to extract a set of geographical tag features from raw geo-spatial profiles of user tags. The main idea is to use these features to select the best expansion terms in a machine learning-based query expansion approach. Specifically, we apply rigorous statistical exploratory analysis of spatial point patterns to extract the geo-spatial features. We use the features both to summarize the spatial characteristics of the spatial distribution of a single term, and to determine the similarity between the spatial profiles of two terms – i.e., term-to-term spatial similarity. To further improve our approach, we investigate the effect of combining our geo-spatial features with temporal features on choosing the expansion terms. To evaluate our method, we perform several experiments, including well-known feature analyzes. Such analyzes show how much our proposed geo-spatial features contribute to improve the overall retrieval performance. The results from our experiments demonstrate the effectiveness and viability of our method. 相似文献
15.
Frieder Meyer-Krahmer 《Research Policy》1989,18(6)
The incorporation of an evaluation procedure in a growing number of innovation policy programmes has now become an accepted feature in the public management of many countries. There already exists substantial experience on the conduct of such evaluations. The purpose of this paper is to present some sample evaluations of measures to promote innovation in a number of European OECD countries (Federal Republic of Germany, France, Netherlands, Sweden). Reference will be made to four of them to illustrate the diversity of the evaluative approaches applied to the wide spectrum of measures to promote innovation. Each of these examples will be studied in detail in order to identify their characteristic features, particularly with attention to their causes, performance and their use in political decision processes.A conceptual framework will be presented in order to propose a typology of various forms of evaluation and to characterize the four case studies. It shows that concept and process of evaluations are strongly influences by the specific context and consensus or dissent on objectives and resources. The case studies give a clearer insight into the factors determining the use of results of evaluations; and to what extent the role of evaluations varies from pure legitimation to a systematic and rational basis for decision making in the area of technology policy.Innovation policy consists of government actions towards technological developments and their implementation in the economy. Innovation, in this case, is defined as the development of technologically new or improved products or techniques and their commercialization in the market or implementation within production. Often evaluation means the examination and assessment of the mode of action and of the effectiveness of government innovation policy. However, finding a general interpretation of the expression “evaluation of an innovation policy” presents greater difficulty, and here wide divergences are evident. In the Federal Republic of Germany and the Netherlands, the term encompasses all forms of monitoring and assessing the operation and/or the effectiveness of an innovation policy. In Sweden, evaluation usually means performance of the a posteriori analysis of a measure. The terms “ex ante evaluation” and “follow-up evaluation” are respectively used for the explicit designation of prospective and retrospective analyses. In France, the term “evaluation” is associated with the notion of value. It often carries the connotation of a value assessment with the full monitoring implications of that expression. 相似文献
16.
Jiqun Liu Yiwei Wang Soumik Mandal Chirag Shah 《Information processing & management》2019,56(3):1010-1025
An individual's Web search behavior can be influenced by a number of factors, including features and functions of a search engine as well as search education. In contrast to the long-lasting attention to the algorithm and interface dimensions of search, there is a lack of research concerned with the potential effects of user education on search behavior. To address this gap, we ran a three-session field-lab-combined study to examine the effects of user education from two distinct sources – peer advice and cognitive authority (operationalized as video-based student's advice and expert's advice respectively) – on Web search behavior in two different search task scenarios (i.e., factual specific and factual amorphous tasks). We also tested if these behavioral effects persist for a short period of time when the explicit search tips are removed. Using 185 task session data generated by 31 participants in two field and one lab sessions, this study demonstrates that: (1) both peer advice and cognitive authority are effective in stimulating immediate behavioral changes in Web search; (2) the immediate behavioral impact of search advice is broader in factual amorphous task than in factual specific task; (3) framing search tips as the advice from cognitive authority is more likely to generate continuing, short-term effects on Web search behaviors. This research has implications for the design of task-aware user education as well as the study of users’ interactions with IR systems in general. 相似文献
17.
本文主要介绍了目前我国网络信息检索用户研究的3个方面,即用户的网络信息检索行为研究现状、用户因素对于网络信息检索过程以及效率的影响以及用户模型构建方面的研究。然后介绍了现有的网络信息检索用户主要的调查方法,最后提出了对于网络信息检索用户研究的展望。 相似文献
18.
《Information processing & management》2001,37(3):435-458
We are interested in how ideas from document clustering can be used to improve the retrieval accuracy of ranked lists in interactive systems. In particular, we are interested in ways to evaluate the effectiveness of such systems to decide how they might best be constructed. In this study, we construct and evaluate systems that present the user with ranked lists and a visualization of inter-document similarities. We first carry out a user study to evaluate the clustering/ranked list combination on instance-oriented retrieval, the task of the TREC-6 Interactive Track. We find that although users generally prefer the combination, they are not able to use it to improve effectiveness. In the second half of this study, we develop and evaluate an approach that more directly combines the ranked list with information from inter-document similarities. Using the TREC collections and relevance judgments, we show that it is possible to realize substantial improvements in effectiveness by doing so, and that although users can use the combined information effectively, the system can provide hints that substantially improve on the user's solo effort. The resulting approach shares much in common with an interactive application of incremental relevance feedback. Throughout this study, we illustrate our work using two prototype systems constructed for these evaluations. The first, AspInQuery, is a classic information retrieval system augmented with a specialized tool for recording information about instances of relevance. The other system, Lighthouse, is a Web-based application that combines a ranked list with a portrayal of inter-document similarity. Lighthouse can work with collections such as TREC, as well as the results of Web search engines. 相似文献
19.