共查询到20条相似文献，搜索用时 31 毫秒
1.
Crowdsourcing relevance judgments for the evaluation of search engines is used increasingly to overcome the issue of scalability that hinders traditional approaches relying on a fixed group of trusted expert judges. However, the benefits of crowdsourcing come with risks due to the engagement of a self-forming group of individuals—the crowd, motivated by different incentives, who complete the tasks with varying levels of attention and success. This increases the need for a careful design of crowdsourcing tasks that attracts the right crowd for the given task and promotes quality work. In this paper, we describe a series of experiments using Amazon’s Mechanical Turk, conducted to explore the ‘human’ characteristics of the crowds involved in a relevance assessment task. In the experiments, we vary the level of pay offered, the effort required to complete a task and the qualifications required of the workers. We observe the effects of these variables on the quality of the resulting relevance labels, measured based on agreement with a gold set, and correlate them with self-reported measures of various human factors. We elicit information from the workers about their motivations, interest and familiarity with the topic, perceived task difficulty, and satisfaction with the offered pay. We investigate how these factors combine with aspects of the task design and how they affect the accuracy of the resulting relevance labels. Based on the analysis of 960 HITs and 2,880 HIT assignments resulting in 19,200 relevance labels, we arrive at insights into the complex interaction of the observed factors and provide practical guidelines to crowdsourcing practitioners. In addition, we highlight challenges in the data analysis that stem from the peculiarity of the crowdsourcing environment where the sample of individuals engaged in specific work conditions are inherently influenced by the conditions themselves. 相似文献
2.
3.
Challenges in the adoption of crisis crowdsourcing and social media in Canadian emergency management
《Government Information Quarterly》2019,36(3):501-509
When a disaster occurs, government agencies are responsible for managing the response and recovery efforts of the impacted communities and infrastructure. Crowdsourcing and social media are widely used in disaster response, yet their primary implementation and application are often via non-governmental agencies and private citizens. A review of the literature suggests that government emergency management agencies in Canada have made little documented progress in adopting crowdsourcing or social media for emergency management. Most of the literature around crowdsourcing and social media for emergency management focus on its use or role outside of Canada (e.g. the USA, Australia, etc.). In order for government agencies in Canada to progress, it is important to identify the Canada-specific barriers and constraints. This study offers a new perspective from Canadian government emergency management agencies to address this gap. Through a series of semi-structured interviews with 15 government officials from 14 agencies, this study identifies the primary challenges and constraints faced by Canadian agencies looking to adopt crisis crowdsourcing. Results indicate that organizational factors, policies, and federal legislation in Canada present barriers to crisis crowdsourcing (including crowdsourcing through social media) adoption within agencies at various levels of government. Based on these results, recommendations are made to support the adoption of crisis crowdsourcing amongst Canadian government agencies. 相似文献
4.
历史档案资源是重要的数字人文实践对象,而众包是聚合与优化历史档案资源的信息组织形式。数字人文项目前端的历史档案资源众包具有面向项目需求贡献和转化资源内容、在网络环境下开放和共享资源与成果、基于共同目标实现全景式协作等特点,同时具有网络平台存在安全隐患、用户行为和任务质量难以掌控等风险。档案馆、图书馆、博物馆等发包方是历史档案资源众包风险控制的实施主体,从历史档案资源众包和数字人文项目的前后端关系出发,可以规划基于历时视角的众包风险控制路径:在前期阶段设计和构建平台并实施用户准入控制,在中期阶段进行指导协调并监督任务执行,在后期阶段检视、审核并发布众包成果。控制成效可在后端的数字人文教学和研究中得到检验。 相似文献
5.
在信息检索系统评估和信息搜寻行为研究中,实验法是常用的研究方法,而实验中的任务设计是实验成功的关键环节。设计模拟仿真工作任务情境是近年来常用的任务设计方式,而它能否替代用户的真实工作任务在很大程度上影响着实验研究结果的科学性。为验证此方法,招募42位本科生参加实验,要求其为完成一项真实工作任务和一项仿真工作任务,在CNKI中展开搜索。通过分析实验参加者在实施两项搜索任务中的交互信息检索行为和检索绩效,发现仿真工作任务与真实工作任务在属性上尽管存在某些显著差异,但并未对用户交互信息搜索行为及绩效产生显著影响。因而,仿真工作任务可以替代真实工作任务用于交互信息检索实验研究,但需要进行合理设计。 相似文献
6.
过去十年,公众科学已经从一个新兴概念逐渐演化成互联网环境下群体协作的利器。然而,由于公众科学项目发起主体的多元性、参与方的广泛性和异质性、实施过程的复杂性和动态性,其运行机制和管理方式还有很多难点。科研众包理念是成功开展公众科学项目的前提和重点,因此可以作为公众科学的理论视角去进行概念解析,而对于科研众包类型的解构也将有助于公众科学项目的模式设计。本研究认为基于科研众包类型和公众科学开展环境两个维度,可以有效构建公众科学项目的业务模式并进行业务划分;以“机构观”的思想去驱动、管理并维系公众科学项目的发展,加入第三方组织机构这一实体可以重构公众科学项目的运作模式。而立足于图书情报学科,可以对数字人文类公众科学项目进行深入探索,尤其可以从特色馆藏利用、数字人文平台构建、科研数据管护以及用户激励和培训等方面开展后续工作。图4。表1。参考文献45。 相似文献
7.
Crowdsourcing interactions: using crowdsourcing for evaluating interactive information retrieval systems 总被引：1，自引：0，他引：1
Guido Zuccon Teerapong Leelanupab Stewart Whiting Emine Yilmaz Joemon M. Jose Leif Azzopardi 《Information Retrieval》2013,16(2):267-305
In the field of information retrieval (IR), researchers and practitioners are often faced with a demand for valid approaches to evaluate the performance of retrieval systems. The Cranfield experiment paradigm has been dominant for the in-vitro evaluation of IR systems. Alternative to this paradigm, laboratory-based user studies have been widely used to evaluate interactive information retrieval (IIR) systems, and at the same time investigate users’ information searching behaviours. Major drawbacks of laboratory-based user studies for evaluating IIR systems include the high monetary and temporal costs involved in setting up and running those experiments, the lack of heterogeneity amongst the user population and the limited scale of the experiments, which usually involve a relatively restricted set of users. In this paper, we propose an alternative experimental methodology to laboratory-based user studies. Our novel experimental methodology uses a crowdsourcing platform as a means of engaging study participants. Through crowdsourcing, our experimental methodology can capture user interactions and searching behaviours at a lower cost, with more data, and within a shorter period than traditional laboratory-based user studies, and therefore can be used to assess the performances of IIR systems. In this article, we show the characteristic differences of our approach with respect to traditional IIR experimental and evaluation procedures. We also perform a use case study comparing crowdsourcing-based evaluation with laboratory-based evaluation of IIR systems, which can serve as a tutorial for setting up crowdsourcing-based IIR evaluations. 相似文献
8.
9.
[目的/意义]鉴于开放资源建设实践面临困境,将众包引入开放会议资源服务系统(简称"系统"),以提高其开放会议资源建设效率。[方法/过程]采用经验总结法分析"系统"现有资源建设模式的不足,结合实际,从基于众包的资源建设工作流程、众包平台开发与功能设计、系统功能与服务完善3方面构建开放会议资源众包建设模式。[结果/结论]资源众包建设模式能增加系统所需的人力、智力资源,提高资源建设效率,实现丰富系统开放会议资源、获得更多使用的建设目的。开放会议资源建设众包模式能提高"系统"的工作效率,可为其他图书馆的开放会议资源建设借鉴和参考。 相似文献
10.
《Government Information Quarterly》2023,40(3):101785
Government chatbots have become increasingly popular as artificial-intelligence-based tools to improve communication between the government and its citizens. This study explores the interaction mode design of a trustworthy government chatbot, which involves multiple social characteristics from the user-centric perspective. A discrete choice experiment was conducted in the context of Chinese government chatbots to examine the effects of various social characteristics on citizen preferences. Participants utilized a crowdsourcing survey platform to report their preferences for interaction processes designed with distinct sets of social characteristics. Valid data were obtained from 371 participants and analyzed using a multinomial logit model. The results indicate that (in order from highest to lowest impact) emotional intelligence, proactivity, identity consistency, and conscientiousness significantly influence the citizens' preferences. Identity consistency has a negative effect, whereas the other factors all have positive impacts. It was also determined that some of these correlations are influenced by the participants' individual characteristics, such as age, gender, and prior experience with chatbots. This work provides empirical evidence for the relative importance of social characteristics and their impacts on user perception, expands the service dimension scope of information provision/communication (one of five categories of digital interaction), and facilitates the identification and operationalization of the social characteristics. We provide a theoretical framework to understand the interaction model design of a trustworthy government chatbot and also offer practical recommendations for government chatbot designers and policy implications. 相似文献
11.
Dongfang Gaozhao 《Government Information Quarterly》2021,38(3):101591
Policymakers are taking action to protect their citizens and democratic systems from online misinformation. However, media consumers usually have a hard time differentiating misinformation from authentic information. There are two explanations for this difficulty, namely lazy reasoning and motivated reasoning. While lazy reasoning suggests that people may feel reluctant to conduct critical reasoning when consuming online information, the motivated reasoning theory points out that individuals are also thinking in alignment with their identities and established viewpoints. A proposed approach to address this issue is adding fact-checking flags in the hope that flags could alert people to information falsehoods and stimulate critical thinking. This study examines the impact of fact-checking flags on media consumers' identification of fake news. Conducting an experiment (n = 717) on Amazon Mechanical Turk, the study finds that experimental participants with different political backgrounds depend heavily on flag-checking results provided by flags. Flags are powerful to influence people's judgments in a way that participants have blind beliefs in flags even if the flag assessments are inaccurate. Furthermore, the study's results indicate that flag assessments made by professional fact-checkers or crowdsourcing are equally influential in shaping participants' identification. These observations provide public and private leaders with suggestions that fact-checking flags can significantly affect media consumers' identification of fake news. However, flags appear to have little ability to promote critical thinking in this experiment. 相似文献
12.
众包在高校图书馆虚拟参考咨询服务中的运用 总被引：1，自引：0，他引：1
众包借助互联网的优势,低成本汇聚各地的人力资源,解决原本需要高昂费用才能解决甚至不能解决的问题。人力资源不足是影响高校虚拟参考咨询服务健康发展的瓶颈之一,学习、借鉴和引进众包模式,将有助于进一步提高其服务水平和服务质量。 相似文献
13.
Regina Lenart-Gansiniec 《Government Information Quarterly》2021,38(3):101593
Modern local governments must learn in order to deal effectively with the growing pressures of a rapidly changing environment. Crowdsourcing has entered the stage of popularization and wide adoption as a part of local government. Based on organizational learning perspective, this study develops a multi-level model of learning from crowdsourcing and a joint relationship between crowdsourcing and organizational learning in local governments. This paper adopts a multi-method research approach: (1) the model was tested with partial least squares structural equation modelling (PLS-SEM) using a survey of 205 local governments units in Poland; (2) in-depth semi-structured interviews conducted with 18 top managers from 18 local governments units in Poland. The empirical results show that: (1) crowdsourcing has significantly positive affects on individual, group, organizational levels of learning, feed-forward, and feed-back flows; (2) type of crowdsourcing, such wisdom crowd, crowd creation, and crowd voting are linked with organizational learning. Moreover, one type of crowdsourcing, such crowdfunding is not related with organizational learning of local governments. These findings contribute to both academic research and practical implications, advancing our understandings of relationship between crowdsourcing and organizational learning in local governments. 相似文献
14.
The influential Text REtrieval Conference (TREC) retrieval conference has always relied upon specialist assessors or occasionally participating groups to create relevance judgements for the tracks that it runs. Recently however, crowdsourcing has been championed as a cheap, fast and effective alternative to traditional TREC-like assessments. In 2010, TREC tracks experimented with crowdsourcing for the very first time. In this paper, we report our successful experience in creating relevance assessments for the TREC Blog track 2010 top news stories task using crowdsourcing. In particular, we crowdsourced both real-time newsworthiness assessments for news stories as well as traditional relevance assessments for blog posts. We conclude that crowdsourcing not only appears to be a feasible, but also cheap and fast means to generate relevance assessments. Furthermore, we detail our experiences running the crowdsourced evaluation of the TREC Blog track, discuss the lessons learned, and provide best practices. 相似文献
15.
[目的/意义] 通过分析时间限制和任务次序在用户实验设计中对被试和实验结果的影响,提出响应的优化方案。[方法/过程] 使用实验法,对被试搜索时间进行严格的控制,分别验证时间限制和任务次序对被试搜索体验的影响。[结果/结论] 在有时间限制的情况下,被试对于成功完成搜索任务的信心较低,且在任务结束后,对检索结果的自我认同度较低;任务次序也会影响被试的检索体验,搜索任务的次序越靠前,用户的信心和体验越易受到较为负面的影响。建议学者在采用实验法进行用户行为分析的时候,要科学全面地对搜索任务和情境变量进行轮换,如采用拉丁方的设计。同时,若非检验实验时间限制对搜索行为的影响,应尽量避免实验过程中对被试的搜索时间进行限制。 相似文献
16.
引入众包模式促进大学生信息素养教育 总被引：1，自引：0，他引：1
众包作为一种全新的理念和运作模式,将在继外包和开源之后成为支持网络时代图书馆创新的有效途径。从信息素养教育引入众包理念的可行性与必要性进行了理论分析,并从众包模式的组织与管理进行了实践探索。同时,通过实例论证众包模式下如何利用互联网用户的技能和知识,收集新创意与解决特定的信息素养教育问题。 相似文献
17.
以构建开放的图书馆为出发点,分析众包馆员的内涵和主要特征,研究众包馆员产生的背景条件,将众包馆员分为三个层次——入门级众包馆员、初级众包馆员和中高级众包馆员,并对其主要职责进行思考,提出众包馆员实施众包的四个注意事项。 相似文献
18.
从数字出版内容资源开发过程这一视角归纳出众创、众编、众筹与综合型4种主要的数字出版众包模式及其实践形式,并运用众包理论,结合相关案例、探讨了4种实践形式的概念、特点、核心理念,具体应用及发展趋势等,从理论层面较为系统地总结了数字出版众包模式的实践情况。 相似文献
19.
Publishers will usually make decisions based on their experience and knowledge in book publishing. However, there are risks of losses as markets can be unpredictable. Now, with the availability of various online social platforms, “crowdsourcing” is being used by publishers and authors to gather new ideas and innovations. This article will analyze how the crowdsourcing platform is used by industry players to attract active participation from the public, especially in the title development process. Publishers or authors also need to understand which platform is appropriate and how to use the platforms as a marketing communication tool. The netnography approach will be used to gather and analyze the data related to the specific subject area of the study, including literature review and online observation. Based on our observations, platforms such Wattpad and Ilham Karangkraf are popular among Malaysian authors and publishers, and had been used to get crowd engagement as well as to communicate with readers. With it, sometimes a work that has many readers or followers will be published. 相似文献
20.
《Government Information Quarterly》2023,40(1):101760
Crowdsourcing is a form of IT-enabled open innovation that has received increased attention in recent years. However, the use of crowdsourcing in public innovation is still in its early stages. To understand the adoption of crowdsourcing in government, this article investigates the antecedents of the intention to adopt crowdsourcing in government organizations. The adoption intention is conceptualized as a rationalistic, goal-directed decision that is driven by multiple strategic intents but tempered by transaction costs. Three strategic intents (accessing complementary knowledge, enhancing organizational legitimacy, and reducing innovation costs) and two transaction costs (codification costs, and broadcasting costs) are hypothesized as antecedents to the adoption intention. Data (n = 205) from municipalities in Belgium and the Netherlands shows that the adoption intention can be explained by the influence of the political executive, the pursuit of organizational legitimacy, and transaction costs associated with broadcasting. Accessing complementary knowledge, reducing innovation costs, and codification costs are not significant predictors. The findings suggest that crowdsourcing is viewed as a tool for political alignment and legitimation. 相似文献