Gamifying the core IR task, i.e. web search, is a challenging, but potentially rewarding task. We present several approaches to tackling it, going beyond the PBL (points-badges-leaderboards) triad: analysing actual and potential hedonic elements, and suggesting appropriate ludic mechanisms.
{"title":"The game of search: what is the fun in that?","authors":"Mark M. Shovman","doi":"10.1145/2594776.2594786","DOIUrl":"https://doi.org/10.1145/2594776.2594786","url":null,"abstract":"Gamifying the core IR task, i.e. web search, is a challenging, but potentially rewarding task. We present several approaches to tackling it, going beyond the PBL (points-badges-leaderboards) triad: analysing actual and potential hedonic elements, and suggesting appropriate ludic mechanisms.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":" 20","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132011931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents the design of Zombilingo, a Game With A Purpose (GWAP) that allows for the dependency syntax annotation of French corpora. The development will start mid-2014 and the game is to be made available by the end of the year. The created language resource will be freely and continuously available on the game Web site.
{"title":"Creating Zombilingo, a game with a purpose for dependency syntax annotation","authors":"Karën Fort, Bruno Guillaume, Hadrien Chastant","doi":"10.1145/2594776.2594777","DOIUrl":"https://doi.org/10.1145/2594776.2594777","url":null,"abstract":"This paper presents the design of Zombilingo, a Game With A Purpose (GWAP) that allows for the dependency syntax annotation of French corpora. The development will start mid-2014 and the game is to be made available by the end of the year. The created language resource will be freely and continuously available on the game Web site.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114453240","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Crowdsourcing is often applied for the task of replacing the scarce or expensive labour of experts with that of untrained workers. In this paper, we argue, that this objective might not always be desirable, but that we should instead aim at leveraging the considerable work force of the crowd in order to support the highly trained expert. In this paper, we demonstrate this different paradigm on the example of detecting malignant breast cancer in medical images. We compare the effectiveness and efficiency of experts to that of crowd workers, finding significantly better performance at greater cost. In a second series of experiments, we show how the comparably cheap results produced by crowdsourcing workers can serve to make experts more efficient AND more effective at the same time.
{"title":"Crowd-powered experts: helping surgeons interpret breast cancer images","authors":"Carsten Eickhoff","doi":"10.1145/2594776.2594788","DOIUrl":"https://doi.org/10.1145/2594776.2594788","url":null,"abstract":"Crowdsourcing is often applied for the task of replacing the scarce or expensive labour of experts with that of untrained workers. In this paper, we argue, that this objective might not always be desirable, but that we should instead aim at leveraging the considerable work force of the crowd in order to support the highly trained expert. In this paper, we demonstrate this different paradigm on the example of detecting malignant breast cancer in medical images. We compare the effectiveness and efficiency of experts to that of crowd workers, finding significantly better performance at greater cost. In a second series of experiments, we show how the comparably cheap results produced by crowdsourcing workers can serve to make experts more efficient AND more effective at the same time.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129635042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Azzopardi, Martin Bevc, Andrew Gardner, David Maxwell, Abdullah Razzouk
This demo poster presents PageFetch 2, the sequel of PageFetch, Fu-Finder and PageHunt. The game is simple. A player is shown a Webpage. The player must then enter a query that they believe will retrieve the displayed page. The shorter the query and the higher the page appears in the returned rankings, the more points the player receives. Players have three minutes to find as many of the displayed pages as possible. What makes this iteration different is the inclusion of several additional gamification features other than just points and an updated scoring system. We include leaderboards, badges and an avatar component - all of which are designed to help and motivate players. The framework is also publicly available for download, and can be customised to create new search games - or even to evaluate different search engines and document collections. This demo is a preliminary report of the progress we have made to date. At the workshop, we hope to solicit feedback on how we can improve the game and its gameplay.
{"title":"PageFetch 2: gamification the sequel","authors":"L. Azzopardi, Martin Bevc, Andrew Gardner, David Maxwell, Abdullah Razzouk","doi":"10.1145/2594776.2594784","DOIUrl":"https://doi.org/10.1145/2594776.2594784","url":null,"abstract":"This demo poster presents PageFetch 2, the sequel of PageFetch, Fu-Finder and PageHunt. The game is simple. A player is shown a Webpage. The player must then enter a query that they believe will retrieve the displayed page. The shorter the query and the higher the page appears in the returned rankings, the more points the player receives. Players have three minutes to find as many of the displayed pages as possible. What makes this iteration different is the inclusion of several additional gamification features other than just points and an updated scoring system. We include leaderboards, badges and an avatar component - all of which are designed to help and motivate players. The framework is also publicly available for download, and can be customised to create new search games - or even to evaluate different search engines and document collections. This demo is a preliminary report of the progress we have made to date. At the workshop, we hope to solicit feedback on how we can improve the game and its gameplay.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"36 11-12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131804337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. M. Fernández-Luna, J. Huete, Humberto Rodríguez-Avila, Julio C. Rodríguez Cano
Collaborative search systems require to make sense of the information to achieve shared goals and reduce additional costs. These systems include cognitive load and the cost to coordinate various events and participants. By applying game mechanics and dynamics to these collaborative systems we can intensify seeker engagement with this type of applications and its collaborative tasks. To the best of our knowledge the use of game design elements in non-gaming contexts has been so far discussed in a few research contexts, without ties to existing literature on Collaborative Information Seeking. This paper presents a set of elements to gamify collaborative search systems.
{"title":"Enhancing collaborative search systems engagement through gamification","authors":"J. M. Fernández-Luna, J. Huete, Humberto Rodríguez-Avila, Julio C. Rodríguez Cano","doi":"10.1145/2594776.2594785","DOIUrl":"https://doi.org/10.1145/2594776.2594785","url":null,"abstract":"Collaborative search systems require to make sense of the information to achieve shared goals and reduce additional costs. These systems include cognitive load and the cost to coordinate various events and participants. By applying game mechanics and dynamics to these collaborative systems we can intensify seeker engagement with this type of applications and its collaborative tasks. To the best of our knowledge the use of game design elements in non-gaming contexts has been so far discussed in a few research contexts, without ties to existing literature on Collaborative Information Seeking. This paper presents a set of elements to gamify collaborative search systems.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132049653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we examine the Keynesian Beauty Contest, a well-known examination of rational agents used to explain the role of consensus predictions in decision making such as price fluctuations in equity markets. Using a game, we study the crowd's ability to judge relevance for both images and textual documents. In addition to asking participants to determine if a document is relevant, we also ask them to rank all choices. One group of participants (N=137) was asked to make judgments based on their own assessment while another group of participants (N = 137) was asked to make judgments based on their estimate of a consensus decision. In addition to measuring recall and precision, our game also uses rank-biased overlap (RBO) to compare each participant's ranked list with the overall consensus decision. Results show the group asked to make ranking decisions based on their estimate of consensus had significantly higher recall for judging relevance in text documents and significantly higher recall and precision when judging relevance for a set of images. We believe this has implications for the determination of consensus across multiple contexts.
{"title":"The beauty contest revisited: measuring consensus rankings of relevance using a game","authors":"Christopher G. Harris","doi":"10.1145/2594776.2594780","DOIUrl":"https://doi.org/10.1145/2594776.2594780","url":null,"abstract":"In this paper, we examine the Keynesian Beauty Contest, a well-known examination of rational agents used to explain the role of consensus predictions in decision making such as price fluctuations in equity markets. Using a game, we study the crowd's ability to judge relevance for both images and textual documents. In addition to asking participants to determine if a document is relevant, we also ask them to rank all choices. One group of participants (N=137) was asked to make judgments based on their own assessment while another group of participants (N = 137) was asked to make judgments based on their estimate of a consensus decision. In addition to measuring recall and precision, our game also uses rank-biased overlap (RBO) to compare each participant's ranked list with the overall consensus decision. Results show the group asked to make ranking decisions based on their estimate of consensus had significantly higher recall for judging relevance in text documents and significantly higher recall and precision when judging relevance for a set of images. We believe this has implications for the determination of consensus across multiple contexts.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133288388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Human computation is a very powerful tool for solving tasks that cannot be solved by computers efficiently. One such problem is ranking images upon their relevance for a semantic query or upon how well they depict a semantic concept. In this paper we investigate a method to leverage human computation in a divide-and-conquer approach to create precise ranking models. We discuss the basic technique, our prototype client, its adoption to a gamification approach, and present the results of a study with the prototype. Results from the study indicate that with our method the ranking aggregated from the user input converges fast to an optimal ranking.
{"title":"PictureSort: gamification of image ranking","authors":"M. Lux, Mario Guggenberger, M. Riegler","doi":"10.1145/2594776.2594789","DOIUrl":"https://doi.org/10.1145/2594776.2594789","url":null,"abstract":"Human computation is a very powerful tool for solving tasks that cannot be solved by computers efficiently. One such problem is ranking images upon their relevance for a semantic query or upon how well they depict a semantic concept. In this paper we investigate a method to leverage human computation in a divide-and-conquer approach to create precise ranking models. We discuss the basic technique, our prototype client, its adoption to a gamification approach, and present the results of a study with the prototype. Results from the study indicate that with our method the ranking aggregated from the user input converges fast to an optimal ranking.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127880522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gamification is a new and burgeoning field, with which Information Retrieval appears to have a particular affinity. In this talk, I consider what it is that Gamification offers Information Retrieval and at what potential cost. I do this by examining the Call for Papers for the workshop, attempting to tease out what Gamification is laying on the table that other approaches - especially games themselves - do not. Gamification is being consciously distanced from Serious Games, with only the "best bits" being used, but there appears to be no consensus as to why and (because it's so young a field and Information Retrieval is getting into it early) only a superficial understanding of what Gamification is. I believe that it's entirely possible that the uses of Gamification in Information Retrieval extend well beyond those envisaged. I conclude by arguing that although Game Design is an art form, Gamification is an application of Psychology. As successful Information Retrieval relies on an understanding of human motivation, it is therefore no surprise that the two are well suited to one another.
{"title":"Information reconstruction: unpicking the GamifIR call for papers","authors":"R. Bartle","doi":"10.1145/2594776.2597423","DOIUrl":"https://doi.org/10.1145/2594776.2597423","url":null,"abstract":"Gamification is a new and burgeoning field, with which Information Retrieval appears to have a particular affinity. In this talk, I consider what it is that Gamification offers Information Retrieval and at what potential cost. I do this by examining the Call for Papers for the workshop, attempting to tease out what Gamification is laying on the table that other approaches - especially games themselves - do not. Gamification is being consciously distanced from Serious Games, with only the \"best bits\" being used, but there appears to be no consensus as to why and (because it's so young a field and Information Retrieval is getting into it early) only a superficial understanding of what Gamification is. I believe that it's entirely possible that the uses of Gamification in Information Retrieval extend well beyond those envisaged.\u0000 I conclude by arguing that although Game Design is an art form, Gamification is an application of Psychology. As successful Information Retrieval relies on an understanding of human motivation, it is therefore no surprise that the two are well suited to one another.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129103996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Maltzahn, A. Jhala, Michael Mateas, E. Whitehead
The super-exponential growth of digital data world-wide is matched by personal digital archives containing songs, ebooks, audio books, photos, movies, textual documents, and documents of other media types. For many types of media it is usually a lot easier to add items than to keep archives from falling into disarray and incurring data loss. The overhead of maintaining these personal archives frequently surpasses the time and patience their owners are willing to dedicate to this important task. The promise of gamification in this context is to significantly extend the willingness to maintain personal archives by enhancing the experience of personal archive management. In this paper we focus on a subcategory of personal archives which we call private archives. These are archives that for a variety of reasons the owner does not want to make available online and which consequently limits archive maintenance to an individual activity and does not allow any form of crowdsourcing out of fear for unwanted information leaks. As an example of private digital archive maintenance gamification we describe InfoGarden, a casual game that turns document tagging into an individual activity of (metaphorically) weeding a garden and protecting plants from gophers and includes a reward system that encourages orthogonal tag usage. The paper concludes with lessons learned and summarizes remaining challenges.
{"title":"Gamification of private digital data archive management","authors":"C. Maltzahn, A. Jhala, Michael Mateas, E. Whitehead","doi":"10.1145/2594776.2594783","DOIUrl":"https://doi.org/10.1145/2594776.2594783","url":null,"abstract":"The super-exponential growth of digital data world-wide is matched by personal digital archives containing songs, ebooks, audio books, photos, movies, textual documents, and documents of other media types. For many types of media it is usually a lot easier to add items than to keep archives from falling into disarray and incurring data loss. The overhead of maintaining these personal archives frequently surpasses the time and patience their owners are willing to dedicate to this important task. The promise of gamification in this context is to significantly extend the willingness to maintain personal archives by enhancing the experience of personal archive management.\u0000 In this paper we focus on a subcategory of personal archives which we call private archives. These are archives that for a variety of reasons the owner does not want to make available online and which consequently limits archive maintenance to an individual activity and does not allow any form of crowdsourcing out of fear for unwanted information leaks. As an example of private digital archive maintenance gamification we describe InfoGarden, a casual game that turns document tagging into an individual activity of (metaphorically) weeding a garden and protecting plants from gophers and includes a reward system that encourages orthogonal tag usage. The paper concludes with lessons learned and summarizes remaining challenges.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114702277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The exponential growth of digital generated content in the form of audio, video and complex data structures calls for novel methods and tools able to cope with the limitation of automated analysis techniques. Gamification, the process of using game design methodologies and game mechanics to enhance traditional applications, is a promising tool that can help to increase the active involvement of humans in the Information Retrieval processes. This work contributes to the emerging research field of Gamification in Information Retrieval by providing an overview on: 1) the fundamental elements of a game; 2) the major game mechanics that have been applied in traditional games and gamication techniques; and 3) an overview of the possible adoption of such techniques in a typical IR scenario. The goal is to lay a path for the adoption of these new tools in IR systems, focusing on their application to the traditional building blocks of the query and content analysis processes.
{"title":"On the application of game mechanics in information retrieval","authors":"Luca Galli, P. Fraternali, A. Bozzon","doi":"10.1145/2594776.2594778","DOIUrl":"https://doi.org/10.1145/2594776.2594778","url":null,"abstract":"The exponential growth of digital generated content in the form of audio, video and complex data structures calls for novel methods and tools able to cope with the limitation of automated analysis techniques.\u0000 Gamification, the process of using game design methodologies and game mechanics to enhance traditional applications, is a promising tool that can help to increase the active involvement of humans in the Information Retrieval processes.\u0000 This work contributes to the emerging research field of Gamification in Information Retrieval by providing an overview on: 1) the fundamental elements of a game; 2) the major game mechanics that have been applied in traditional games and gamication techniques; and 3) an overview of the possible adoption of such techniques in a typical IR scenario.\u0000 The goal is to lay a path for the adoption of these new tools in IR systems, focusing on their application to the traditional building blocks of the query and content analysis processes.","PeriodicalId":170006,"journal":{"name":"GamifIR '14","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114232457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：