More and more crisis managers, crisis communicators and laypeople use Twitter and other social media to provide or seek crisis information. In this paper, we focus on retrospective conversion of human-safety related data to crisis management knowledge. First, we study how Twitter data can be classified into the seven categories of the United Nations Development Program Security Model (i.e., Food, Health, Politics, Economic, Personal, Community, and Environment). We conclude that these topic categories are applicable, and supplementing them with classification of individual authors into more generic sources of data (i.e., Official authorities, Media, and Laypeople) allows curating data and assessing crisis maturity. Second, we introduce automated classifiers, based on supervised learning and decision rules, for both tasks and evaluate their correctness. This evaluation uses two datasets collected during the crises of Queensland floods and NZ Earthquake in 2011. The topic classifier performs well in the major categories (i.e., 120--190 training instances) of Economic (F = 0.76) and Community (F = 0.67) while in the minor categories (i.e., 0--60 training instances) the results are more modest (F ≤ 0.41). The source classifier shows excellent results (F ≥ 0.83) in all categories.
{"title":"Crisis management knowledge from social media","authors":"K. Kreiner, A. Immonen, H. Suominen","doi":"10.1145/2537734.2537740","DOIUrl":"https://doi.org/10.1145/2537734.2537740","url":null,"abstract":"More and more crisis managers, crisis communicators and laypeople use Twitter and other social media to provide or seek crisis information. In this paper, we focus on retrospective conversion of human-safety related data to crisis management knowledge. First, we study how Twitter data can be classified into the seven categories of the United Nations Development Program Security Model (i.e., Food, Health, Politics, Economic, Personal, Community, and Environment). We conclude that these topic categories are applicable, and supplementing them with classification of individual authors into more generic sources of data (i.e., Official authorities, Media, and Laypeople) allows curating data and assessing crisis maturity. Second, we introduce automated classifiers, based on supervised learning and decision rules, for both tasks and evaluate their correctness. This evaluation uses two datasets collected during the crises of Queensland floods and NZ Earthquake in 2011. The topic classifier performs well in the major categories (i.e., 120--190 training instances) of Economic (F = 0.76) and Community (F = 0.67) while in the minor categories (i.e., 0--60 training instances) the results are more modest (F ≤ 0.41). The source classifier shows excellent results (F ≥ 0.83) in all categories.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122990295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Using eye tracking in the evaluation of web search interfaces can provide rich information on users' information search behaviour, particularly in the matter of user interaction with different informative components on a search results screen. One of the main issues affecting the use of eye tracking in research is the quality of captured eye movements (calibration), therefore, in this paper, we propose a method that allows us to determine the quality of calibration, since the existing eye tracking system (Tobii Studio) does not provide any criteria for this aspect. Another issue addressed in this paper is the adaptation of gaze direction. We use a black screen displaying for 3 seconds between screens to avoid the effect of the previous screen on user gaze direction on the coming screen. A further issue when employing eye tracking in the evaluation of web search interfaces is the selection of the appropriate filter for the raw gaze-points data. In our studies, we filtered this data by removing noise, identifying gaze points that occur in Area of Interests (AOIs), optimising gaze data and identifying viewed AOIs.
{"title":"Using eye tracking for evaluating web search interfaces","authors":"H. A. Maqbali, Falk Scholer, J. Thom, Mingfang Wu","doi":"10.1145/2537734.2537747","DOIUrl":"https://doi.org/10.1145/2537734.2537747","url":null,"abstract":"Using eye tracking in the evaluation of web search interfaces can provide rich information on users' information search behaviour, particularly in the matter of user interaction with different informative components on a search results screen. One of the main issues affecting the use of eye tracking in research is the quality of captured eye movements (calibration), therefore, in this paper, we propose a method that allows us to determine the quality of calibration, since the existing eye tracking system (Tobii Studio) does not provide any criteria for this aspect. Another issue addressed in this paper is the adaptation of gaze direction. We use a black screen displaying for 3 seconds between screens to avoid the effect of the previous screen on user gaze direction on the coming screen. A further issue when employing eye tracking in the evaluation of web search interfaces is the selection of the appropriate filter for the raw gaze-points data. In our studies, we filtered this data by removing noise, identifying gaze points that occur in Area of Interests (AOIs), optimising gaze data and identifying viewed AOIs.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114860020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Enterprise query auto-completion (QAC) can allow website or intranet visitors to satisfy a need more efficiently than traditional searching and browsing. The limited scope of an enterprise makes it possible to satisfy a high proportion of information needs through completion. Further, the availability of structured sources of completions such as product catalogues compensates for sparsity of log data. Extended forms (X-QAC) can give access to information that is inaccessible via a conventional crawled index. We show that it can be guaranteed that for every suggestion there is a prefix which causes it to appear in the top k suggestions. Using university query logs and structured lists, we quantify the significant keystroke savings attributable to this guarantee (worst case). Such savings may be of particular value for mobile devices. A user experiment showed that a staff lookup task took an average of 61% longer with a conventional search interface than with an X-QAC system. Using wine catalogue data we demonstrate a further extension which allows a user to home in on desired items in faceted-navigation style. We also note that advertisements can be triggered from QAC. Given the advantages and power of X-QAC systems, we envisage that websites and intranets of the [near] future will provide less navigation and rely less on conventional search.
{"title":"An enterprise search paradigm based on extended query auto-completion: do we still need search and navigation?","authors":"D. Hawking, K. Griffiths","doi":"10.1145/2537734.2537743","DOIUrl":"https://doi.org/10.1145/2537734.2537743","url":null,"abstract":"Enterprise query auto-completion (QAC) can allow website or intranet visitors to satisfy a need more efficiently than traditional searching and browsing. The limited scope of an enterprise makes it possible to satisfy a high proportion of information needs through completion. Further, the availability of structured sources of completions such as product catalogues compensates for sparsity of log data. Extended forms (X-QAC) can give access to information that is inaccessible via a conventional crawled index.\u0000 We show that it can be guaranteed that for every suggestion there is a prefix which causes it to appear in the top k suggestions. Using university query logs and structured lists, we quantify the significant keystroke savings attributable to this guarantee (worst case). Such savings may be of particular value for mobile devices. A user experiment showed that a staff lookup task took an average of 61% longer with a conventional search interface than with an X-QAC system.\u0000 Using wine catalogue data we demonstrate a further extension which allows a user to home in on desired items in faceted-navigation style. We also note that advertisements can be triggered from QAC.\u0000 Given the advantages and power of X-QAC systems, we envisage that websites and intranets of the [near] future will provide less navigation and rely less on conventional search.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"125 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132101759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Analysis of Collocation is targeted for Natural Language Processing (NLP). From a linguistic perspective, collocation provides us with a way to place words close together in a natural manner. By this approach, we can examine deep structure of semantics through words and their situation. Although there have been some investigation based on co-occurrence, few discussion has been made about conditional collocation. In this investigation, we discuss a computational approach to extract conditional collocation by using data mining and statistical techniques.
{"title":"Conditional collocation in Japanese","authors":"Takumi Sonoda, T. Miura","doi":"10.1145/2537734.2537736","DOIUrl":"https://doi.org/10.1145/2537734.2537736","url":null,"abstract":"Analysis of Collocation is targeted for Natural Language Processing (NLP). From a linguistic perspective, collocation provides us with a way to place words close together in a natural manner. By this approach, we can examine deep structure of semantics through words and their situation. Although there have been some investigation based on co-occurrence, few discussion has been made about conditional collocation. In this investigation, we discuss a computational approach to extract conditional collocation by using data mining and statistical techniques.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128325627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Previous work has examined space saving and throughput increasing techniques for long postings lists in an inverted file search engine. In this contribution we show that highly sporadic terms (terms that occur in 1 or 2 documents) are a high proportion of the unique terms in the collection and that these terms are seen in queries. The previously known space saving method of storing their short postings lists in the vocabulary is compared to storing in the postings file. We quantify the saving as about 6.5%, with no loss in precision, and suggest the adoption of this technique.
{"title":"Managing short postings lists","authors":"A. Trotman, Xiangfei Jia, Matt Crane","doi":"10.1145/2537734.2537738","DOIUrl":"https://doi.org/10.1145/2537734.2537738","url":null,"abstract":"Previous work has examined space saving and throughput increasing techniques for long postings lists in an inverted file search engine. In this contribution we show that highly sporadic terms (terms that occur in 1 or 2 documents) are a high proportion of the unique terms in the collection and that these terms are seen in queries. The previously known space saving method of storing their short postings lists in the vocabulary is compared to storing in the postings file. We quantify the saving as about 6.5%, with no loss in precision, and suggest the adoption of this technique.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133220356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nosocomial infections (NIs, any infection that a patient contracts in a healthcare institution) cost 100, 000 lives and five billion dollars per year for 300 million Americans alone. Surveillance in hospitals holds the potential of reducing NI rates by more than thirty per cent but performing this task by hand is impossible at scale of every appointment, examination, intervention, and other event in healthcare. Narratives in patient records can indicate NIs and their automated processing could scale out surveillance. This paper describes a text summarisation system for NI surveillance and situational awareness in hospitals. The system is a cascaded sentence, report, and patient classifier. It generates three types of visual summaries for an input of patient narratives and ward maps: cross-sectional statuses at the same point of time, longitudinal trends in time, and highlighted text to see the textual evidence leading to a given status or trend. This gives evidence for and against a given NI in the levels of hospitals, wards, patients, reports, and sentences. The system has excellent recall and precision (e.g., 0.95 and 0.71 for reports) in summarisation for the subset of NIs from fungal species on 1,880 authentic records of 527 patients from 3 hospitals. To demonstrate the system design, we have developed a mobile iPad compatible web-application and a simulation with eighteen patients on three medical wards in one hospital during one month with 61 records in total. The design is extendable to other summarisation tasks.
{"title":"Visual summarisation of text for surveillance and situational awareness in hospitals","authors":"H. Suominen, L. Hanlen","doi":"10.1145/2537734.2537739","DOIUrl":"https://doi.org/10.1145/2537734.2537739","url":null,"abstract":"Nosocomial infections (NIs, any infection that a patient contracts in a healthcare institution) cost 100, 000 lives and five billion dollars per year for 300 million Americans alone. Surveillance in hospitals holds the potential of reducing NI rates by more than thirty per cent but performing this task by hand is impossible at scale of every appointment, examination, intervention, and other event in healthcare. Narratives in patient records can indicate NIs and their automated processing could scale out surveillance. This paper describes a text summarisation system for NI surveillance and situational awareness in hospitals. The system is a cascaded sentence, report, and patient classifier. It generates three types of visual summaries for an input of patient narratives and ward maps: cross-sectional statuses at the same point of time, longitudinal trends in time, and highlighted text to see the textual evidence leading to a given status or trend. This gives evidence for and against a given NI in the levels of hospitals, wards, patients, reports, and sentences. The system has excellent recall and precision (e.g., 0.95 and 0.71 for reports) in summarisation for the subset of NIs from fungal species on 1,880 authentic records of 527 patients from 3 hospitals. To demonstrate the system design, we have developed a mobile iPad compatible web-application and a simulation with eighteen patients on three medical wards in one hospital during one month with 61 records in total. The design is extendable to other summarisation tasks.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129835035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Web search services process thousands of queries per second, and filter their answers from collections containing very large amounts of data. Fast response to queries is a critical service expectation. The well-known WAND processing strategy is one way of reducing the amount of computation necessary when executing such a query. The value of WAND has now been validated in a wide range of studies, and has become one of the key baselines against which all new top-k processing algorithms are benchmarked. However, most previous implementations of WAND-based retrieval approaches have been in the context of the BM25 Okapi similarity scoring regime. Here we measure the performance of WAND in the context of the alternative Language Model similarity score computation, and find that the dramatic efficiency gains reported in previous studies are no longer achievable. That is, when the primary goal of a retrieval system is to maximize effectiveness, WAND is relatively unhelpful in terms of attaining the secondary objective of maximizing query throughput rates. However, the BM-WAND algorithm does in fact help reducing the percentage of postings to be scored, but with additional computational overhead. We explore a variety of tradeoffs between scoring metric and processing regime and present new insight into how score-safe algorithms interact with rank scoring.
{"title":"Exploring the magic of WAND","authors":"M. Petri, J. Culpepper, Alistair Moffat","doi":"10.1145/2537734.2537744","DOIUrl":"https://doi.org/10.1145/2537734.2537744","url":null,"abstract":"Web search services process thousands of queries per second, and filter their answers from collections containing very large amounts of data. Fast response to queries is a critical service expectation. The well-known WAND processing strategy is one way of reducing the amount of computation necessary when executing such a query. The value of WAND has now been validated in a wide range of studies, and has become one of the key baselines against which all new top-k processing algorithms are benchmarked. However, most previous implementations of WAND-based retrieval approaches have been in the context of the BM25 Okapi similarity scoring regime. Here we measure the performance of WAND in the context of the alternative Language Model similarity score computation, and find that the dramatic efficiency gains reported in previous studies are no longer achievable. That is, when the primary goal of a retrieval system is to maximize effectiveness, WAND is relatively unhelpful in terms of attaining the secondary objective of maximizing query throughput rates. However, the BM-WAND algorithm does in fact help reducing the percentage of postings to be scored, but with additional computational overhead. We explore a variety of tradeoffs between scoring metric and processing regime and present new insight into how score-safe algorithms interact with rank scoring.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114490684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Measurement of graph centrality provides us with an indication of the importance or popularity of each vertex in a graph. When dealing with graphs that are not centrally controlled (such as the Web, social networks and academic citation graphs), centrality measure must 1) correlate with vertex importance/popularity, 2) scale well in terms of computation, and 3) be difficult to manipulate by individuals. The Random Surfer probability transition model, combined with Eigenvalue Centrality produced PageRank, which has shown to satisfy the required properties. Existing centrality measures (including PageRank) make the assumption that all directed edges are positive, implying an endorsement. Recent work on sentiment analysis has shown that this assumption is not valid. In this article, we introduce a new method of transitioning a graph, called Power Walk, that can successfully compute centrality scores for graphs with real weighted edges. We show that it satisfies the desired properties, and that its computation time and centrality ranking is similar to when using the Random Surfer model for non-negative matrices. Finally, stability and convergence analysis shows us that both stability and convergence when using the power method, are dependent on the Power Walk parameter β.
{"title":"Power walk: revisiting the random surfer","authors":"L. Park, S. Simoff","doi":"10.1145/2537734.2537749","DOIUrl":"https://doi.org/10.1145/2537734.2537749","url":null,"abstract":"Measurement of graph centrality provides us with an indication of the importance or popularity of each vertex in a graph. When dealing with graphs that are not centrally controlled (such as the Web, social networks and academic citation graphs), centrality measure must 1) correlate with vertex importance/popularity, 2) scale well in terms of computation, and 3) be difficult to manipulate by individuals. The Random Surfer probability transition model, combined with Eigenvalue Centrality produced PageRank, which has shown to satisfy the required properties. Existing centrality measures (including PageRank) make the assumption that all directed edges are positive, implying an endorsement. Recent work on sentiment analysis has shown that this assumption is not valid. In this article, we introduce a new method of transitioning a graph, called Power Walk, that can successfully compute centrality scores for graphs with real weighted edges. We show that it satisfies the desired properties, and that its computation time and centrality ranking is similar to when using the Random Surfer model for non-negative matrices. Finally, stability and convergence analysis shows us that both stability and convergence when using the power method, are dependent on the Power Walk parameter β.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115233638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.
{"title":"Towards information retrieval evaluation with reduced and only positive judgements","authors":"Diego Mollá Aliod, David Martínez, Iman Amini","doi":"10.1145/2537734.2537748","DOIUrl":"https://doi.org/10.1145/2537734.2537748","url":null,"abstract":"This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126793482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we discuss some of the document encoding errors that were found when scaling our indexer and search engine up to large collections crawled from the web, such as ClueWeb09. In this paper we describe the encoding errors, what effect they could have on indexing and searching, how they are processed within our indexer and search engine and how they relate to the quality of the page measured by another method.
{"title":"Malformed UTF-8 and spam","authors":"Matt Crane, A. Trotman, Richard A. O'Keefe","doi":"10.1145/2537734.2537746","DOIUrl":"https://doi.org/10.1145/2537734.2537746","url":null,"abstract":"In this paper we discuss some of the document encoding errors that were found when scaling our indexer and search engine up to large collections crawled from the web, such as ClueWeb09. In this paper we describe the encoding errors, what effect they could have on indexing and searching, how they are processed within our indexer and search engine and how they relate to the quality of the page measured by another method.","PeriodicalId":402985,"journal":{"name":"Australasian Document Computing Symposium","volume":"200 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116155963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：