首页 > 最新文献

Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval最新文献

英文 中文
Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning 提到Twitter中使用协同多智能体强化学习的推荐
Tao Gui, Peng Liu, Qi Zhang, Liang Zhu, Minlong Peng, Yunhua Zhou, Xuanjing Huang
In Twitter-like social networking services, the "@'' symbol can be used with the tweet to mention users whom the user wants to alert regarding the message. An automatic suggestion to the user of a small list of candidate names can improve communication efficiency. Previous work usually used several most recent tweets or randomly select historical tweets to make an inference about this preferred list of names. However, because there are too many historical tweets by users and a wide variety of content types, the use of several tweets cannot guarantee the desired results. In this work, we propose the use of a novel cooperative multi-agent approach to mention recommendation, which incorporates dozens of more historical tweets than earlier approaches. The proposed method can effectively select a small set of historical tweets and cooperatively extract relevant indicator tweets from both the user and mentioned users. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.
在类似twitter的社交网络服务中,“@”符号可以与tweet一起使用,以提及用户想要提醒的用户。自动向用户推荐少量候选名单可以提高通信效率。以前的工作通常使用最近的几条推文或随机选择历史推文来对这个首选名称列表进行推断。但是,由于用户的历史tweets太多,内容类型繁多,使用多个tweets并不能保证达到预期的效果。在这项工作中,我们提出使用一种新颖的合作多智能体方法来提及推荐,它比以前的方法包含了更多的历史推文。该方法可以有效地选择一小部分历史推文,并从用户和被提及用户中协同提取相关的指标推文。实验结果表明,该方法优于现有的方法。
{"title":"Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning","authors":"Tao Gui, Peng Liu, Qi Zhang, Liang Zhu, Minlong Peng, Yunhua Zhou, Xuanjing Huang","doi":"10.1145/3331184.3331237","DOIUrl":"https://doi.org/10.1145/3331184.3331237","url":null,"abstract":"In Twitter-like social networking services, the \"@'' symbol can be used with the tweet to mention users whom the user wants to alert regarding the message. An automatic suggestion to the user of a small list of candidate names can improve communication efficiency. Previous work usually used several most recent tweets or randomly select historical tweets to make an inference about this preferred list of names. However, because there are too many historical tweets by users and a wide variety of content types, the use of several tweets cannot guarantee the desired results. In this work, we propose the use of a novel cooperative multi-agent approach to mention recommendation, which incorporates dozens of more historical tweets than earlier approaches. The proposed method can effectively select a small set of historical tweets and cooperatively extract relevant indicator tweets from both the user and mentioned users. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"17 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77783667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Effective Online Evaluation for Web Search 网络搜索的有效在线评估
Alexey Drutsa, Gleb Gusev, E. Kharitonov, Denis Kulemyakin, P. Serdyukov, I. Yashkov
We present you a program of a balanced mix between an overview of academic achievements in the field of online evaluation and a portion of unique industrial practical experience shared by both the leading researchers and engineers from global Internet companies. First, we give basic knowledge from mathematical statistics. This is followed by foundations of main evaluation methods such as A/B testing, interleaving, and observational studies. Then, we share rich industrial experiences on constructing of an experimentation pipeline and evaluation metrics (emphasizing best practices and common pitfalls). A large part of our tutorial is devoted to modern and state-of-the-art techniques (including the ones based on machine learning) that allow to conduct online experimentation efficiently. We invite software engineers, designers, analysts, and managers of web services and software products, as well as beginners, advanced specialists, and researchers to learn how to make web service development effectively data-driven.
我们为您提供一个平衡组合的课程,既有在线评估领域的学术成就概述,也有来自全球互联网公司的领先研究人员和工程师分享的部分独特的工业实践经验。首先,我们给出数理统计的基础知识。其次是主要评估方法的基础,如A/B测试、交错和观察性研究。然后,我们在构建实验管道和评估度量(强调最佳实践和常见缺陷)方面分享了丰富的工业经验。我们教程的很大一部分致力于现代和最先进的技术(包括基于机器学习的技术),这些技术允许有效地进行在线实验。我们邀请web服务和软件产品的软件工程师、设计师、分析师和管理人员,以及初学者、高级专家和研究人员来学习如何使web服务开发有效地由数据驱动。
{"title":"Effective Online Evaluation for Web Search","authors":"Alexey Drutsa, Gleb Gusev, E. Kharitonov, Denis Kulemyakin, P. Serdyukov, I. Yashkov","doi":"10.1145/3331184.3331378","DOIUrl":"https://doi.org/10.1145/3331184.3331378","url":null,"abstract":"We present you a program of a balanced mix between an overview of academic achievements in the field of online evaluation and a portion of unique industrial practical experience shared by both the leading researchers and engineers from global Internet companies. First, we give basic knowledge from mathematical statistics. This is followed by foundations of main evaluation methods such as A/B testing, interleaving, and observational studies. Then, we share rich industrial experiences on constructing of an experimentation pipeline and evaluation metrics (emphasizing best practices and common pitfalls). A large part of our tutorial is devoted to modern and state-of-the-art techniques (including the ones based on machine learning) that allow to conduct online experimentation efficiently. We invite software engineers, designers, analysts, and managers of web services and software products, as well as beginners, advanced specialists, and researchers to learn how to make web service development effectively data-driven.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"110 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76048875","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Which Diversity Evaluation Measures Are "Good"? 哪些多样性评估措施是“好的”?
T. Sakai, Zhaohao Zeng
This study evaluates 30 IR evaluation measures or their instances, of which nine are for adhoc IR and 21 are for diversified IR, primarily from the viewpoint of whether their preferences of one SERP (search engine result page) over another actually align with users' preferences. The gold preferences were contructed by hiring 15 assessors, who independently examined 1,127 SERP pairs and made preference assessments. Two sets of preference assessments were obtained: one based on a relevance question "Which SERP is more relevant?'' and the other based on a diversity question "Which SERP is likely to satisfy a higher number of users?'' To our knowledge, our study is the first to have collected diversity preference assessments in this way and evaluated diversity measures successfully. Our main results are that (a) Popular adhoc IR measures such as nDCG actually align quite well with the gold relevance preferences; and that (b) While the ♯-measures align well with the gold diversity preferences, intent-aware measures perform relatively poorly. Moreover, as by-products of our analysis of existing evaluation measures, we define new adhoc measures called iRBU (intentwise Rank-Biased Utility) and EBR (Expected Blended Ratio); we demonstrate that an instance of iRBU performs as well as nDCG when compared to the gold relevance preferences. On the other hand, the original RBU, a recently-proposed diversity measure, underperforms the best ♯-measures when compared to the gold diversity preferences.
本研究评估了30个IR评估措施或其实例,其中9个用于特殊IR, 21个用于多样化IR,主要是从他们对一个SERP(搜索引擎结果页面)的偏好是否与另一个用户的偏好相一致的角度出发。黄金偏好由15名评估人员构建,他们独立检查了1,127对SERP并进行了偏好评估。获得了两组偏好评估:一组基于相关性问题“哪个SERP更相关?”,另一个则是基于一个多样性问题:“哪个SERP可能满足更多的用户?”“据我们所知,我们的研究是第一个以这种方式收集多样性偏好评估并成功评估多样性措施的研究。我们的主要结果是:(a)流行的临时IR指标,如nDCG,实际上与黄金相关性偏好相当一致;并且(b)虽然# -措施与黄金多样性偏好很好地一致,但意图意识措施表现相对较差。此外,作为我们对现有评价措施分析的副产品,我们定义了新的特别措施,称为iRBU(故意秩偏效用)和EBR(预期混合比率);我们证明,与黄金相关偏好相比,iRBU实例的表现与nDCG一样好。另一方面,与黄金多样性偏好相比,最初的RBU,最近提出的多样性指标,表现不如最佳# -指标。
{"title":"Which Diversity Evaluation Measures Are \"Good\"?","authors":"T. Sakai, Zhaohao Zeng","doi":"10.1145/3331184.3331215","DOIUrl":"https://doi.org/10.1145/3331184.3331215","url":null,"abstract":"This study evaluates 30 IR evaluation measures or their instances, of which nine are for adhoc IR and 21 are for diversified IR, primarily from the viewpoint of whether their preferences of one SERP (search engine result page) over another actually align with users' preferences. The gold preferences were contructed by hiring 15 assessors, who independently examined 1,127 SERP pairs and made preference assessments. Two sets of preference assessments were obtained: one based on a relevance question \"Which SERP is more relevant?'' and the other based on a diversity question \"Which SERP is likely to satisfy a higher number of users?'' To our knowledge, our study is the first to have collected diversity preference assessments in this way and evaluated diversity measures successfully. Our main results are that (a) Popular adhoc IR measures such as nDCG actually align quite well with the gold relevance preferences; and that (b) While the ♯-measures align well with the gold diversity preferences, intent-aware measures perform relatively poorly. Moreover, as by-products of our analysis of existing evaluation measures, we define new adhoc measures called iRBU (intentwise Rank-Biased Utility) and EBR (Expected Blended Ratio); we demonstrate that an instance of iRBU performs as well as nDCG when compared to the gold relevance preferences. On the other hand, the original RBU, a recently-proposed diversity measure, underperforms the best ♯-measures when compared to the gold diversity preferences.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"62 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85783501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 36
Web Table Extraction, Retrieval and Augmentation Web表提取,检索和增强
Shuo Zhang, K. Balog
This tutorial synthesizes and presents research on web tables over the past two decades. We group the tasks into six main categories of information access tasks: (i) table extraction, (ii) table interpretation, (iii) table search, (iv) question answering on tables, (v) knowledge base augmentation, and (vi) table completion. For each category, we identify and introduce seminal approaches, present relevant resources, and point out interdependencies among the different tasks.
本教程综合并介绍了过去二十年来对web表的研究。我们将任务分为六大类信息访问任务:(i)表提取,(ii)表解释,(iii)表搜索,(iv)表上的问题回答,(v)知识库扩充,(vi)表补全。对于每个类别,我们确定并介绍开创性的方法,提供相关资源,并指出不同任务之间的相互依赖性。
{"title":"Web Table Extraction, Retrieval and Augmentation","authors":"Shuo Zhang, K. Balog","doi":"10.1145/3331184.3331385","DOIUrl":"https://doi.org/10.1145/3331184.3331385","url":null,"abstract":"This tutorial synthesizes and presents research on web tables over the past two decades. We group the tasks into six main categories of information access tasks: (i) table extraction, (ii) table interpretation, (iii) table search, (iv) question answering on tables, (v) knowledge base augmentation, and (vi) table completion. For each category, we identify and introduce seminal approaches, present relevant resources, and point out interdependencies among the different tasks.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88185123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 36
Fast Approximate Filtering of Search Results Sorted by Attribute 按属性排序搜索结果的快速近似过滤
F. M. Nardini, Roberto Trani, Rossano Venturini
Several Web search services enable their users with the possibility of sorting the list of results by a specific attribute, e.g., sort "by price" in e-commerce. However, sorting the results by attribute could bring marginally relevant results in the top positions thus leading to a poor user experience. This motivates the definition of the relevance-aware filtering problem. This problem asks to remove results from the attribute-sorted list to maximize its final overall relevance. Recently, an optimal solution to this problem has been proposed. However, it has strong limitations in the Web scenario due to its high computational cost. In this paper, we propose ϵ-Filtering: an efficient approximate algorithm with strong approximation guarantees on the relevance of the final list. More precisely, given an allowed approximation error ϵ, the proposed algorithm finds a(1-ϵ)"optimal filtering, i.e., the relevance of its solution is at least (1-ϵ) times the optimum. We conduct a comprehensive evaluation of ϵ-Filtering against state-of-the-art competitors on two real-world public datasets. Experiments show that ϵ-Filtering achieves the desired levels of effectiveness with a speedup of up to two orders of magnitude with respect to the optimal solution while guaranteeing very small approximation errors.
一些Web搜索服务使用户能够按特定属性对结果列表进行排序,例如,电子商务中的“按价格”排序。然而,按属性排序结果可能会在顶部位置带来不太相关的结果,从而导致糟糕的用户体验。这激发了相关感知过滤问题的定义。这个问题要求从属性排序列表中删除结果,以最大化其最终的总体相关性。最近,有人提出了这个问题的最优解。然而,由于计算成本高,它在Web场景中有很强的局限性。在本文中,我们提出了ϵ-Filtering:一个有效的近似算法,对最终列表的相关性有很强的近似保证。更准确地说,给定一个允许的近似误差,所提出的算法找到一个(1- ε)最优滤波,即其解的相关性至少是最优的(1- λ)倍。我们在两个真实世界的公共数据集上对ϵ-Filtering与最先进的竞争对手进行了全面的评估。实验表明,ϵ-Filtering在保证非常小的近似误差的同时,相对于最优解的加速高达两个数量级,达到了所需的效率水平。
{"title":"Fast Approximate Filtering of Search Results Sorted by Attribute","authors":"F. M. Nardini, Roberto Trani, Rossano Venturini","doi":"10.1145/3331184.3331227","DOIUrl":"https://doi.org/10.1145/3331184.3331227","url":null,"abstract":"Several Web search services enable their users with the possibility of sorting the list of results by a specific attribute, e.g., sort \"by price\" in e-commerce. However, sorting the results by attribute could bring marginally relevant results in the top positions thus leading to a poor user experience. This motivates the definition of the relevance-aware filtering problem. This problem asks to remove results from the attribute-sorted list to maximize its final overall relevance. Recently, an optimal solution to this problem has been proposed. However, it has strong limitations in the Web scenario due to its high computational cost. In this paper, we propose ϵ-Filtering: an efficient approximate algorithm with strong approximation guarantees on the relevance of the final list. More precisely, given an allowed approximation error ϵ, the proposed algorithm finds a(1-ϵ)\"optimal filtering, i.e., the relevance of its solution is at least (1-ϵ) times the optimum. We conduct a comprehensive evaluation of ϵ-Filtering against state-of-the-art competitors on two real-world public datasets. Experiments show that ϵ-Filtering achieves the desired levels of effectiveness with a speedup of up to two orders of magnitude with respect to the optimal solution while guaranteeing very small approximation errors.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"33 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88444377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Learning from Fact-checkers: Analysis and Generation of Fact-checking Language 向事实核查者学习:事实核查语言的分析与生成
Nguyen Vo, Kyumin Lee
In fighting against fake news, many fact-checking systems comprised of human-based fact-checking sites (e.g., snopes.com and politifact.com) and automatic detection systems have been developed in recent years. However, online users still keep sharing fake news even when it has been debunked. It means that early fake news detection may be insufficient and we need another complementary approach to mitigate the spread of misinformation. In this paper, we introduce a novel application of text generation for combating fake news. In particular, we (1) leverage online users named fact-checkers, who cite fact-checking sites as credible evidences to fact-check information in public discourse; (2) analyze linguistic characteristics of fact-checking tweets; and (3) propose and build a deep learning framework to generate responses with fact-checking intention to increase the fact-checkers' engagement in fact-checking activities. Our analysis reveals that the fact-checkers tend to refute misinformation and use formal language (e.g. few swear words and Internet slangs). Our framework successfully generates relevant responses, and outperforms competing models by achieving up to 30% improvements. Our qualitative study also confirms that the superiority of our generated responses compared with responses generated from the existing models.
在打击假新闻方面,近年来开发了许多由人工事实核查网站(如snopes.com和politifact.com)和自动检测系统组成的事实核查系统。然而,即使假新闻已经被揭穿,网民们仍然会继续分享假新闻。这意味着早期的假新闻检测可能是不够的,我们需要另一种补充方法来减轻错误信息的传播。在本文中,我们介绍了一种新的文本生成用于打击假新闻的应用。特别是,我们(1)利用被称为事实核查者的在线用户,他们引用事实核查网站作为可信证据,对公共话语中的信息进行事实核查;(2)分析事实核查推文的语言特征;(3)提出并构建一个深度学习框架,生成具有事实核查意图的回应,以提高事实核查者在事实核查活动中的参与度。我们的分析表明,事实核查者倾向于驳斥错误信息,并使用正式语言(例如很少使用脏话和网络俚语)。我们的框架成功地产生了相关的响应,并通过实现高达30%的改进而优于竞争模型。我们的定性研究也证实了我们生成的响应与现有模型生成的响应相比的优越性。
{"title":"Learning from Fact-checkers: Analysis and Generation of Fact-checking Language","authors":"Nguyen Vo, Kyumin Lee","doi":"10.1145/3331184.3331248","DOIUrl":"https://doi.org/10.1145/3331184.3331248","url":null,"abstract":"In fighting against fake news, many fact-checking systems comprised of human-based fact-checking sites (e.g., snopes.com and politifact.com) and automatic detection systems have been developed in recent years. However, online users still keep sharing fake news even when it has been debunked. It means that early fake news detection may be insufficient and we need another complementary approach to mitigate the spread of misinformation. In this paper, we introduce a novel application of text generation for combating fake news. In particular, we (1) leverage online users named fact-checkers, who cite fact-checking sites as credible evidences to fact-check information in public discourse; (2) analyze linguistic characteristics of fact-checking tweets; and (3) propose and build a deep learning framework to generate responses with fact-checking intention to increase the fact-checkers' engagement in fact-checking activities. Our analysis reveals that the fact-checkers tend to refute misinformation and use formal language (e.g. few swear words and Internet slangs). Our framework successfully generates relevant responses, and outperforms competing models by achieving up to 30% improvements. Our qualitative study also confirms that the superiority of our generated responses compared with responses generated from the existing models.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"40 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88057025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 51
Teach Machine How to Read: Reading Behavior Inspired Relevance Estimation 教机器如何阅读:阅读行为启发相关性估计
Xiangsheng Li, Jiaxin Mao, Chao Wang, Yiqun Liu, Min Zhang, Shaoping Ma
Retrieval models aim to estimate the relevance of a document to a certain query. Although existing retrieval models have gained much success in both deepening our understanding of information seeking behavior and constructing practical retrieval systems (e.g. Web search engines), we have to admit that the models work in a rather different manner than how humans make relevance judgments. In this paper, we aim to reexamine the existing models as well as to propose new ones based on the findings in how human read documents during relevance judgment. First, we summarize a number of reading heuristics from practical user behavior patterns, which are categorized into implicit and explicit heuristics. By reviewing a variety of existing retrieval models, we find that most of them only satisfy a part of these reading heuristics. To evaluate the effectiveness of each heuristic, we conduct an ablation study and find that most heuristics have positive impacts on retrieval performance. We further integrate all the effective heuristics into a new retrieval model named Reading Inspired Model (RIM). Specifically, implicit reading heuristics are incorporated into the model framework and explicit reading heuristics are modeled as a Markov Decision Process and learned by reinforcement learning. Experimental results on a large-scale public available benchmark dataset and two test sets from NTCIR WWW tasks show that RIM outperforms most existing models, which illustrates the effectiveness of the reading heuristics. We believe that this work contributes to constructing retrieval models with both higher retrieval performance and better explainability.
检索模型的目的是估计文档与某个查询的相关性。尽管现有的检索模型在加深我们对信息寻找行为的理解和构建实用的检索系统(例如Web搜索引擎)方面取得了很大的成功,但我们不得不承认,这些模型的工作方式与人类做出相关性判断的方式相当不同。在本文中,我们的目的是重新审视现有的模型,并提出新的基于人类如何阅读文件在相关性判断的研究结果。首先,我们从实际用户行为模式中总结了一些阅读启发式,它们分为内隐启发式和外显启发式。通过对现有的各种检索模型的回顾,我们发现大多数检索模型只能满足这些阅读启发式的一部分。为了评估每个启发式的有效性,我们进行了一个消融研究,发现大多数启发式对检索性能有积极的影响。我们进一步将所有有效的启发式方法整合到一个新的检索模型中,称为阅读启发模型(RIM)。具体而言,内隐阅读启发式被纳入模型框架,外显阅读启发式被建模为马尔可夫决策过程,并通过强化学习进行学习。在大规模公共基准数据集和NTCIR WWW任务的两个测试集上的实验结果表明,RIM优于大多数现有模型,这说明了阅读启发式算法的有效性。我们相信这项工作有助于构建具有更高检索性能和更好可解释性的检索模型。
{"title":"Teach Machine How to Read: Reading Behavior Inspired Relevance Estimation","authors":"Xiangsheng Li, Jiaxin Mao, Chao Wang, Yiqun Liu, Min Zhang, Shaoping Ma","doi":"10.1145/3331184.3331205","DOIUrl":"https://doi.org/10.1145/3331184.3331205","url":null,"abstract":"Retrieval models aim to estimate the relevance of a document to a certain query. Although existing retrieval models have gained much success in both deepening our understanding of information seeking behavior and constructing practical retrieval systems (e.g. Web search engines), we have to admit that the models work in a rather different manner than how humans make relevance judgments. In this paper, we aim to reexamine the existing models as well as to propose new ones based on the findings in how human read documents during relevance judgment. First, we summarize a number of reading heuristics from practical user behavior patterns, which are categorized into implicit and explicit heuristics. By reviewing a variety of existing retrieval models, we find that most of them only satisfy a part of these reading heuristics. To evaluate the effectiveness of each heuristic, we conduct an ablation study and find that most heuristics have positive impacts on retrieval performance. We further integrate all the effective heuristics into a new retrieval model named Reading Inspired Model (RIM). Specifically, implicit reading heuristics are incorporated into the model framework and explicit reading heuristics are modeled as a Markov Decision Process and learned by reinforcement learning. Experimental results on a large-scale public available benchmark dataset and two test sets from NTCIR WWW tasks show that RIM outperforms most existing models, which illustrates the effectiveness of the reading heuristics. We believe that this work contributes to constructing retrieval models with both higher retrieval performance and better explainability.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88639624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 26
Similarity-Based Synthetic Document Representations for Meta-Feature Generation in Text Classification 基于相似度的文本分类元特征生成合成文档表示
Sérgio D. Canuto, Thiago Salles, Thierson Couto, Marcos André Gonçalves
We propose new solutions that enhance and extend the already very successful application of meta-features to text classification. Our newly proposed meta-features are capable of: (1) improving the correlation of small pieces of evidence shared by neighbors with labeled categories by means of synthetic document representations and (local and global) hyperplane distances; and (2) estimating the level of error introduced by these newly proposed and the existing meta-features in the literature, specially for hard-to-classify regions of the feature space. Our experiments with large and representative number of datasets show that our new solutions produce the best results in all tested scenarios, achieving gains of up to 12% over the strongest meta-feature proposal of the literature.
我们提出了新的解决方案,以增强和扩展已经非常成功的元特征在文本分类中的应用。我们新提出的元特征能够:(1)通过合成文档表示和(局部和全局)超平面距离,改善带有标记类别的邻居共享的小块证据的相关性;(2)估计这些新提出的元特征和文献中已有的元特征引入的误差水平,特别是对特征空间中难以分类的区域。我们对大量具有代表性的数据集进行的实验表明,我们的新解决方案在所有测试场景中都产生了最好的结果,比文献中最强的元特征提案获得了高达12%的收益。
{"title":"Similarity-Based Synthetic Document Representations for Meta-Feature Generation in Text Classification","authors":"Sérgio D. Canuto, Thiago Salles, Thierson Couto, Marcos André Gonçalves","doi":"10.1145/3331184.3331239","DOIUrl":"https://doi.org/10.1145/3331184.3331239","url":null,"abstract":"We propose new solutions that enhance and extend the already very successful application of meta-features to text classification. Our newly proposed meta-features are capable of: (1) improving the correlation of small pieces of evidence shared by neighbors with labeled categories by means of synthetic document representations and (local and global) hyperplane distances; and (2) estimating the level of error introduced by these newly proposed and the existing meta-features in the literature, specially for hard-to-classify regions of the feature space. Our experiments with large and representative number of datasets show that our new solutions produce the best results in all tested scenarios, achieving gains of up to 12% over the strongest meta-feature proposal of the literature.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75651549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 15
Session details: Session 1B: Health and Social Media 会议详情:会议1B:健康和社交媒体
Mark D. Smucker
{"title":"Session details: Session 1B: Health and Social Media","authors":"Mark D. Smucker","doi":"10.1145/3349676","DOIUrl":"https://doi.org/10.1145/3349676","url":null,"abstract":"","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90301308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reinforcement Learning for User Intent Prediction in Customer Service Bots 客服机器人中用户意图预测的强化学习
Cen Chen, Chilin Fu, Xujun Hu, Xiaolu Zhang, Jun Zhou, Xiaolong Li, F. S. Bao
A customer service bot is now a necessary component of an e-commerce platform. As a core module of the customer service bot, user intent prediction can help predict user questions before they ask. A typical solution is to find top candidate questions that a user will be interested in. Such solution ignores the inter-relationship between questions and often aims to maximize the immediate reward such as clicks, which may not be ideal in practice. Hence, we propose to view the problem as a sequential decision making process to better capture the long-term effects of each recommendation in the list. Intuitively, we formulate the problem as a Markov decision process and consider using reinforcement learning for the problem. With this approach, questions presented to users are both relevant and diverse. Experiments on offline real-world dataset and online system demonstrate the effectiveness of our proposed approach.
客户服务机器人现在是电子商务平台的必要组成部分。用户意图预测是客服机器人的核心模块,可以在用户提问之前预测用户的问题。一个典型的解决方案是找到用户可能感兴趣的最佳候选问题。这样的解决方案忽略了问题之间的相互关系,通常旨在最大化点击等即时奖励,这在实践中可能并不理想。因此,我们建议将问题视为一个连续的决策制定过程,以更好地捕获列表中每个建议的长期影响。直观地,我们将问题表述为马尔可夫决策过程,并考虑使用强化学习来解决问题。通过这种方法,呈现给用户的问题既相关又多样。在离线真实数据集和在线系统上的实验证明了该方法的有效性。
{"title":"Reinforcement Learning for User Intent Prediction in Customer Service Bots","authors":"Cen Chen, Chilin Fu, Xujun Hu, Xiaolu Zhang, Jun Zhou, Xiaolong Li, F. S. Bao","doi":"10.1145/3331184.3331370","DOIUrl":"https://doi.org/10.1145/3331184.3331370","url":null,"abstract":"A customer service bot is now a necessary component of an e-commerce platform. As a core module of the customer service bot, user intent prediction can help predict user questions before they ask. A typical solution is to find top candidate questions that a user will be interested in. Such solution ignores the inter-relationship between questions and often aims to maximize the immediate reward such as clicks, which may not be ideal in practice. Hence, we propose to view the problem as a sequential decision making process to better capture the long-term effects of each recommendation in the list. Intuitively, we formulate the problem as a Markov decision process and consider using reinforcement learning for the problem. With this approach, questions presented to users are both relevant and diverse. Experiments on offline real-world dataset and online system demonstrate the effectiveness of our proposed approach.","PeriodicalId":20700,"journal":{"name":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":"12 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81381496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 15
期刊
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1