The ubiquitous availability of mobile devices with GPS capabilities and the popularity of social media platforms have created a rich source for textual data with spatio-temporal information. Also, other domains like crime incident description and search engine queries, can provide spatio-temporal textual data. These data sources can be used to discover space-time related insights of human behavior. This work focuses on modeling text that is associated with a particular time and place. We extend the traditional language modeling task from natural language processing to language modeling under spatio-temporal conditions. This task definition allows us to use the same evaluation framework used in language modeling. A model for spatio-temporal text data representation should be able to capture the patterns that guide how text is generated in a spatio-temporal context. We aim to develop neural network models for language modeling conditioned on spatio-temporal variables with the ability to capture properties such as: neighborhood, periodicity and hierarchy.
{"title":"Spatio-temporal Conditioned Language Models","authors":"Juglar Diaz","doi":"10.1145/3397271.3401450","DOIUrl":"https://doi.org/10.1145/3397271.3401450","url":null,"abstract":"The ubiquitous availability of mobile devices with GPS capabilities and the popularity of social media platforms have created a rich source for textual data with spatio-temporal information. Also, other domains like crime incident description and search engine queries, can provide spatio-temporal textual data. These data sources can be used to discover space-time related insights of human behavior. This work focuses on modeling text that is associated with a particular time and place. We extend the traditional language modeling task from natural language processing to language modeling under spatio-temporal conditions. This task definition allows us to use the same evaluation framework used in language modeling. A model for spatio-temporal text data representation should be able to capture the patterns that guide how text is generated in a spatio-temporal context. We aim to develop neural network models for language modeling conditioned on spatio-temporal variables with the ability to capture properties such as: neighborhood, periodicity and hierarchy.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127780140","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The most important unsolved problem with artificial neural networks is how to do unsupervised learning as effectively as the brain. There are currently two main approaches to unsupervised learning. In the first approach, exemplified by BERT and Variational Autoencoders, a deep neural network is used to reconstruct its input. This is problematic for images because the deepest layers of the network need to encode the fine details of the image. An alternative approach, introduced by Becker and Hinton in 1992, is to train two copies of a deep neural network to produce output vectors that have high mutual information when given two different crops of the same image as their inputs. This approach was designed to allow the representations to be untethered from irrelevant details of the input. The method of optimizing mutual information used by Becker and Hinton was flawed (for a subtle reason that I will explain) so Pacannaro and Hinton (2001) replaced it by a discriminative objective in which one vector representation must select a corresponding vector representation from among many alternatives. With faster hardware, contrastive learning of representations has recently become very popular and is proving to be very effective, but it suffers from a major flaw: To learn pairs of representation vectors that have N bits of mutual information we need to contrast the correct corresponding vector with about 2N incorrect alternatives. I will describe a novel and effective way of dealing with this limitation. I will also show that this leads to a simple way of implementing perceptual learning in cortex.
{"title":"The Next Generation of Neural Networks","authors":"Geoffrey E. Hinton","doi":"10.1145/3397271.3402425","DOIUrl":"https://doi.org/10.1145/3397271.3402425","url":null,"abstract":"The most important unsolved problem with artificial neural networks is how to do unsupervised learning as effectively as the brain. There are currently two main approaches to unsupervised learning. In the first approach, exemplified by BERT and Variational Autoencoders, a deep neural network is used to reconstruct its input. This is problematic for images because the deepest layers of the network need to encode the fine details of the image. An alternative approach, introduced by Becker and Hinton in 1992, is to train two copies of a deep neural network to produce output vectors that have high mutual information when given two different crops of the same image as their inputs. This approach was designed to allow the representations to be untethered from irrelevant details of the input. The method of optimizing mutual information used by Becker and Hinton was flawed (for a subtle reason that I will explain) so Pacannaro and Hinton (2001) replaced it by a discriminative objective in which one vector representation must select a corresponding vector representation from among many alternatives. With faster hardware, contrastive learning of representations has recently become very popular and is proving to be very effective, but it suffers from a major flaw: To learn pairs of representation vectors that have N bits of mutual information we need to contrast the correct corresponding vector with about 2N incorrect alternatives. I will describe a novel and effective way of dealing with this limitation. I will also show that this leads to a simple way of implementing perceptual learning in cortex.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127376582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yashar Deldjoo, T. D. Noia, E. Sciascio, Felice Antonio Merra
Shilling attacks against collaborative filtering (CF) models are characterized by several fake user profiles mounted on the system by an adversarial party to harvest recommendation outcomes toward a malicious desire. The vulnerability of CF models is directly tied with their reliance on the underlying interaction data ---like user-item rating matrix (URM) --- to train their models and their inherent inability to distinguish genuine profiles from non-genuine ones. The majority of works conducted so far for analyzing shilling attacks mainly focused on properties such as confronted recommendation models, recommendation outputs, and even users under attack. The under-researched element has been the impact of data characteristics on the effectiveness of shilling attacks on CF models. Toward this goal, this work presents a systematic and in-depth study by using an analytical modeling approach built on a regression model to test the hypothesis of whether URM properties can impact the outcome of CF recommenders under a shilling attack. We ran extensive experiments involving 97200 simulations on three different domains (movie, business, and music), and showed that URM properties considerably affect the robustness of CF models in shilling attack scenarios. Obtained results can be of great help for the system designer in understanding the cause of variations in a recommender system performance due to a shilling attack.
{"title":"How Dataset Characteristics Affect the Robustness of Collaborative Recommendation Models","authors":"Yashar Deldjoo, T. D. Noia, E. Sciascio, Felice Antonio Merra","doi":"10.1145/3397271.3401046","DOIUrl":"https://doi.org/10.1145/3397271.3401046","url":null,"abstract":"Shilling attacks against collaborative filtering (CF) models are characterized by several fake user profiles mounted on the system by an adversarial party to harvest recommendation outcomes toward a malicious desire. The vulnerability of CF models is directly tied with their reliance on the underlying interaction data ---like user-item rating matrix (URM) --- to train their models and their inherent inability to distinguish genuine profiles from non-genuine ones. The majority of works conducted so far for analyzing shilling attacks mainly focused on properties such as confronted recommendation models, recommendation outputs, and even users under attack. The under-researched element has been the impact of data characteristics on the effectiveness of shilling attacks on CF models. Toward this goal, this work presents a systematic and in-depth study by using an analytical modeling approach built on a regression model to test the hypothesis of whether URM properties can impact the outcome of CF recommenders under a shilling attack. We ran extensive experiments involving 97200 simulations on three different domains (movie, business, and music), and showed that URM properties considerably affect the robustness of CF models in shilling attack scenarios. Obtained results can be of great help for the system designer in understanding the cause of variations in a recommender system performance due to a shilling attack.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133738276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hongshen Chen, Z. Ren, Pengjie Ren, Dawei Yin, Xiaodong He
Nowadays, intelligent information systems, especially the interactive information systems (e.g., conversational interaction systems like Siri, and Cortana; news feed recommender systems, and interactive search engines, etc.), are ubiquitous in real-world applications. These systems either converse with users explicitly through natural languages, or mine users interests and respond to users requests implicitly. Interactivity has become a crucial element towards intelligent information systems. Despite the fact that interactive information systems have gained significant progress, there are still many challenges to be addressed when applying these models to real-world scenarios. This half day workshop explores challenges and potential research, development, and application directions in applied interactive information systems. We aim to discuss the issues of applying interactive information models to production systems, as well as to shed some light on the fundamental characteristics, i.e., interactivity and applicability, of different interactive tasks. We welcome practical, theoretical, experimental, and methodological studies that advances the interactivity towards intelligent information systems. The workshop aims to bring together a diverse set of practitioners and researchers interested in investigating the interaction between human and information systems to develop more intelligent information systems.
{"title":"AIIS: The SIGIR 2020 Workshop on Applied Interactive Information Systems","authors":"Hongshen Chen, Z. Ren, Pengjie Ren, Dawei Yin, Xiaodong He","doi":"10.1145/3397271.3401461","DOIUrl":"https://doi.org/10.1145/3397271.3401461","url":null,"abstract":"Nowadays, intelligent information systems, especially the interactive information systems (e.g., conversational interaction systems like Siri, and Cortana; news feed recommender systems, and interactive search engines, etc.), are ubiquitous in real-world applications. These systems either converse with users explicitly through natural languages, or mine users interests and respond to users requests implicitly. Interactivity has become a crucial element towards intelligent information systems. Despite the fact that interactive information systems have gained significant progress, there are still many challenges to be addressed when applying these models to real-world scenarios. This half day workshop explores challenges and potential research, development, and application directions in applied interactive information systems. We aim to discuss the issues of applying interactive information models to production systems, as well as to shed some light on the fundamental characteristics, i.e., interactivity and applicability, of different interactive tasks. We welcome practical, theoretical, experimental, and methodological studies that advances the interactivity towards intelligent information systems. The workshop aims to bring together a diverse set of practitioners and researchers interested in investigating the interaction between human and information systems to develop more intelligent information systems.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133535892","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jun Xu, Zeng Wei, Long Xia, Yanyan Lan, Dawei Yin, Xueqi Cheng, Ji-rong Wen
This paper concerns reinforcement learning~(RL) of the document ranking models for information retrieval~(IR). One branch of the RL approaches to ranking formalize the process of ranking with Markov decision process~(MDP) and determine the model parameters with policy gradient. Though preliminary success has been shown, these approaches are still far from achieving their full potentials. Existing policy gradient methods directly utilize the absolute performance scores (returns) of the sampled document lists in its gradient estimations, which may cause two limitations: 1) fail to reflect the relative goodness of documents within the same query, which usually is close to the nature of IR ranking; 2) generate high variance gradient estimations, resulting in slow learning speed and low ranking accuracy. To deal with the issues, we propose a novel policy gradient algorithm in which the gradients are determined using pairwise comparisons of two document lists sampled within the same query. The algorithm, referred to as Pairwise Policy Gradient (PPG), repeatedly samples pairs of document lists, estimates the gradients with pairwise comparisons, and finally updates the model parameters. Theoretical analysis shows that PPG makes an unbiased and low variance gradient estimations. Experimental results have demonstrated performance gains over the state-of-the-art baselines in search result diversification and text retrieval.
{"title":"Reinforcement Learning to Rank with Pairwise Policy Gradient","authors":"Jun Xu, Zeng Wei, Long Xia, Yanyan Lan, Dawei Yin, Xueqi Cheng, Ji-rong Wen","doi":"10.1145/3397271.3401148","DOIUrl":"https://doi.org/10.1145/3397271.3401148","url":null,"abstract":"This paper concerns reinforcement learning~(RL) of the document ranking models for information retrieval~(IR). One branch of the RL approaches to ranking formalize the process of ranking with Markov decision process~(MDP) and determine the model parameters with policy gradient. Though preliminary success has been shown, these approaches are still far from achieving their full potentials. Existing policy gradient methods directly utilize the absolute performance scores (returns) of the sampled document lists in its gradient estimations, which may cause two limitations: 1) fail to reflect the relative goodness of documents within the same query, which usually is close to the nature of IR ranking; 2) generate high variance gradient estimations, resulting in slow learning speed and low ranking accuracy. To deal with the issues, we propose a novel policy gradient algorithm in which the gradients are determined using pairwise comparisons of two document lists sampled within the same query. The algorithm, referred to as Pairwise Policy Gradient (PPG), repeatedly samples pairs of document lists, estimates the gradients with pairwise comparisons, and finally updates the model parameters. Theoretical analysis shows that PPG makes an unbiased and low variance gradient estimations. Experimental results have demonstrated performance gains over the state-of-the-art baselines in search result diversification and text retrieval.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117284235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many top-k document retrieval strategies have been proposed based on the WAND and MaxScore heuristics and yet, from recent work, it is surprisingly difficult to identify the "fastest" strategy. This becomes even more challenging when considering various retrieval criteria, like different ranking models and values of k. In this paper, we conduct the first extensive comparison between ten effective strategies, many of which were never compared before to our knowledge, examining their efficiency under five representative ranking models. Based on a careful analysis of the comparison, we propose LazyBM, a remarkably simple retrieval strategy that bridges the gap between the best performing WAND-based and MaxScore-based approaches. Empirically, LazyBM considerably outperforms all of the considered strategies across ranking models, values of k, and index configurations under both mean and tail query latency.
{"title":"Finding the Best of Both Worlds: Faster and More Robust Top-k Document Retrieval","authors":"O. Khattab, Mohammad Hammoud, T. Elsayed","doi":"10.1145/3397271.3401076","DOIUrl":"https://doi.org/10.1145/3397271.3401076","url":null,"abstract":"Many top-k document retrieval strategies have been proposed based on the WAND and MaxScore heuristics and yet, from recent work, it is surprisingly difficult to identify the \"fastest\" strategy. This becomes even more challenging when considering various retrieval criteria, like different ranking models and values of k. In this paper, we conduct the first extensive comparison between ten effective strategies, many of which were never compared before to our knowledge, examining their efficiency under five representative ranking models. Based on a careful analysis of the comparison, we propose LazyBM, a remarkably simple retrieval strategy that bridges the gap between the best performing WAND-based and MaxScore-based approaches. Empirically, LazyBM considerably outperforms all of the considered strategies across ranking models, values of k, and index configurations under both mean and tail query latency.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131634269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we propose a multi-source domain adaptation method with a Granger-causal objective (MDA-GC) for cross-domain sentiment classification. Specifically, for each source domain, we build an expert model by using a novel sentiment-guided capsule network, which captures the domain invariant knowledge that bridges the knowledge gap between the source and target domains. Then, an attention mechanism is devised to assign importance weights to a mixture of experts, each of which specializes in a different source domain. In addition, we propose a Granger causal objective to make the weights assigned to individual experts correlate strongly with their contributions to the decision at hand. Experimental results on a benchmark dataset demonstrate that the proposed MDA-GC model significantly outperforms the compared methods.
{"title":"Multi-source Domain Adaptation for Sentiment Classification with Granger Causal Inference","authors":"Min Yang, Ying Shen, Xiaojun Chen, Chengming Li","doi":"10.1145/3397271.3401314","DOIUrl":"https://doi.org/10.1145/3397271.3401314","url":null,"abstract":"In this paper, we propose a multi-source domain adaptation method with a Granger-causal objective (MDA-GC) for cross-domain sentiment classification. Specifically, for each source domain, we build an expert model by using a novel sentiment-guided capsule network, which captures the domain invariant knowledge that bridges the knowledge gap between the source and target domains. Then, an attention mechanism is devised to assign importance weights to a mixture of experts, each of which specializes in a different source domain. In addition, we propose a Granger causal objective to make the weights assigned to individual experts correlate strongly with their contributions to the decision at hand. Experimental results on a benchmark dataset demonstrate that the proposed MDA-GC model significantly outperforms the compared methods.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134623116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Incorporating temporal information into recommender systems has recently attracted increasing attention from both the industrial and academic research communities. Existing methods mostly reduce the temporal information of behaviors to behavior sequences for subsequently RNN-based modeling. In such a simple manner, crucial time-related signals have been largely neglected. This paper aims to systematically investigate the effects of the temporal information in sequential recommendations. In particular, we firstly discover two elementary temporal patterns of user behaviors: "absolute time patterns'' and "relative time patterns'', where the former highlights user time-sensitive behaviors, e.g., people may frequently interact with specific products at certain time point, and the latter indicates how time interval influences the relationship between two actions. For seamlessly incorporating these information into a unified model, we devise a neural architecture that jointly learns those temporal patterns to model user dynamic preferences. Extensive experiments on real-world datasets demonstrate the superiority of our model, comparing with the state-of-the-arts.
{"title":"Time Matters: Sequential Recommendation with Complex Temporal Information","authors":"Wenwen Ye, Shuaiqiang Wang, Xu Chen, Xuepeng Wang, Zheng Qin, Dawei Yin","doi":"10.1145/3397271.3401154","DOIUrl":"https://doi.org/10.1145/3397271.3401154","url":null,"abstract":"Incorporating temporal information into recommender systems has recently attracted increasing attention from both the industrial and academic research communities. Existing methods mostly reduce the temporal information of behaviors to behavior sequences for subsequently RNN-based modeling. In such a simple manner, crucial time-related signals have been largely neglected. This paper aims to systematically investigate the effects of the temporal information in sequential recommendations. In particular, we firstly discover two elementary temporal patterns of user behaviors: \"absolute time patterns'' and \"relative time patterns'', where the former highlights user time-sensitive behaviors, e.g., people may frequently interact with specific products at certain time point, and the latter indicates how time interval influences the relationship between two actions. For seamlessly incorporating these information into a unified model, we devise a neural architecture that jointly learns those temporal patterns to model user dynamic preferences. Extensive experiments on real-world datasets demonstrate the superiority of our model, comparing with the state-of-the-arts.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132925081","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. In this paper, we leverage on human intelligence for embedding-based interactive entity linking. We adopt an active learning approach to select mentions for human annotation that can best improve entity linking accuracy at the same time updating the embedding model. We propose two mention selection strategies based on: (1) coherence of entities linked, and (2) contextual closeness of candidate entities with respect to mention. Our experiments show that our proposed interactive entity linking methods outperform their batch counterpart in all our experimented datasets with relatively small amount of human annotations.
{"title":"Interactive Entity Linking Using Entity-Word Representations","authors":"Pei-Chi Lo, Ee-Peng Lim","doi":"10.1145/3397271.3401254","DOIUrl":"https://doi.org/10.1145/3397271.3401254","url":null,"abstract":"To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. In this paper, we leverage on human intelligence for embedding-based interactive entity linking. We adopt an active learning approach to select mentions for human annotation that can best improve entity linking accuracy at the same time updating the embedding model. We propose two mention selection strategies based on: (1) coherence of entities linked, and (2) contextual closeness of candidate entities with respect to mention. Our experiments show that our proposed interactive entity linking methods outperform their batch counterpart in all our experimented datasets with relatively small amount of human annotations.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133133066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xin Dong, Yaxin Zhu, Yupeng Zhang, Zuohui Fu, Dongkuan Xu, Sen Yang, Gerard de Melo
In cross-lingual text classification, one seeks to exploit labeled data from one language to train a text classification model that can then be applied to a completely different language. Recent multilingual representation models have made it much easier to achieve this. Still, there may still be subtle differences between languages that are neglected when doing so. To address this, we present a semi- supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. The resulting model then serves as a teacher to induce labels for unlabeled target lan- guage samples that can be used during further adversarial training, allowing us to gradually adapt our model to the target language. Compared with a number of strong baselines, we observe signifi- cant gains in effectiveness on document and intent classification for a diverse set of languages.
{"title":"Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification","authors":"Xin Dong, Yaxin Zhu, Yupeng Zhang, Zuohui Fu, Dongkuan Xu, Sen Yang, Gerard de Melo","doi":"10.1145/3397271.3401209","DOIUrl":"https://doi.org/10.1145/3397271.3401209","url":null,"abstract":"In cross-lingual text classification, one seeks to exploit labeled data from one language to train a text classification model that can then be applied to a completely different language. Recent multilingual representation models have made it much easier to achieve this. Still, there may still be subtle differences between languages that are neglected when doing so. To address this, we present a semi- supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. The resulting model then serves as a teacher to induce labels for unlabeled target lan- guage samples that can be used during further adversarial training, allowing us to gradually adapt our model to the target language. Compared with a number of strong baselines, we observe signifi- cant gains in effectiveness on document and intent classification for a diverse set of languages.","PeriodicalId":252050,"journal":{"name":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133300204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：