Conversational recommender systems offer a way for users to engage in multi-turn conversations to find items they enjoy. For users to trust an agent and give effective feedback, the recommender system must be able to explain its suggestions and rationales. We develop a two-part framework for training multi-turn conversational recommenders that provide recommendation rationales that users can effectively interact with to receive better recommendations. First, we train a recommender system to jointly suggest items and explain its reasoning via subjective rationales. We then fine-tune this model to incorporate iterative user feedback via self-supervised bot-play. Experiments on three real-world datasets demonstrate that our system can be applied to different recommendation models across diverse domains to achieve state-of-the-art performance in multi-turn recommendation. Human studies show that systems trained with our framework provide more useful, helpful, and knowledgeable suggestions in warm- and cold-start settings. Our framework allows us to use only product reviews during training, avoiding the need for expensive dialog transcript datasets that limit the applicability of previous conversational recommender agents.
{"title":"Self-Supervised Bot Play for Transcript-Free Conversational Recommendation with Rationales","authors":"Shuyang Li, Bodhisattwa Prasad Majumder, Julian McAuley","doi":"10.1145/3523227.3546783","DOIUrl":"https://doi.org/10.1145/3523227.3546783","url":null,"abstract":"Conversational recommender systems offer a way for users to engage in multi-turn conversations to find items they enjoy. For users to trust an agent and give effective feedback, the recommender system must be able to explain its suggestions and rationales. We develop a two-part framework for training multi-turn conversational recommenders that provide recommendation rationales that users can effectively interact with to receive better recommendations. First, we train a recommender system to jointly suggest items and explain its reasoning via subjective rationales. We then fine-tune this model to incorporate iterative user feedback via self-supervised bot-play. Experiments on three real-world datasets demonstrate that our system can be applied to different recommendation models across diverse domains to achieve state-of-the-art performance in multi-turn recommendation. Human studies show that systems trained with our framework provide more useful, helpful, and knowledgeable suggestions in warm- and cold-start settings. Our framework allows us to use only product reviews during training, avoiding the need for expensive dialog transcript datasets that limit the applicability of previous conversational recommender agents.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116414532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Shivaram, Ping Liu, Matthew Shapiro, M. Bilgic, A. Culotta
Content-based news recommenders learn words that correlate with user engagement and recommend articles accordingly. This can be problematic for users with diverse political preferences by topic — e.g., users that prefer conservative articles on one topic but liberal articles on another. In such instances, recommenders can have a homogenizing effect by recommending articles with the same political lean on both topics, particularly if both topics share salient, politically polarized terms like “far right” or “radical left.” In this paper, we propose attention-based neural network models to reduce this homogenization effect by increasing attention on words that are topic specific while decreasing attention on polarized, topic-general terms. We find that the proposed approach results in more accurate recommendations for simulated users with such diverse preferences.
{"title":"Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation","authors":"K. Shivaram, Ping Liu, Matthew Shapiro, M. Bilgic, A. Culotta","doi":"10.1145/3523227.3546782","DOIUrl":"https://doi.org/10.1145/3523227.3546782","url":null,"abstract":"Content-based news recommenders learn words that correlate with user engagement and recommend articles accordingly. This can be problematic for users with diverse political preferences by topic — e.g., users that prefer conservative articles on one topic but liberal articles on another. In such instances, recommenders can have a homogenizing effect by recommending articles with the same political lean on both topics, particularly if both topics share salient, politically polarized terms like “far right” or “radical left.” In this paper, we propose attention-based neural network models to reduce this homogenization effect by increasing attention on words that are topic specific while decreasing attention on polarized, topic-general terms. We find that the proposed approach results in more accurate recommendations for simulated users with such diverse preferences.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128357074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Movielens dataset has become a default choice for recommender systems evaluation. In this paper we analyze the best strategies of a Reinforcement Learning agent on Movielens (1M) dataset studying the balance between precision and diversity of recommendations. We found that trivial strategies are able to maximize ranking quality criteria, but useless for users of the recommendation system due to the lack of diversity in final predictions. Our proposed method stimulates the agent to explore the environment using the stochasticity of Ornstein-Uhlenbeck processes. Experiments show that optimization of the Ornstein-Uhlenbeck process drift coefficient improves the diversity of recommendations while maintaining high nDCG and HR criteria. To the best of our knowledge, the analysis of agent strategies in recommendation environments has not been studied excessively in previous works.
{"title":"Multiobjective Evaluation of Reinforcement Learning Based Recommender Systems","authors":"A. Grishanov, A. Ianina, K. Vorontsov","doi":"10.1145/3523227.3551485","DOIUrl":"https://doi.org/10.1145/3523227.3551485","url":null,"abstract":"Movielens dataset has become a default choice for recommender systems evaluation. In this paper we analyze the best strategies of a Reinforcement Learning agent on Movielens (1M) dataset studying the balance between precision and diversity of recommendations. We found that trivial strategies are able to maximize ranking quality criteria, but useless for users of the recommendation system due to the lack of diversity in final predictions. Our proposed method stimulates the agent to explore the environment using the stochasticity of Ornstein-Uhlenbeck processes. Experiments show that optimization of the Ornstein-Uhlenbeck process drift coefficient improves the diversity of recommendations while maintaining high nDCG and HR criteria. To the best of our knowledge, the analysis of agent strategies in recommendation environments has not been studied excessively in previous works.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132689968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We apply a transformer using sequential browse history to generate next-item product recommendations. Interpreting the learned item embeddings, we show that the model is able to implicitly learn price, popularity, style and functionality attributes without being explicitly passed these features during training. Our real-life test of this model on Wayfair’s different international stores show mixed results (but overall win). Diagnosing the cause, we identify a useful metric (average number of customers browsing each product) to ensure good model convergence. We also find limitations of using standard metrics like recall and nDCG, which do not correctly account for the positional effects of showing items on the Wayfair website, and empirically determine a more accurate discount factor.
{"title":"A Lightweight Transformer for Next-Item Product Recommendation","authors":"M. J. Mei, Cole Zuber, Y. Khazaeni","doi":"10.1145/3523227.3547491","DOIUrl":"https://doi.org/10.1145/3523227.3547491","url":null,"abstract":"We apply a transformer using sequential browse history to generate next-item product recommendations. Interpreting the learned item embeddings, we show that the model is able to implicitly learn price, popularity, style and functionality attributes without being explicitly passed these features during training. Our real-life test of this model on Wayfair’s different international stores show mixed results (but overall win). Diagnosing the cause, we identify a useful metric (average number of customers browsing each product) to ensure good model convergence. We also find limitations of using standard metrics like recall and nDCG, which do not correctly account for the positional effects of showing items on the Wayfair website, and empirically determine a more accurate discount factor.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133056979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Giacomo Balloccu, Ludovico Boratto, G. Fenu, M. Marras
The goal of this tutorial is to present the RecSys community with recent advances on explainable recommender systems with knowledge graphs. We will first introduce conceptual foundations, by surveying the state of the art and describing real-world examples of how knowledge graphs are being integrated into the recommendation pipeline, also for the purpose of providing explanations. This tutorial will continue with a systematic presentation of algorithmic solutions to model, integrate, train, and assess a recommender system with knowledge graphs, with particular attention to the explainability perspective. A practical part will then provide attendees with concrete implementations of recommender systems with knowledge graphs, leveraging open-source tools and public datasets; in this part, tutorial participants will be engaged in the design of explanations accompanying the recommendations and in articulating their impact. We conclude the tutorial by analyzing emerging open issues and future directions. Website: https://explainablerecsys.github.io/recsys2022/.
{"title":"Hands on Explainable Recommender Systems with Knowledge Graphs","authors":"Giacomo Balloccu, Ludovico Boratto, G. Fenu, M. Marras","doi":"10.1145/3523227.3547374","DOIUrl":"https://doi.org/10.1145/3523227.3547374","url":null,"abstract":"The goal of this tutorial is to present the RecSys community with recent advances on explainable recommender systems with knowledge graphs. We will first introduce conceptual foundations, by surveying the state of the art and describing real-world examples of how knowledge graphs are being integrated into the recommendation pipeline, also for the purpose of providing explanations. This tutorial will continue with a systematic presentation of algorithmic solutions to model, integrate, train, and assess a recommender system with knowledge graphs, with particular attention to the explainability perspective. A practical part will then provide attendees with concrete implementations of recommender systems with knowledge graphs, leveraging open-source tools and public datasets; in this part, tutorial participants will be engaged in the design of explanations accompanying the recommendations and in articulating their impact. We conclude the tutorial by analyzing emerging open issues and future directions. Website: https://explainablerecsys.github.io/recsys2022/.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114684640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, many methods for machine learning on tabular data were introduced that use either factorization machines, neural networks or both. This created a great variety of methods making it non-obvious which method should be used in practice. We begin by extending the previously established theoretical connection between polynomial neural networks and factorization machines (FM) to recently introduced FM techniques. This allows us to propose a single neural-network-based framework that can switch between the deep learning and FM paradigms by a simple change of an activation function. We further show that an activation function exists which can adaptively learn to select the optimal paradigm. Another key element in our framework is its ability to learn high-dimensional embeddings by low-rank factorization. Our framework can handle numeric and categorical data as well as multiclass outputs. Extensive empirical experiments verify our analytical claims. Source code is available at https://github.com/ChenAlmagor/FiFa
{"title":"You Say Factorization Machine, I Say Neural Network - It’s All in the Activation","authors":"Chen Almagor, Yedid Hoshen","doi":"10.1145/3523227.3551499","DOIUrl":"https://doi.org/10.1145/3523227.3551499","url":null,"abstract":"In recent years, many methods for machine learning on tabular data were introduced that use either factorization machines, neural networks or both. This created a great variety of methods making it non-obvious which method should be used in practice. We begin by extending the previously established theoretical connection between polynomial neural networks and factorization machines (FM) to recently introduced FM techniques. This allows us to propose a single neural-network-based framework that can switch between the deep learning and FM paradigms by a simple change of an activation function. We further show that an activation function exists which can adaptively learn to select the optimal paradigm. Another key element in our framework is its ability to learn high-dimensional embeddings by low-rank factorization. Our framework can handle numeric and categorical data as well as multiclass outputs. Extensive empirical experiments verify our analytical claims. Source code is available at https://github.com/ChenAlmagor/FiFa","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115387742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Karl Higley, Even Oldridge, Ronay Ak, Sara Rabhi, G. Moreira
Newcomers to recommender systems often face challenges related to their lack of understanding of how these systems operate in real life. In most online content related to this topic, the focus is on models and algorithms that score items based on the user’s preferences. However, the recommender model alone does not comprise everything needed for serving optimized recommender systems that meet the company’s business objectives. An industry-standard recommender system involves a number of steps, including data preprocessing, defining and training recommender models, as well as filtering and business logic for serving. In this work, we propose the four-stage recommender system, an industry-wide design pattern we have identified for production recommender systems. The four-stage pipeline includes an item retrieval step that prepares a small subset of relevant items for scoring. The filtering stage then cleans up the subset of items based on business logic such as removing out-of-stock or previously seen items. As for the ranking component, it uses a recommender model to score each item in the presented list based on the preferences of the user. In the final step, the scores are re-ordered to provide a final recommendation list aligned with other business needs or constraints such as diversity. In particular, the presented demo demonstrates how easy it is to build and deploy a four-stage recommender system pipeline using the NVIDIA Merlin open-source framework.
{"title":"Building and Deploying a Multi-Stage Recommender System with Merlin","authors":"Karl Higley, Even Oldridge, Ronay Ak, Sara Rabhi, G. Moreira","doi":"10.1145/3523227.3551468","DOIUrl":"https://doi.org/10.1145/3523227.3551468","url":null,"abstract":"Newcomers to recommender systems often face challenges related to their lack of understanding of how these systems operate in real life. In most online content related to this topic, the focus is on models and algorithms that score items based on the user’s preferences. However, the recommender model alone does not comprise everything needed for serving optimized recommender systems that meet the company’s business objectives. An industry-standard recommender system involves a number of steps, including data preprocessing, defining and training recommender models, as well as filtering and business logic for serving. In this work, we propose the four-stage recommender system, an industry-wide design pattern we have identified for production recommender systems. The four-stage pipeline includes an item retrieval step that prepares a small subset of relevant items for scoring. The filtering stage then cleans up the subset of items based on business logic such as removing out-of-stock or previously seen items. As for the ranking component, it uses a recommender model to score each item in the presented list based on the preferences of the user. In the final step, the scores are re-ordered to provide a final recommendation list aligned with other business needs or constraints such as diversity. In particular, the presented demo demonstrates how easy it is to build and deploy a four-stage recommender system pipeline using the NVIDIA Merlin open-source framework.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115279616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ziyang Tang, Yiheng Duan, Steven H. Zhu, Stephanie S. Zhang, Lihong Li
A/B testing is a powerful tool for a company to make informed decisions about their services and products. A limitation of A/B tests is that they do not easily extend to measure post-experiment (long-term) differences. In this talk, we study a different approach inspired by recent advances in off-policy evaluation in reinforcement learning (RL). The basic RL approach assumes customer behavior follows a stationary Markovian process, and estimates the average engagement metric when the process reaches the steady state. However, in realistic scenarios, the stationary assumption is often violated due to weekly variations and seasonality effects. To tackle this challenge, we propose a variation by relaxing the stationary assumption. We empirically tested both stationary and nonstationary approaches in a synthetic dataset and an online store dataset.
{"title":"Estimating Long-term Effects from Experimental Data","authors":"Ziyang Tang, Yiheng Duan, Steven H. Zhu, Stephanie S. Zhang, Lihong Li","doi":"10.1145/3523227.3547398","DOIUrl":"https://doi.org/10.1145/3523227.3547398","url":null,"abstract":"A/B testing is a powerful tool for a company to make informed decisions about their services and products. A limitation of A/B tests is that they do not easily extend to measure post-experiment (long-term) differences. In this talk, we study a different approach inspired by recent advances in off-policy evaluation in reinforcement learning (RL). The basic RL approach assumes customer behavior follows a stationary Markovian process, and estimates the average engagement metric when the process reaches the steady state. However, in realistic scenarios, the stationary assumption is often violated due to weekly variations and seasonality effects. To tackle this challenge, we propose a variation by relaxing the stationary assumption. We empirically tested both stationary and nonstationary approaches in a synthetic dataset and an online store dataset.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122655347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Weixin Chen, Mingkai He, Yongxin Ni, Weike Pan, L. Chen, Zhong Ming
Heterogeneous sequential recommendation (HSR) is a very important recommendation problem, which aims to predict a user’s next interacted item under a target behavior type (e.g., purchase in e-commerce sites) based on his/her historical interactions with different behaviors. Though existing sequential methods have achieved advanced performance by considering the varied impacts of interactions with sequential information, a large body of them still have two major shortcomings. Firstly, they usually model different behaviors separately without considering the correlations between them. The transitions from item to item under diverse behaviors indicate some users’ potential behavior manner. Secondly, though the behavior information contains a user’s fine-grained interests, the insufficient consideration of the local context information limits them from well understanding user intentions. Utilizing the adjacent interactions to better understand a user’s behavior could improve the certainty of prediction. To address these two issues, we propose a novel solution utilizing global and personalized graphs for HSR (GPG4HSR) to learn behavior transitions and user intentions. Specifically, our GPG4HSR consists of two graphs, i.e., a global graph to capture the transitions between different behaviors, and a personalized graph to model items with behaviors by further considering the distinct user intentions of the adjacent contextually relevant nodes. Extensive experiments on four public datasets with the state-of-the-art baselines demonstrate the effectiveness and general applicability of our method GPG4HSR.
{"title":"Global and Personalized Graphs for Heterogeneous Sequential Recommendation by Learning Behavior Transitions and User Intentions","authors":"Weixin Chen, Mingkai He, Yongxin Ni, Weike Pan, L. Chen, Zhong Ming","doi":"10.1145/3523227.3546761","DOIUrl":"https://doi.org/10.1145/3523227.3546761","url":null,"abstract":"Heterogeneous sequential recommendation (HSR) is a very important recommendation problem, which aims to predict a user’s next interacted item under a target behavior type (e.g., purchase in e-commerce sites) based on his/her historical interactions with different behaviors. Though existing sequential methods have achieved advanced performance by considering the varied impacts of interactions with sequential information, a large body of them still have two major shortcomings. Firstly, they usually model different behaviors separately without considering the correlations between them. The transitions from item to item under diverse behaviors indicate some users’ potential behavior manner. Secondly, though the behavior information contains a user’s fine-grained interests, the insufficient consideration of the local context information limits them from well understanding user intentions. Utilizing the adjacent interactions to better understand a user’s behavior could improve the certainty of prediction. To address these two issues, we propose a novel solution utilizing global and personalized graphs for HSR (GPG4HSR) to learn behavior transitions and user intentions. Specifically, our GPG4HSR consists of two graphs, i.e., a global graph to capture the transitions between different behaviors, and a personalized graph to model items with behaviors by further considering the distinct user intentions of the adjacent contextually relevant nodes. Extensive experiments on four public datasets with the state-of-the-art baselines demonstrate the effectiveness and general applicability of our method GPG4HSR.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121157522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Reinforcement learning (RL) is gaining traction as a complementary approach to supervised learning for RecSys due to its ability to solve sequential decision-making processes for delayed rewards. Recent advances in offline reinforcement learning, off-policy evaluation, and more scalable, performant system design with the ability to run code in parallel, have made RL more tractable for the RecSys real time use cases. This tutorial introduces RLlib [9], a comprehensive open-source Python RL framework built for production workloads. RLlib is built on top of open-source Ray [8], an easy-to-use, distributed computing framework for Python that can handle complex, heterogeneous applications. Ray and RLlib run on compute clusters on any cloud without vendor lock. Using Colab notebooks, you will leave this tutorial with a complete, working example of parallelized Python RL code using RLlib for RecSys on a github repo.
由于强化学习(RL)能够解决延迟奖励的顺序决策过程，因此它作为监督学习的补充方法在RecSys中获得了越来越多的关注。最近在离线强化学习、离线策略评估、更可扩展、性能更好的系统设计以及并行运行代码的能力方面取得的进展，使得强化学习在RecSys实时用例中更容易处理。本教程介绍了RLlib[9]，一个为生产工作负载构建的全面的开源Python RL框架。RLlib建立在开源的Ray[8]之上，Ray是一个易于使用的Python分布式计算框架，可以处理复杂的异构应用程序。Ray和RLlib可以在没有供应商锁定的任何云上的计算集群上运行。使用Colab笔记本，您将在本教程中留下一个完整的，使用RLlib for RecSys在github repo上并行化Python RL代码的工作示例。
{"title":"Hands-on Reinforcement Learning for Recommender Systems - From Bandits to SlateQ to Offline RL with Ray RLlib","authors":"Christy D. Bergman, Kourosh Hakhamaneshi","doi":"10.1145/3523227.3547370","DOIUrl":"https://doi.org/10.1145/3523227.3547370","url":null,"abstract":"Reinforcement learning (RL) is gaining traction as a complementary approach to supervised learning for RecSys due to its ability to solve sequential decision-making processes for delayed rewards. Recent advances in offline reinforcement learning, off-policy evaluation, and more scalable, performant system design with the ability to run code in parallel, have made RL more tractable for the RecSys real time use cases. This tutorial introduces RLlib [9], a comprehensive open-source Python RL framework built for production workloads. RLlib is built on top of open-source Ray [8], an easy-to-use, distributed computing framework for Python that can handle complex, heterogeneous applications. Ray and RLlib run on compute clusters on any cloud without vendor lock. Using Colab notebooks, you will leave this tutorial with a complete, working example of parallelized Python RL code using RLlib for RecSys on a github repo.","PeriodicalId":443279,"journal":{"name":"Proceedings of the 16th ACM Conference on Recommender Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125151247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：