A multi-agent reinforcement learning framework for cross-domain sequential recommendation

Abstract

Sequential recommendation models aim to predict the next item based on the sequence of items users interact with, ordered chronologically. However, these models face the challenge of data sparsity. Recent studies have explored cross-domain sequential recommendation, where users’ interaction data across multiple source domains are leveraged to enhance recommendations in data-sparse target domains. Despite this, users’ interests in the target and source domains may not align perfectly. Additionally, current research often neglects the collaboration between different transfer strategies across source domains, leading to suboptimal performance. To address these challenges, we propose a multi-agent reinforcement learning framework for cross-domain sequential recommendation (MARL4CDSR). Unlike traditional approaches that transfer knowledge from the entire source domain sequence, MARL4CDSR uses agents to select relevant items from source domain sequences for transfer. This approach optimizes the transfer process by coordinating agents’ strategies within each source domain through a multi-agent reinforcement learning framework. Additionally, we introduce an information fusion module with a cross-attention mechanism to align the embedding representations of selected source domain items with target domain items. A reward function based on score differences for the next item optimizes the multi-agent system. We evaluate the method on three Amazon domains: Movies_and_TV, Toys_and_Games, and Books. Our proposed model MARL4CDSR outperforms all baselines on all metrics. Specifically, for the Movies&Books$\to$Toys task, where the target domain interaction sequence is relatively sparse, MARL4CDSR improves NDCG@10 and HR@10 by 14.76% and 10.25%, respectively.