CoreNet: Leveraging context-aware representations via MLP networks for CTR prediction

Abstract

Click-through rate (CTR) prediction is pivotal for industrial recommendation systems and has been driving intensive research. Recent studies emphasized the effectiveness of adaptive methods that use context-aware representations to enhance predictions by dynamically adjusting feature representations across instances and overcoming fixed embedding limitations. The typical architecture for learning context-aware representations involves a network block built on Multi-Head Self-Attention (MHA) or Multi-Layer Perceptron (MLP). Despite promising results, three main challenges arise from these methods. First, relying on a single network block limits the learning potential of the model by providing only one perspective on the interactions. Second, implementing the MHA mechanism requires multiple attention layers for its effectiveness, thereby increasing the complexity of the model. Third, using only a vanilla MLP makes it difficult to combine implicit and explicit feature interactions, which is crucial for successful CTR solutions. To address these issues, we propose a novel model called Context-Aware Net (CoreNet). CoreNet incorporates an advanced module, Context-Aware Module (CAM), which employs a combination of MLP and Hadamard products to generate comprehensive context-aware representations. The CAM component integrates a two-stream network with first-order and second-order aware streams, extracting insights from different perspectives to complement each other and enhance overall performance. Extensive experiments on four public datasets consistently demonstrate that CoreNet outperforms other state-of-the-art models. Notably, our CAM component is lightweight and model-agnostic, facilitating seamless integration into streaming CTR models to enhance performance in a plug-and-play manner¹.