Inferring Intents From Equivariant–Invariant Representations and Relational Learning in Multiagent Systems

Abstract

Accurately understanding intentions is crucial in various real-world multiagent scenarios, which helps comprehend motives and predict actions within these contexts. Existing methods tend to either concentrate too much on single agents' isolated characteristics or model complex interactions among multiple agents, failing to adequately address both aspects simultaneously. To address this challenge, we propose a novel framework called integrative multiagent behavior prediction framework to systematically incorporate individual features and interagent relational dynamics. Our approach not only models multiagent interactions by learning from visual data, but also integrates mining of imagery and videos to leverage intrinsic invariant and variant qualities of each agent's extrinsic morphology. Meanwhile, inspired by time -series forecasting, we represent interagent history and connections as seasonal and trend features in time-series patterns, capturing past behavioral influences that are often ignored. We also design an encoder that efficiently learns time-dependencies and concatenates individual invariant–variant feature learning modules with multiagent interaction representations to accurately infer intentions and trajectory predictions. Our approach not only models multiagent interactions by learning from visual data, but also integrates mining of imagery and videos to leverage intrinsic invariant qualities of each agent's extrinsic morphology (e.g., body shape, color) and variant qualities (e.g., pose, expression, attire). Extensive experiments demonstrate that, compared to current state-of-the-art intention analysis models, our framework improves behavioral prediction performance in multiagent environments.