Integration of memory systems supporting non-symbolic representations in an architecture for lifelong development of artificial agents

Compared to autonomous agent learning, lifelong agent learning tackles the additional challenge of accumulating skills in a way favourable to long term development. What an agent learns at a given moment can be an element for the future creation of behaviours of greater complexity, whose purpose cannot be anticipated.

Beyond its initial low-level sensorimotor development phase, the agent is expected to acquire, in the same manner as skills, values and goals which support the development of complex behaviours beyond the reactive level. To do so, it must have a way to represent and memorize such information.

In this article, we identify the properties suitable for a representation system supporting the lifelong development of agents through a review of a wide range of memory systems and related literature. Following this analysis, our second contribution is the proposition and implementation of such a representation system in MIND, a modular architecture for lifelong development. The new variable module acts as a simple memory system which is strongly integrated to the hierarchies of skill modules of MIND, and allows for the progressive structuration of behaviour around persistent non-symbolic representations. Variable modules have many applications for the development and structuration of complex behaviours, but also offer designers and operators explicit models of values and goals facilitating human interaction, control and explainability.

We show through experiments two possible uses of variable modules. In the first experiment, skills exchange information by using a variable representing the concept of “target”, which allows the generalization of navigation behaviours. In the second experiment, we show how a non-symbolic representation can be learned and memorized to develop beyond simple reactive behaviour, and keep track of the steps of a process whose state cannot be inferred by observing the environment.