Diversifying Emergent Behaviours with Age-Layered MAP-Elites

Abstract

Emergent behaviour can arise unexpectedly as a by-product of the complex interactions of an autonomous system, and with the increasing desire for such systems, emergent behaviour has become an important area of interest for AI research. One aspect of this research is in searching for a diverse set of emergent behaviours which not only provides a useful tool for finding unwanted emergent behaviour, but also in finding interesting emergent behaviour. The multi-dimensional archive of phenotypic elites (MAP-Elites) algorithm is a popular evolutionary algorithm which returns a highly diverse set of elite solutions at the end of a run. The population is separated into a grid-like feature space defined by a set of behaviour dimensions specified by the user where each cell of the grid corresponds to a unique behaviour combination. The algorithm is conceptually simple and effective at producing high-quality, diverse solutions, but it comes with a major limitation on its exploratory capabilities. With each additional behaviour, the set of solutions grows exponentially, making high-dimensional feature spaces infeasible. This thesis proposes an option for increasing behaviours with a novel Age-Layered MAP-Elites (ALME) algorithm where the population is separated into age layers and each layer has its own feature space. By using different behaviours in the different layers, the population migrates up through the layers experiencing selective pressure towards different behaviours. This algorithm is applied to a simulated intelligent agent environment to observe interesting emergent behaviours. It is observed that ALME is capable of producing a set of solutions with diversity in all behaviour dimensions while keeping the final population size low. It is also observed that ALME is capable of filling its top layer feature space more consistently than MAP-Elites with the same behaviour dimensions.