David B. D'Ambrosio and Kenneth O. Stanley (2008)
Generative Encoding for Multiagent Learning
In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008). New York, NY: ACM, 2008 (8 pages)
Winner of the Best Paper Award in Generative and Developmental Systems at GECCO-2008
Note: This paper is accompanied with a set of videos at http://eplex.cs.ucf.edu/multiagenthyperneat
Abstract
This paper argues that multiagent learning is a potential "killer application" for generative and developmental systems (GDS) because key challenges in learning to coordinate a team of agents are naturally addressed through indirect encodings and information reuse. For example, a significant problem for multiagent learning is that policies learned separately for different agent roles may nevertheless need to share a basic skill set, forcing the learning algorithm to reinvent the wheel for each agent. GDS is a good match for this kind of problem because it specializes in ways to encode patterns of related yet varying motifs. In this paper, to establish the promise of this capability, the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) generative approach to evolving neurocontrollers learns a set of coordinated policies encoded by a single genome representing a team of predator agents that work together to capture prey. Experimental results show that it is not only possible, but beneficial to encode a heterogeneous team of agents with an indirect encoding. The main contribution is thus to open up a significant new application domain for GDS.