This page is for those seeking information on the use and implementation of the HyperNEAT neuroevolution method, an extension of the NEAT method. The information herein aims to address common questions about HyperNEAT and to provide knowledge to those who wish to apply or extend the method.
If you haven't heard of HyperNEAT, it is a neuroevolution method, which means it evolves artificial neural networks through an evolutionary algorithm. It is extended from a prior neuroevolution algorithm called NeuroEvolution of Augmenting Topologies (NEAT), which also has its own NEAT Users Page. The HyperNEAT publications (link at left) offer a complete introduction to the method and its underlying theory of representation. This section briefly explains the general idea behind it.
In short, HyperNEAT is based on a theory of representation that hypothesizes that a good representation for an artificial neural network should be able to describe its pattern of connectivity compactly. This kind of description is called an encoding. The encoding in HyperNEAT, called compositional pattern producing networks, is designed to represent patterns with regularities such as symmetry, repetition, and repetition with variation. (Click here for an example of CPPN-generated patterns.) Thus HyperNEAT is able to evolve neural networks with these properties. The main implication of this capability is that HyperNEAT can efficiently evolve very large neural networks that look more like neural connectivity patterns in the brain (which are repetitious with many regularities, in addition to some irregularities) that are generally much larger than what prior approaches to neural learning could produce.
The other unique and important facet of HyperNEAT is that it actually sees the geometry of the problem domain. It is strange to consider, but most neuroevolution algorithms (and most neural learning algorithms in general) are completely blind to domain geometry. For example, when a checkers board position is input into an artificial neural network, it has no idea which piece is next to which piece. If it ever comes to understand the board geometry, it must figure it out for itself. In contrast, when humans play checkers, we know right away the geometry of the board; we do not have to infer it from hundreds of examples of gameplay. HyperNEAT has the same capability. It actually sees the geometry of its inputs (and outputs) and can exploit that geometry to significantly enhance learning. To put it more technically, HyperNEAT computes the connectivity of its neural networks as a function of their geometry.
One implication of HyperNEAT's ability to exploit geometry is that it gives the user a completely new kind of influence over neural network learning. The user can now describe the geometry of the domain to HyperNEAT, which means there is room to be creative. If someone believes that a domain can be described best in a different geometry, it can be tested with HyperNEAT. Thus HyperNEAT opens up a new kind of research direction for artificial neural networks. This geometric layout is called a substrate, which is depicted in the images above.
Thus one way to express what HyperNEAT does is to say it evolves the connectivity pattern for a neural network with a particular substrate geometry.
Presently, two versions of HyperNEAT are available for different platforms and languages. If you are interested in sharing your own version, we are happy to link to it from here. Please email kstanley@eecs.ucf.edu with information on your implementation. Note that HyperNEAT is possible to build from an existing package of NEAT, so you may want to start with an existing NEAT package instead of HyperNEAT.
The remainder of this section is designed to help the user choose the right HyperNEAT package.
The current crop of HyperNEAT software includes C# and C++ versions: The question for many people first coming to HyperNEAT is which package is right for me? Both David D'Ambrosio's extension of SharpNEAT, HyperSharpNEAT or Jason Gauci's HyperNEAT C++ are good sources. Both are cross-platform.The experiments included in these two packages are different. HyperSharpNEAT provides a multi-agent predator-prey experiment, while HyperNEAT C++ includes a visual discrimination task and a checkers experiment. If you are planning a new experiment, it may be helpful to look at the code for similar experiments.
Your best option will be based on some combination of the above considerations. Of course, if you want HyperNEAT for platform X or language Y and such an implementation is not available, you may want to write your own version of HyperNEAT.
HyperNEAT extends the NEAT method. (NEAT evolves the CPPNs that generate networks in HyperNEAT). Much information is available on NEAT and many implementations are supported. NEAT stands for NeuroEvolution of Augmenting Topologies. It is a method for evolving artificial neural networks with an evolutionary algorithm. NEAT implements the idea that it is most effective to start evolution with small, simple networks and allow them to become increasingly complex over generations. That way, just as organisms in nature increased in complexity since the first cell, so do neural networks in NEAT. This process of continual elaboration allows finding highly sophisticated and complex neural networks.
For more information about NEAT and NEAT software, please visit the NEAT Users Page.
Derek James created a NEAT Users Group on Yahoo! to encourage the discussion of ideas, questions, and variations of NEAT. The community of HyperNEAT users and those interested in HyperNEAT can benefit greatly from the availability of this forum. Please feel free to join the discussion!
What HyperNEAT software and source is available?
HyperNEAT was originally invented at the Evolutionary Complexity Research Group (EPlex) at the University of Central Florida. HyperNEAT-related publications from this group are listed below.
Significant research on HyperNEAT is ongoing at a number of research groups around the world:
Computational Intelligence Group at Czech Technical University in Prague
Drchal J., Koutník J., Šnorek M., HyperNEAT Controlled Robots Learn to Drive on Roads in Simulated Environment, to appear in CEC 2009 (pdf)
Digital Evolution Lab at Michigan State University
Clune J, Beckmann BE, Pennock RT, and Ofria C.
HybrID: A Hybridization of Indirect and Direct Encodings for Evolutionary Computation.
Proceedings of the European Conference on Artificial Life (ECAL), 2009. Budapest, Hungary. (pdf)
Clune J, Pennock RT, and Ofria C.
The sensitivity of HyperNEAT to different geometric representations of a
problem.
Proceedings of the Genetic and Evolutionary Computation Conference
(GECCO), 2009. Montreal, Canada. (pdf)
Clune J, Beckmann BE, Ofria C, and Pennock RT.
Evolving coordinated gaits with the HyperNEAT generative encoding.
Proceedings of the IEEE Congress on Evolutionary Computing Special
Section on Evolutionary Robotics, 2009. Trondheim, Norway. (pdf)
Clune J, Ofria C, and Pennock RT.
How a generative encoding fares as problem-regularity decreases.
Proceedings of the 10th International Conference on Parallel Problem
Solving From Nature. pp 358-367. Dortmund, Germany. 2008. (pdf)
Institute of Computing Science at Poznan University of Technology
NeuroHunter, a HyperNEAT neural net evolved by this team won the GECCO'2008 Balanced Diet contest.
Neural Networks Research Group at University of Texas at Austin
Erkin Bahceci and Risto Miikkulainen, Transfer of Evolved Pattern-Based Heuristics in Games, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (CIG), 2008.