Acknowledgment
Thanks to contributors and collaborators:
- Prof. Micheal Guerzhoy — Pong environment
- Jet Chiang — collaboration and DQN baseline
- Eric Xie, Paul Dong — Smash-style experiments
- UTMIST — platform support
NEAT evolves network structure and weights via genetic algorithms, starting from minimal networks and progressively complexifying them. Speciation helps preserve innovation by protecting new structures.
Objective (intuition)
A common framing for policy learning is likelihood maximization:
In NEAT, we search over network parameters and topology using evolutionary operators (selection, crossover, mutation) rather than gradient descent.
Pong experiment
Collaboration with Jet Chiang — see his post. Built with neat-python.
Demo of a self-play Pong agent trained after ~5 minutes.
How NEAT works
Speciation, crossover, and mutation are core components. A standard compatibility distance used for speciation is:
where $$E$$ is excess genes, $$D$$ is disjoint genes, and $$W$$ is average weight difference for matching genes.
Smash-style evolution
We also explored combining NEAT variants with sequence models for a Smash-style environment on UTMIST’s AI^2 platform.