← All research spotlights

Research spotlight · Unit 12

David Silver

Google DeepMind

Research field
Reinforcement learning, game playing, and search

David Silver led the AlphaGo project and the programs that followed it. In the 2017 preprint and the 2018 paper in Science presented in the unit, he and his colleagues showed that one algorithm, AlphaZero, learns chess, shogi, and Go from self-play alone, starting from random play and knowing nothing but the rules. A single network outputs move probabilities and a position value, a Monte Carlo tree search guided by that network plays the games, and the network is trained on the games the search plays against itself. Within hours of training on thousands of specialized processors it surpassed the champion programs Stockfish and Elmo while examining about a thousand times fewer positions per second than they do. The unit presents the work at the end of the section on search and learning, as the step from AlphaGo, which started from human games, to a method that starts from nothing.

In the masterclass

Section 7, after AlphaGo and the match against Lee Sedol, presents AlphaZero as one algorithm that learns three games from the rules alone.

The research is presented in Unit 12: Reinforcement Learning, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., and Hassabis, D. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140‑1144. https://doi.org/10.1126/science.aar6404

The scientists named here are not contributors to the AI Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.