Martin Wattenberg
Harvard University
- Research field
- Interpretability of neural networks, visualization of machine learning
Martin Wattenberg works on ways of seeing what a neural network has learned, from visualizations of embeddings to probes and interventions on a model's internal activations. With Kenneth Li, Aspen Hopkins, David Bau, Fernanda Viégas, and Hanspeter Pfister he trained a small transformer on nothing but sequences of Othello moves and showed that it builds a picture of the board inside its activations: nonlinear probes read the board with an error of 1.7 per cent, and editing the activations until the probe reports one square flipped changes the model's legal-move predictions to those of the edited board. The unit presents this experiment as its worked case of probing and intervention, together with the follow-up paper in which a linear probe succeeds once the board is encoded relative to the player about to move.
In the masterclass
Section 2, after the probing subsection, presents the Othello model, the probe results, and the intervention test.
The research is presented in Unit 16: Interpretability and Alignment, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.