Atılım Güneş Baydin
University of Oxford
- Research field
- Automatic differentiation, probabilistic programming, machine learning for the sciences
Atılım Güneş Baydin works on how computers compute derivatives of programs and on probabilistic programming, and wrote the survey of automatic differentiation that the field cites when it explains what its learning libraries do. The 2018 survey with Barak Pearlmutter, Alexey Radul, and Jeffrey Siskind, written while he was a researcher at the University of Oxford, defines the forward and the reverse mode, shows that backpropagation is the reverse mode applied to a neural network, and explains why the reverse mode obtains the derivative with respect to every parameter at a small constant multiple of the cost of running the program. He later taught machine learning at Oxford's Department of Computer Science and has since moved to industry. The unit presents the survey after the chain rule, as the mechanical application of that rule to every operation of a program.
In the masterclass
Section 6, after the chain rule, shows how his survey turns the chain rule into the automatic differentiation that every learning library performs.
The research is presented in Unit 1: The AI Researcher's Toolkit, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.