← All research spotlights

Research spotlight · Unit 1

Atılım Güneş Baydin

University of Oxford

Research field
Automatic differentiation, probabilistic programming, machine learning for the sciences

Atılım Güneş Baydin works on how computers compute derivatives of programs and on probabilistic programming, and wrote the survey of automatic differentiation that the field cites when it explains what its learning libraries do. The 2018 survey with Barak Pearlmutter, Alexey Radul, and Jeffrey Siskind, written while he was a researcher at the University of Oxford, defines the forward and the reverse mode, shows that backpropagation is the reverse mode applied to a neural network, and explains why the reverse mode obtains the derivative with respect to every parameter at a small constant multiple of the cost of running the program. He later taught machine learning at Oxford's Department of Computer Science and has since moved to industry. The unit presents the survey after the chain rule, as the mechanical application of that rule to every operation of a program.

In the masterclass

Section 6, after the chain rule, shows how his survey turns the chain rule into the automatic differentiation that every learning library performs.

The research is presented in Unit 1: The AI Researcher's Toolkit, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Baydin, A. G., Pearlmutter, B. A., Radul, A. A., and Siskind, J. M. (2018). Automatic Differentiation in Machine Learning: a Survey. Journal of Machine Learning Research, 18(153), 1‑43.

The scientists named here are not contributors to the AI Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.