← All research spotlights

Research spotlight · Unit 4

Mikhail Belkin

University of California San Diego

Research field
Theory of machine learning, generalization of over-parameterized models

Mikhail Belkin works on the mathematical foundations of machine learning, in particular on why models with far more parameters than training examples can predict well. In a 2019 paper with Daniel Hsu, Siyuan Ma, and Soumik Mandal he showed that the test error of a model, plotted against its size, follows the classical U-shaped curve only up to the point where the model first fits its training data exactly, and then falls again as the model grows further. The unit draws this double descent curve on its own ten-point example and presents the paper's experiments on the MNIST digits.

In the masterclass

Section 7, after the unit's own double descent plot, presents the paper's random Fourier feature experiment on MNIST and its explanation of the second descent.

The research is presented in Unit 4: Learning from Data, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Belkin, M., Hsu, D., Ma, S., and Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116(32), 15849‑15854. https://doi.org/10.1073/pnas.1903070116

The scientists named here are not contributors to the AI Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.