Mikhail Belkin
University of California San Diego
- Research field
- Theory of machine learning, generalization of over-parameterized models
Mikhail Belkin works on the mathematical foundations of machine learning, in particular on why models with far more parameters than training examples can predict well. In a 2019 paper with Daniel Hsu, Siyuan Ma, and Soumik Mandal he showed that the test error of a model, plotted against its size, follows the classical U-shaped curve only up to the point where the model first fits its training data exactly, and then falls again as the model grows further. The unit draws this double descent curve on its own ten-point example and presents the paper's experiments on the MNIST digits.
In the masterclass
Section 7, after the unit's own double descent plot, presents the paper's random Fourier feature experiment on MNIST and its explanation of the second descent.
The research is presented in Unit 4: Learning from Data, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.