← All research spotlights

Research spotlight · Unit 7

Kaiming He

Massachusetts Institute of Technology

Research field
Deep learning and computer vision, the training of very deep networks

Kaiming He is an associate professor of electrical engineering and computer science at MIT, and before that worked at Facebook AI Research and at Microsoft Research in Beijing, where his group found in 2015 that a plain network of 56 layers trained worse than one of 20 layers even on the training set. The residual block they proposed, a shortcut that carries a block's input around its layers and adds it to the output, removed the problem and allowed networks of 152 layers, whose ensemble won the ImageNet classification challenge of 2015 with a top-5 error of 3.57 per cent. The same paper reports a 1,202-layer network that still trained. The unit presents the result as the architectural cure for the vanishing gradient, after the initialization rule from his 2015 paper on rectifiers.

In the masterclass

Section 4, after the vanishing gradient and the residual block, presents the degradation experiment and the 152-layer networks of the 2016 paper.

The research is presented in Unit 7: Training Deep Networks, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770‑778. https://doi.org/10.1109/CVPR.2016.90

The scientists named here are not contributors to the AI Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.