Kaiming He
Massachusetts Institute of Technology
- Research field
- Deep learning and computer vision, the training of very deep networks
Kaiming He is an associate professor of electrical engineering and computer science at MIT, and before that worked at Facebook AI Research and at Microsoft Research in Beijing, where his group found in 2015 that a plain network of 56 layers trained worse than one of 20 layers even on the training set. The residual block they proposed, a shortcut that carries a block's input around its layers and adds it to the output, removed the problem and allowed networks of 152 layers, whose ensemble won the ImageNet classification challenge of 2015 with a top-5 error of 3.57 per cent. The same paper reports a 1,202-layer network that still trained. The unit presents the result as the architectural cure for the vanishing gradient, after the initialization rule from his 2015 paper on rectifiers.
In the masterclass
Section 4, after the vanishing gradient and the residual block, presents the degradation experiment and the 152-layer networks of the 2016 paper.
The research is presented in Unit 7: Training Deep Networks, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.