← All research spotlights

Research spotlight · Unit 12

Pieter Abbeel

University of California, Berkeley

Research field
Robot learning, reinforcement learning, and imitation learning

Pieter Abbeel works on robots that learn their skills instead of being programmed with them. In a 2016 paper with Sergey Levine, Chelsea Finn, and Trevor Darrell he showed that a single convolutional network with about 92,000 parameters can map the image of one camera directly to the motor torques of a PR2 robot, and that training the perception and the control together works better than training them apart. The network was trained with guided policy search, which first learns simple controllers with full knowledge of the scene and then teaches the network to reproduce them from the camera alone, so that the robot learned to hang a coat hanger, insert a block into a shape-sorting cube, fit a hammer claw under a nail, and screw a cap onto a bottle. The unit presents the work after the policy gradient section, as the answer to how a robot can learn from reward with a number of trials a physical machine can afford.

In the masterclass

Section 6, after the policy gradient and the actor-critic idea, presents his group's robot that learns from camera pixels to motor torques in one network.

The research is presented in Unit 12: Reinforcement Learning, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Levine, S., Finn, C., Darrell, T., and Abbeel, P. (2016). End-to-End Training of Deep Visuomotor Policies. Journal of Machine Learning Research, 17(39), 1‑40.

The scientists named here are not contributors to the AI Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.