Robots can now learn new tasks by studying humans

Alan Martin May 21, 2018

Making a robot that appears to think and behave like a human is (relatively) straightforward. Okay, that doesn’t make it any the less unnerving when you see one out for a jog, but it is reassuring to know that its technique and form is programmed rather than learned as it overtakes you, breaking into neither a figurative or literal sweat.

Then there are robots that learn by doing: AlphaGo, the robot that learned to beat the world’s number one Go player, did so by playing a lot of Go, for example.

But a robot that learns by watching humans: that’s new. It’s new and a whole different flavour of threatening for those convinced that their employers would rather hire a mechanical version of them, minus the penchant for reddit and sick days.

We’re one step closer to that reality. Nvidia’s researchers have built a deep learning-based system which teaches robots to mimic humans just by watching them perform first. In other words, robots can learn on the job in exactly the same way humans do: monkeybot see, monkeybot do.

Okay, as the video above suggests, it’s pretty limited, and as long as your job doesn’t involve stacking brightly coloured cubes, you’re probably safe for now. But as trivial as that looks from a fleshy human perspective, this kind of imitation is a huge challenge for robots which have to identify objects and how they interact with each other in real time.

Better still, the system also outputs an explanation of the steps it’s taking in plain English, so that if it makes a mistake you’ll be able to see what it thought it was doing. Although that’s probably of little comfort if you’ve just seen your car crushed under a giant yellow block.

“For robots to perform useful tasks in real-world settings, it must be easy to communicate the task to the robot; this includes both the desired result and any hints as to the best means to achieve that result,” the researchers explained. “With demonstrations, a user can communicate a task to the robot and provide clues as to how to best perform the task.”

For now these tasks are limited to toy cars and blocks, but eventually you could be demonstrating the finer points of your job to a mechanical intern. At that point, you’ll have to hope that the human touch is worth your employer paying extra for.