Robot Chef Recreates Recipes by Watching Videos

Researchers have created a robotic chef that can recreate recipes by watching and replicating cooking videos.

The team from the University of Cambridge initially programmed the robot with eight salad recipes. After watching one video of someone preparing the recipes, the robot was able to identify the different dishes and recreate them.

After tests of these recipes, the robot was also able to create its own recipe to add to its “cookbook.”

A neural network system was used to help the robot identify ingredients and follow video demonstrations. Using computer vision, the robot analyzes each frame of the video to identify and mirror movements.

The team said its design creates a new way of training robots to cook, without the time-consuming act of programming it to make a wide variety of dishes, creating an “easier and cheaper deployment of robot chefs.”

“We wanted to see whether we could train a robot chef to learn in the same incremental way that humans can – by identifying the ingredients and how they go together in the dish,” said Grzegorz Sochacki, the study’s first author.

The robot was shown a total of 16 videos and it correctly identified the recipe 93% of the time, according to the team, which also noted it correctly detected only 83% of the chef’s actions.

“It’s amazing how much nuance the robot was able to detect,” said Sochacki. “These recipes aren’t complex – they’re essentially chopped fruits and vegetables, but it was really effective at recognizing, for example, that two chopped apples and two chopped carrots is the same recipe as three chopped apples and three chopped carrots.

“As these robot chefs get better and faster at identifying ingredients in food videos, they might be able to use sites like YouTube to learn a whole range of recipes."