This DeepMind robot is a resilient football player
In a video from DeepMind's scientists, a little robot football player was knocked down from different directions but sprang up immediately.
Courtesy of ROBOTIS
When I saw the video of this little robot getting repeatedly knocked over but springing up immediately, I almost felt bad for it, as if the lady was bullying the robot football player. But this further proves how resilient and agile this humanoid robot is—it will never get defeated on the football court and will keep getting up when it falls.
This OP3 robot is trained by DeepMind, an AI research lab that serves as Google’s subsidiary. Scientists from the lab used Deep Reinforcement Learning (Deep RL) to train the robot. Deep RL is one of the most trending types in machine learning. It can solve a wide range of complex decision-making tasks that were previously out of reach for a machine to solve real-world problems with human-like intelligence.
The DeepMind scientists wanted to see if a small, cheap robot (which still costs $11,000) could learn to do complex movements and play football through Deep RL. So they first trained it in a computer simulation, teaching it individual skills like walking, turning, kicking, and more. Then, they let the robot play against itself in the simulation to practice and combine those skills. Surprisingly, the robot learned to do things beyond their expectations and even developed some strategies for the game.
It still performed well when bringing the robot to real-life situations without any further training. With some small improvements to the robot’s hardware and further training on safety and agility, the robot could walk faster, get up more quickly, and kick the ball faster than a pre-programmed robot. It could also combine its skills effectively to reach its goals in the game.
The research was inspired by RoboCup, the annual international robotics competition. But the experimented environment and tasks were substantially simpler than the challenges that appeared in RoboCup. While the limitation of the research was also obvious: no real data was used for transfer, and the robot was really small. But the team believes this could be applied to more complex settings and larger robots.
However, this study paves the way for future robot football players. Robots could potentially play the game solely relying on their own sensors instead of getting pre-programmed. Imagine combining this with computer vision, which gives the robots the ability to “see,” there will be a great leap forward in the development of robotics. The scientists have already successfully trained vision-based robots using onboard RGB cameras and proprioception.
