Nothing sparks excitement quite like the sight of mini-adorable robots. Orange and Green are two pint-sized marvels created by Disney Research and which learnt to walk within Nvidia’s simulator.
They padded up to Nvidia chief executive Jensen Huang, during his keynote speech at its GTC developer event at the SAP Centre in San Jose last week.
While they captivated the audience of over 11,000 engineers and software developers, the true focus of the robots was Project Gr00t.
Unveiled by Huang, it is a new platform which goes beyond basic programming, enabling robots to understand natural language and mimic human movements.
A model for humanoid robots, it opens doors for robots to be trained more efficiently and intuitively With it, they can learn by observation and interaction rather than relying solely on complex code.
Project Gr00t will help roboticists build better performing robots that can quickly acquire coordination, dexterity, and other vital skills for navigating and adapting to real-world interactions.
This addresses a key challenge in robotics: the time and resources required to train robots for real-world applications.
By enabling robots to learn through natural language and observation, Project Gr00t has the potential to streamline training processes and accelerate robot deployments across various industries.
The new platform positions Nvidia as a key player in the robotics ecosystem, providing the foundational technology for developing sophisticated AI-powered robots.
However, Project Gr00t isn’t the only game-changer. Toyota Research Institute is developing robots that can learn from real-world experiences and even share their knowledge with others.
It has developed Diffusion Policy, which allows robotic AIs to self-learn by watching how a human carries out a given physical task.
At the same time, Figure AI, a two-year-old robotics company, is building general purpose robots that can quickly be used, starting with easy jobs like moving things around in a warehouse. Its robots are still under development, as is Tesla Optimus, a humanoid robot for industrial use.
Notably, Boston Dynamics which has been around for over 30 years, already has thousands of “legged” robots used in warehouses and industrial facilities. They are mobile robots with four legs that can traverse different environments and terrain.
The ultimate goal of robots with GenAI is to be able to understand natural language and self-learn, a goal which remains on the horizon.
For Nvidia, Project Gr00t enhances its Isaac platform. First announced in 2018, it is a toolbox for roboticists to accelerate the development, training, and deployment of AI-powered robots.
In his keynote last week, Huang said he was excited that the Gr00t foundation model could train robots to observe and self-learn in a real-world environment. This is unlike current robots which are mostly pre-programmed, moving on digital rails.
Project Gr00t’s core technology is Jetson Thor, a state-of-the-art computer based on the company’s latest cutting edge super graphics processing unit called Blackwell. It will speed up the training of robots, enabling them to understand natural language and copy human movements by observing them.
Jetson Thor runs an impressive 800 teraflops of 8-bit floating point AI performance, crucial for processing complex AI models to run multimodal GenAI models like Gr00t.
Engineers and roboticists can advance their work by using Gr00t to train robots for enhanced capabilities like dexterity and 3D surround-vision.
At GTC, Nvidia also announced it is building a comprehensive AI platform for leading humanoid robot companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics and Figure AI.
