Nvidia Unveils AI Tools to Accelerate Humanoid Robot Development

Nvidia has unveiled a suite of AI models and services to enable robotics developers to accelerate the development of humanoid robots.

At the annual Siggraph event held in Colorado, Nvidia announced a host of new solutions to help manufacturers and developers “build the next generation of humanoid robots.”

“The next wave of AI is robotics and one of the most exciting developments is humanoid robots,” said Jensen Huang, Nvidia’s founder and CEO. “We’re advancing the entire Nvidia robotics stack, opening access for worldwide humanoid developers and companies to use the platforms, acceleration libraries and AI models best suited for their needs.”

Among the offerings are new versions of Nvidia NIM, a suite of microservices designed to provide pre-built containers for AI tasks.

The MimicGen NIM microservice generates synthetic motion data based on recorded teleoperated data from spatial computing devices like the Apple Vision Pro.

The Robocasa NIM microservice generates robot tasks and simulation-ready environments in OpenUSD, enabling robotics developers to create 3D testing environments.

Also unveiled at Siggraph was Osmo, a cloud-native managed service where users can orchestrate and scale complex ‌robotics development workflows across distributed computing resources either on-premises or in the cloud.

Related:Nvidia, Pfizer Back AI Drug Discovery Startup

Users can visualize and manage tasks such as conducting reinforcement learning for humanoids at scale. Nvidia said that Osmo can be used to cut deployment and development times from months to under a week.

The new tools are being made available through the new Nvidia Humanoid Robot Developer Program, which also provides access to Project GR00T, a general-purpose foundation model designed to power humanoid robots and Issac Sim, a reference application for robotics simulation built on Nvidia’s Omniverse platform.

Developers have to apply for access with big-name robotics players including 1x, Boston Dynamics, Figure, Fourier and Skild AI among those given early access.

“Boston Dynamics and Nvidia have a long history of close collaboration to push the boundaries of what’s possible in robotics,” said Aaron Saunders, Boston Dynamics’ chief technology officer. “We’re really excited to see the fruits of this work accelerating the industry at large and the early-access program is a fantastic way to access best-in-class technology.”

New Solutions Work in Tandem

At Siggraph, Nvidia said the new suite of tools was a way to reduce expensive humanoid development costs and time as developers can use simulation technologies to synthetically generate training data.

Related:Nvidia Overtakes Microsoft as World's Most Valuable Company

Developers wearing a device like the Apple Vision Pro can capture demonstrations, which can then be fed into the new MimicGen NIM microservice, which uses the data to create synthetic datasets.

Next, Nvidia’s Project GR00T model can be trained on the data generated by MimicGen NIM, with the Robocasa NIM then brought in to generate experiences to retrain the robot model.

As the researchers are reworking and improving the model, the new Osmo tool assigns jobs to different resources, which automates administrative tasks that would have added additional burden to the developers.

“Developing humanoid robots is extremely complex, requiring an incredible amount of real data, tediously captured from the real world,” said Alex Gu, CEO of the general-purpose robotics company Fourier. “Nvidia’s new simulation and generative AI developer tools will help bootstrap and accelerate our model development workflows.”