Nvidia Unveils AI Tools to Boost 3D World Creation at Siggraph 2024

Nvidia has unveiled a host of new generative AI tools at the annual Siggraph event designed to supercharge the creation of 3D worlds and digital twins.

At Siggraph 2024 in Colorado, Nvidia announced an expansion of its support for the Universal Scene Description (OpenUSD) standard for virtual environment applications.

The chipmaker will add new generative AI frameworks that would enable businesses to develop simulation applications like testing environments for robots or digital twins of factory sites.

Nvidia also introduced a new Nim microservice enabling AI models to generate outputs in the OpenUSD language allowing users to generate code to build assets for their 3D environments. Nim is a set of microservices that provide AI-powered tools to help with software development.

“The generative AI boom for heavy industries is here,” said Rev Lebaredian, Nvidia’s vice president for Omniverse and simulation technologies. “Until recently, digital worlds have been primarily used by creative industries; now, with the enhancements and accessibility Nvidia Nim microservices are bringing to OpenUSD, industries of all kinds can build physically based virtual worlds and digital twins to drive innovation while preparing for the next wave of AI: robotics.”

Related:Nvidia Unveils AI Tools to Accelerate Humanoid Robot Development

New Nim AI Tools

OpenUSD is a file format used in computer graphics that breaks down a 3D scene into different layers.

It is an open-source standard, originally developed by Disney-owned animation studio Pixar, that has seen wide use in entertainment, engineering and manufacturing, enabling users to construct virtual environments for testing and simulations.

Nvidia’s new tools unveiled at Siggraph 2024 are designed to expand the adoption of 3D data to enable more businesses to build highly accurate virtual worlds.

Among them are the world’s first generative AI models designed for OpenUSD development. 

Available as Nim microservices, the models enable developers to add generative AI tools to help with workflows, like copilots to help with code generation.

The USD Code Nim microservice, for example, can automatically generate OpenUSD-Python code based on text prompts and the USD Search Nim can speed up searches of asset libraries, enabling developers to find 3D objects faster.

The new USD Validate Nim microservice service can check file compatibility against OpenUSD release versions, generating a fully rendered version better optimized for 3D environments.

Nvidia also showcased USD Layout Nim, which can automatically assemble OpenUSD scenes from text prompts, fVDB Mesh Generation Nim, which can generate meshes for objects and fVDB Physics Super-Res Nim, which generates physics for frames making them suitable for simulation.

Related:Nvidia Overtakes Microsoft as World's Most Valuable Company

Semiconductor manufacturing firm Foxconn used the new Nim microservices along with Nvidia’s Omniverse platform to build a digital twin of a chip factory currently under development.

“Digital twins will help us accelerate the next wave of industrial manufacturing and autonomous machines,” said Zhe Shi, Foxconn’s chief digital officer. “Nvidia Omniverse and the new Nim microservices will democratize the ability to develop digital twins and help our teams build physically based virtual factories faster than ever.”

Marketing and communications services company WPP was also given early access, using the new Nim microservices to build a generative AI-enabled content creation pipeline for customers like The Coca-Cola Company.

“Using Nvidia Nim microservices with Omniverse has made it possible for us to launch innovative new production tools with companies like The Coca-Cola Company at unprecedented speed,” said Stephan Pretorius, WPP’s chief technology officer.

Robotics Connectors, Spatial Streaming Support

At Siggraph 2024, Nvidia also unveiled new USD connectors designed specifically for robotics data formats.

The connection allows robotics developers to transfer their robot data across USD-supported applications, enabling them to use 3D environments to conduct simulations or to power reinforcement learning efforts.

Developers can also build their own OpenUSD data connectors through Nvidia’s new OpenUSD Exchange software development kit.

Nvidia also unveiled a suite of tools that would let users stream datasets to spatial computing devices like Apple’s Vision Pro headset.

Users can stream large-scale OpenUSD scenes from an application built on Omniverse to their headsets, enabling them to interact with highly detailed environments on their mixed-reality devices.

Nvidia partnered with Siemens to expand the application to enable more industrial workloads to run on OpenUSD.

Siemens will integrate OpenUSD pipelines in its Simcenter simulation suite, providing high-fidelity simulation data so users can obtain insights on real-world operating environments, virtually.

Nvidia and Pixar joined the likes of Apple, Adobe and Autodesk to found the Alliance for OpenUSD, a nonprofit seeking to expand the use of OpenUSD beyond the entertainment industry.

“OpenUSD is revolutionizing the way we create and interact with 3D content,” said Steve May, Pixar’s chief technology officer and chair of the Alliance for OpenUSD. “Now, with these new services and APIs for OpenUSD built by Nvidia, we expect to see accelerated growth and adoption of USD, paving the way for new users and industries to more easily engage with our ecosystem.”