Announced at the SIGGRAPH 2024 conference, developers in the humanoid robotics space will have access to a new suite of tools and services from NVIDIA, including NVIDIA NIM microservices for robot simulation in Isaac Lab and Isaac Sim, OSMO robot cloud computing orchestration services, teleoperation data capture workflows, and several other services.
NVIDIA founder and CEO Jensen Huang spoke at the conference, highlighting the company’s efforts in advancing humanoid robots. “The next wave of AI is robotics, and one of the most exciting developments is humanoid robots,” Huang said. “We’re advancing the entire NVIDIA robotics stack to ensure humanoid developers and companies around the world have access to the platform, acceleration libraries and AI models that best suit their needs.”
NVIDIA NIM microservices include pre-built containers with NVIDIA inference software and are designed to significantly reduce deployment time. Two new AI microservices, MimicGen and Robocasa, are set to enhance NVIDIA Isaac Sim’s generative physics AI simulation workflows. MimicGen generates synthetic motion data from spatial computing devices such as Apple Vision Pro. Robocasa creates robot tasks and simulation-enabled environments with OpenUSD, a framework for 3D world development and collaboration.
In addition to NIM, NVIDIA introduced OSMO, a cloud-native management service that aims to simplify and accelerate robot training and simulation workflows, reducing development cycle times from months to less than a week. Tasks managed by OSMO include generating synthetic data, training models, conducting reinforcement learning, and implementing large-scale software-in-the-loop testing for a variety of robotics applications.
Data collection remains a critical component in the development of humanoid robots, and NVIDIA demonstrated an AI- and Omniverse-enabled teleoperation reference workflow at the conference. Researchers can now generate large amounts of synthetic motion and perception data from minimal teleoperated human demonstrations. The process involves capturing several teleoperated demonstrations using Apple Vision Pro, which are then simulated in NVIDIA Isaac Sim using the MimicGen NIM microservice to generate a synthetic dataset.
The collected synthetic and real data is used to train the Project GR00T humanoid foundation model. This approach saves developers time and reduces costs. They use Isaac Lab’s Robocasa NIM microservices to generate new experiences and retrain models. Throughout this workflow, NVIDIA OSMO handles the allocation of computing jobs across different resources, further optimizing the process.
Alex Gu, CEO of Fourier, a company developing a general-purpose robotics platform, expressed optimism about the new tools. “Developing humanoid robots is extremely complex and requires huge amounts of real data painstakingly collected from the real world,” Gu said. “NVIDIA’s new simulation and generative AI developer tools will help us launch and accelerate our model development workflow.”
NVIDIA is also expanding access to technology for humanoid developers through three major computing platforms: the NVIDIA AI supercomputer for model training, NVIDIA Isaac Sim built on Omniverse to refine robot skills in simulated environments, and the NVIDIA Jetson Thor humanoid robotics computer to run these models. The new NVIDIA Humanoid Robotics Developer Program provides early access to these new tools, as well as the latest updates to NVIDIA Isaac Sim, NVIDIA Isaac Lab, Jetson Thor and Project GR00T.
Several companies have already joined the early access program, including 1x, Boston Dynamics, ByteDance Research, Field AI, Figure, Fourier, Galbot, LimX Dynamics, Mentee, Neura Robotics, RobotEra and Skild AI. Aaron Saunders, CTO of Boston Dynamics, said about the collaboration: “Boston Dynamics and NVIDIA have worked closely for many years to push the boundaries of what’s possible in robotics. We’re really excited to see the results of this work accelerate the entire industry. The early access program is a great way to gain access to best-in-class technology.”
Developers can join the NVIDIA Humanoid Robot Developer Program today to get access to the NVIDIA OSMO and Isaac Lab, with NVIDIA NIM microservices available shortly thereafter.
