NVIDIA Cosmos

NVIDIA's family of world foundation models and tools for physical AI; generates and reasons over video of the physical world for robot training, simulation and synthetic sensor data.

Cosmos is NVIDIA's platform of world foundation models for physical AI. The current flagship, Cosmos 3, is described as a single model that combines reasoning, world generation and action generation on a mixture-of-transformers architecture; around it sit Cosmos Curator for filtering and annotating sensor data, Cosmos Evaluator for scoring generated video, and scripts for post-training, optimisation and inference.

NVIDIA lists four uses: as a vision-language model that reasons over video, for tasks such as quality inspection and traffic monitoring; as a backbone to post-train into robot policies on a specific camera and body; as a controllable simulator that predicts outcomes in a closed loop; and as a generator of synthetic video from text, images, video, sound or actions, including sensor data for autonomous-vehicle testing.

The models are released under the OpenMDW 1.1 licence from the Linux Foundation and can be downloaded from Hugging Face, with code on GitHub and recipes in a cookbook; NVIDIA also hosts them in its build.nvidia.com catalogue and as NIM microservices. Status: operational.