NVIDIA opens four Cosmos 3 world models so robots and self-driving cars can practice in simulation

NVIDIA has opened up the part of its world-model platform that robotics and self-driving teams have been waiting to touch: the weights. On November 5, the company published four open-weight checkpoints from its Cosmos 3 family, built for interactive world simulation — software that predicts what a camera, a LiDAR sensor, or a robot would see next, so a policy can be trained and tested against a world that answers back.

It targets the bottleneck in physical AI: real-world testing is slow, expensive, and dangerous, while simulation has historically been too crude to trust. Cosmos 3 tries to close that gap by generating the observations themselves — synchronized video across many camera views, matching LiDAR, and video that responds to a robot’s own actions — under a license that permits commercial use.

Four checkpoints, two industries#

The four models are all post-trained versions of Cosmos 3 Nano, each with 16 billion trainable parameters, and each aimed at a different slice of simulation work. Two serve autonomous driving; two serve robotics.

The split is deliberate. Driving teams need appearance and geometry to agree — the camera view and the LiDAR return have to describe the same world — while robotics teams need a world that reacts when the robot moves. NVIDIA frames the four as complementary tools for building those environments, not one model stretched across both jobs.

What each one actually does#

Transfer-Auto is the foundation of the set. It is a bidirectional model for generating synthetic driving data offline: synchronized video across as many as 11 camera views, including fisheye lenses, with joint RGB and LiDAR generation. Per-camera World Scenario Map controls set the scene structure and traffic motion; text prompts steer appearance; optional initial images anchor what the scene looks like, and aligned HD-map range controls guide the LiDAR. It is also the starting point for adapting Cosmos to a team’s own sensor rig and driving domain.

Sim-Auto is that streaming variant. Causally distilled from Transfer-Auto, it generates successive short chunks of synchronized camera video and LiDAR in a handful of denoising steps, using its rolling history and updated driving controls to keep long, interactive rollouts going. Closed-loop use still needs a simulation runtime to update those controls from driving inputs.

On the robotics side, Sim-Transfer takes a different deal: the physics simulator keeps the ground truth — motion, contacts, physical state — and the model renders it. Depth, segmentation, edges, and blur controls plus a text prompt become photorealistic video a policy can consume. Sim-Bimanual goes one step further and predicts the next observation directly from the robot’s actions, supporting interactive two-arm simulation for YAM and AgiBot robot designs, driven by a learned policy or by a person on joysticks.

The recipe ships with the weights#

The weights are only half the release. NVIDIA also published the full post-training recipe for each checkpoint — a four-stage process that starts by adapting the base model to a domain from complete recorded clips, then trains it to predict each chunk from earlier observations, distills it into a student that generates each chunk in four denoising steps, and finally trains on the student’s own generated history so small errors do not compound into drift. Inference notebooks and agent-runnable skills for each checkpoint are published too, so a coding agent can run the adaptation itself.

A companion post published three days earlier describes the broader Cosmos Framework that trains and serves these checkpoints — and extends the same tooling to Alpamayo, GR00T, Qwen, and pi0.5 models. The release is the largest expansion of the Cosmos 3 family since the platform’s main launch at GTC Taipei in May.

What it takes to run — and what NVIDIA won’t claim yet#

Running the checkpoints is workstation-class work, not laptop work. NVIDIA lists Linux, an NVIDIA GPU, Docker, and the NVIDIA Container Toolkit as requirements, and the published streaming setup was tested on a single GB200. The company’s published memory table puts Cosmos 3 Nano generation at 58 GiB of VRAM in BF16, or 44 GiB in FP8. The models run on Diffusers, PyTorch, SGLang, vLLM, vLLM-Omni, and NVIDIA’s TensorRT stacks,

Just as notable is what the announcement does not claim. NVIDIA says plainly that more work is needed to measure where the models faithfully support policy learning and evaluation, and to understand their limits. A world model that looks right can still teach a policy the wrong lessons, and the company is not pretending that question is settled.

Why it matters#

For teams building robots and self-driving systems, the practical effect is a shared starting point. Until now, a team that wanted a reactive simulated world either built its own generator or bought into a closed stack. Open weights plus the training recipe mean a lab can adapt the model to its own sensors, compare methods on common ground, and contribute improvements back. The weights are free; the compute to adapt and run them is not, and whether a policy trained in a generated world survives contact with the real one is now the field’s problem to measure.

What to watch: whether robotics and driving teams publish evaluations showing policies trained on Cosmos 3 transferring to real hardware, whether the larger Cosmos 3 variants get open checkpoints of their own, and whether the post-training recipes get picked up by teams working outside NVIDIA’s own robot designs.

Sources: NVIDIA (Hugging Face blog), ‘New NVIDIA Cosmos 3 Open Models for Interactive World Simulation’ (November 5, 2026), here; Pondero, ‘NVIDIA opens four Cosmos 3 Nano checkpoints for robot and self-driving simulation’ (November 7, 2026), here; Extrapolator, ‘NVIDIA releases four Cosmos 3 open-weight models for world simulation’ (November 5, 2026), here.