Nvidia's Cosmos 3 open AI world model helps robots, autonomous vehicl…

Nvidia Introduces Cosmos 3 Open AI World Model for Robots and Vehicles

Nvidia has released Cosmos 3, an open AI world model intended to assist robots, autonomous vehicles, and other physical systems in understanding and predicting real-world environments. The company continues to expand beyond semiconductor production into AI models and software, seeking to establish infrastructure for physical AI applications.

Nvidia states that Cosmos 3 was trained on 20 trillion tokens of multimodal data. This dataset includes nearly a billion images, 400 million real and synthetic videos, ambient audio, text, and action data collected from humans and robots. These specific figures originate from Nvidia and have not been independently verified in public sources. Ming-Yu Liu, vice president of Nvidia’s Cosmos Lab, said the action data component sets Cosmos apart from standard video generators because it models how machines move rather than only how scenes appear. Autonomous actions form a central element of the design.

Developers can apply Cosmos 3 to simulate actions inside physical environments and then construct task-specific models for robots and other machines. The system produces action data such as robot joint angles, gripper positions, and trajectories to support training for navigation and physical manipulation tasks.

Cosmos is distributed as an open model, comparable to Nvidia’s earlier Nemotron family. Liu indicated this approach allows hardware manufacturers to customize the model according to their requirements and to help future versions align more closely with industry needs. Nvidia has formed a coalition of supporting companies. Initial members include Agile Robots, Black Forest Labs, and Runway. Nvidia reports that Cosmos can generate simulations of rare or hazardous events, including robot collisions and unusual road incidents, that are difficult, costly, or unsafe to record repeatedly in real conditions.

Nvidia is releasing two versions at launch. The “super” model targets applications that require high physics accuracy, such as training robots and autonomous vehicles. The “nano” model produces outputs in fractions of a second. An “edge” model designed for local operation is scheduled for release soon.

World models have emerged as a focus area within AI development as organizations seek to extend the capabilities of chatbots and agents toward real-world tasks. Among active participants in this field are World Labs, founded by Fei-Fei Li, and AMI Labs, associated with Yann LeCun. Liu stated that a world model ultimately aims to improve how physical agents generalize across situations. Greater generalizability requires understanding how the world operates in order to form plans.

Nvidia’s stated position is that upcoming AI systems will need to predict, simulate, and act within physical environments. The company positions its open models and supporting infrastructure as resources developers can use as a starting point for such work.