Waymo, the autonomous driving unit of Alphabet Inc, announced a new simulation capability built on DeepMind’s Genie 3 AI model. Named the Waymo World Model, the system produces realistic digital driving environments intended to accelerate the training of Waymo’s self-driving technology.
In a company blog post published Friday, Waymo said the World Model leverages Genie 3’s pre-training on diverse video sources to generate synthetic camera and lidar sensor data. The company described the output as hyper-realistic and said engineers can control and alter scenes using plain language prompts, driving inputs and scene layouts. Waymo framed the collaboration with DeepMind as a way to support expansion of its self-driving services into additional markets.
Waymo noted that its Waymo Driver has already completed nearly 200 million fully autonomous miles in major U.S. cities and has driven billions of miles inside virtual environments. The World Model is presented as a complementary method to those virtual miles, enabling training on highly unusual or hard-to-capture scenarios - examples cited include tornados and encounters with elephants.
According to the company, the Waymo World Model differs from traditional simulation approaches that start from scratch with only collected on-road data. Instead, Waymo builds on Genie 3’s extensive pre-training from varied video inputs to extend the breadth of scenarios it can reproduce. The generated simulations combine visual detail from camera data with depth and range detail from lidar, offering a multi-sensor training input for the Waymo Driver.
Waymo characterized the World Model as a "frontier generative model" and said it represents a new benchmark for large-scale autonomous driving simulation. The company also called the capability a critical component of its broader AI ecosystem.
The blog post does not include quantitative measures of how Waymo expects the World Model to change on-road outcomes, nor does it provide performance comparisons between World Model-trained systems and those trained only on prior virtual or real-world miles.
Summary
Waymo has introduced the Waymo World Model, a simulation engine powered by DeepMind’s Genie 3 that generates camera and lidar data from text prompts and pre-trained video knowledge to enable training on rare driving scenarios.
Key points
- Waymo World Model uses Genie 3’s pre-training to create hyper-realistic virtual environments for autonomous vehicle training.
- The system produces both camera imagery and lidar depth information and allows engineers to modify scenarios with simple language prompts, driving inputs and scene layouts.
- Sectors impacted include automotive and transportation services, autonomous vehicle software and broader technology platforms that supply AI and simulation tools.