AI's next big leap is models that understand the world.

"For all the book smarts of LLMs, they currently have little sense for how the real world works. Driving the news: Some of the biggest names in AI are working on world models, including Fei-Fei Li whose World Labs announced Marble, its first commercial release. Machine learning veteran Yann LeCun plans to launch a world model startup when he leaves Meta, reportedly in the coming months."

"Google and Meta are also developing world models, both for robotics and to make their video models more realistic. Meanwhile, OpenAI has posited that building better video models could also be a pathway toward a world model. As with the broader AI race, it's also a global battle. Chinese tech companies, including Tencent, are developing world models that include an understanding of both physics and three-dimensional data."

"World models learn by watching video or digesting simulation data and other spatial inputs, building internal representations of objects, scenes and physical dynamics. Instead of predicting the next word, as a language model does, they predict what will happen next in the world, modeling how things move, collide, fall, interact and persist over time. The goal is to create models that understand concepts like gravity, occlusion, object permanence and cause-and-effect without having been explicitly programmed on those topics."

World models enable AI to learn physical and temporal dynamics by watching video and processing simulation and spatial data, building internal representations of objects, scenes and interactions. Major AI groups are developing world models: World Labs released Marble; Yann LeCun plans a startup after leaving Meta; Google and Meta pursue models for robotics and more realistic video; OpenAI views improved video models as a path to world models. Chinese firms like Tencent and institutions such as MBZUAI (PAN) are building models with physics and 3D understanding. The aim is models that grasp gravity, occlusion, object permanence and cause-and-effect.

Read at Axios