The next decade of AI isn’t about understanding data that already exists, but about understanding new data—particularly the three-dimensional, physical world we inhabit. This core philosophy drives Fei-Fei Li and Justin Johnson, pioneers whose work from ImageNet to neural radiance fields (NERFs) has defined modern AI. They describe the current moment as a “Cambrian explosion,” where AI is expanding beyond language into pixels, video, audio, and, most fundamentally, spatial intelligence. For them, this evolution is a multi-decade continuum now reaching an inflection point, leading to their new venture, World Labs. The conversation traces the key unlocks—compute scaling, large datasets, and algorithmic breakthroughs—that brought us from supervised learning on labeled images to today’s generative models. However, they argue the next frontier is fundamentally different: moving from a one-dimensional, language-centric view of intelligence to machines that perceive, reason, and act within 3D space and time.

The discussion positions spatial intelligence as a capability as ancient and essential as language, critical for any entity—human, robot, or virtual agent—to interact with reality. While large language models (LLMs) operate on 1D sequences of tokens, spatial intelligence requires inherently 3D representations to model the physical world’s structure, physics, and affordances. This isn’t just about generating static 3D scenes; the vision encompasses dynamic, interactive worlds for gaming, education, and new media, as well as the operating system for augmented reality and robotics. The line between reconstructing the real world and generating imagined ones is blurring, thanks to techniques like NERFs, creating a unified approach to understanding and creating spatial environments.

World Labs aims to build the foundational models for this spatial layer of intelligence. The founders see the convergence of massively scaled compute, sophisticated algorithms, and new types of 3D-aware data making this possible now. The applications are vast, from drastically reducing the cost of creating rich, interactive 3D worlds to enabling seamless AR interfaces that could one day replace all traditional screens, to providing the “brain” for robots to navigate and manipulate the physical world. The team, including other luminaries like Ben Mildenhall (NERF) and Christoph Lassner (precursors to Gaussian splatting), embodies the multidisciplinary depth required for this deep tech challenge, combining expertise in machine learning, computer vision, graphics, and systems engineering.

Surprising Insights

• The staggering scale of compute growth: A training run for the landmark 2012 AlexNet model that took six days on two consumer GPUs would now take less than five minutes on a single state-of-the-art NVIDIA GB200 chip, highlighting the often-underestimated hardware explosion behind AI progress.
• Data’s role shifted from explicit labels to implicit structure: The key unlock for generative AI wasn’t just more compute, but learning to use data without explicit human labels. The era of supervised learning (e.g., ImageNet’s hand-labeled categories) gave way to systems that learn from the implicit structure in data, like the relationship between images and their alt-text on the internet.
• Reconstruction and generation are merging in computer vision: In the world of pixels, the historical fields of 3D reconstruction (deducing structure from photos) and generative AI are converging. Techniques like NERFs mean the same fundamental models can now both reconstruct a real scene and generate a novel one, a pivotal but under-discussed shift compared to the LLM narrative.
• Language is a “lossy” representation of the physical world: The hosts argue that language, being a purely human-generated, 1D signal, is an abstract and often lossy description of the rich, 3D world governed by physics. This makes spatial intelligence a fundamentally different problem from language modeling, requiring native 3D representations.
• Spatial intelligence could deprecate physical screens: In a future with advanced AR, spatial intelligence models that seamlessly blend digital information with the physical environment could make dedicated screens (phones, monitors, TVs) obsolete, as information is presented contextually directly in the user’s field of view.

Practical Takeaways

• When thinking about next-gen AI applications, consider the 3D and spatial context. The most transformative uses may not be in chat interfaces but in areas involving navigation, manipulation, design, and interaction with the physical or virtual 3D world.
• For technical builders, investing in understanding 3D data representations (like NERFs, Gaussian splats, and neural fields) is crucial. The underlying architecture of future models will likely prioritize these over sequences of tokens for spatial tasks.
• The convergence of AI and computer graphics is a major trend. Expertise in rendering, simulation, and graphics pipelines is becoming as valuable as traditional machine learning knowledge for building spatial intelligence systems.
• Start exploring how your domain could be changed by a shift from 2D to 3D media. Whether it’s education, entertainment, e-commerce, or professional training, the ability to generate or interact with interactive 3D worlds cheaply will open new frontiers.
• Leverage open models and tools in the 3D vision space to experiment. The field is moving rapidly, with many foundational techniques (like those underlying NERF) being accessible and implementable, providing a playground for prototyping spatial AI ideas.