Nvidia Lyra 2.0 turns a single photo into a 3D world you can walk through

What's going on

Nvidia has unveiled Lyra 2.0, a research system that turns a single photo into a 3D environment you can explore. You're not stuck with the original framing: you steer a virtual camera through the scene yourself, peek around corners and go back to places you've already been. The world stays consistent the whole time. Lyra 2.0 was developed by Nvidia's Spatial Intelligence Lab (SIL).

The system isn't brand new. According to the GitHub repository, Lyra 2.0 was released on April 15, 2026, and the paper is on arXiv (2604.13036). A graphical user interface (GUI) and the training code followed on July 20, 2026. Wccftech reports that Nvidia is now putting the technology back in the spotlight ahead of the SIGGRAPH Asia 2026 conference. The code lives at github.com/nv-tlabs/lyra, and the weights are on Hugging Face. The weights, however, come under an Nvidia license that only permits research use.

Lyra 2.0 succeeds Lyra 1.0 from September 2025, which generated 3D and 4D scenes from an image or video and was presented at ICLR 2026.

The problem: AI worlds forget and drift

Standard image-to-video generators produce a fixed clip you have no control over. With Lyra 2.0, you set the camera path yourself, and the system generates new views piece by piece as you move. According to The Decoder, this is exactly where previous models fall short in two ways:

• Spatial forgetting: Once an area leaves the frame, the model forgets it. When the camera comes back, it reinvents the surroundings. Suddenly the sofa is a cabinet.
• Temporal drifting: The video is built step by step, and small errors in colors and shapes add up. The farther the camera moves from its starting point, the more distorted the scene becomes.

How Lyra 2.0 solves it

To fight forgetting, Lyra 2.0 gets a spatial memory. For every generated frame, the system stores its 3D geometry, meaning where everything sits in space. When the camera moves toward a known area, it pulls up the matching earlier frames and uses their spatial information as a guide. According to The Decoder, though, the video model still draws the new frames itself. That way, errors in the stored geometry don't end up directly in the new image.

To fight drift, the researchers use a training trick. During training, the model is deliberately shown its own flawed outputs. That teaches it to spot degrading quality and correct it instead of carrying errors forward and amplifying them.

The result is video sequences that can be converted into two kinds of 3D data:

• 3D Gaussian Splats: The scene is made up of millions of tiny, semi-transparent blobs of color. It looks highly photorealistic and can be rendered in real time.
• Polygon meshes: Classic triangle geometry, the standard in games and physics engines.

An interactive interface lets you explore the scenes step by step. As meshes, they can be exported to physics simulators such as Nvidia Isaac Sim, Nvidia's simulation environment for robots.

The numbers: speed and how it stacks up

The full generation process is expensive, so Nvidia built a distilled model. Distillation here means a faster model learns to reproduce the larger model's results with far fewer compute steps. Instead of 35 denoising steps (the image emerges gradually from noise), it needs only four.

That's about 13 times faster, with comparable quality according to The Decoder. Wccftech points out, however, that this still isn't real-time generation. Only the finished, converted 3D scene can be rendered in real time. Generating the world itself remains compute-heavy, and the GB200 is a data center chip, not a gaming graphics card.

According to The Decoder, Lyra 2.0 beats six competing methods, including GEN3C, Yume-1.5 and CaM, in benchmarks on two datasets. That holds for almost every measured criterion, such as image quality, style consistency and camera control. These results come from Nvidia itself. According to the paper, generated scenes can span around 90 meters.

What it's good for

Nvidia's main pitch is robotics. Robots are often trained in virtual worlds before they go to work in the real one. Building those environments by hand or painstakingly 3D-scanning real rooms takes a lot of time. Just how much effort it takes to import even ready-made generated worlds is shown in an Nvidia tutorial on World Labs' rival product Marble: Gaussian Splats and a collision mesh have to be aligned, scaled and given physics by hand before a robot can drive through the kitchen. Nvidia says Lyra 2.0 could let robots train in fully generated environments, with no real 3D captures at all.

Wccftech also sees potential for game development and virtual reality. As a comparison, the site points to Google's Project Genie, which also generates explorable worlds. Lyra 2.0 takes a different approach: it converts its output into real 3D geometry that common software can render and simulate.

What you need to try it

Lyra 2.0 is a research project, not an app you just install. Code, GUI and training instructions are on GitHub. The sources don't list minimum requirements for home PCs, and Nvidia's measurements come from a data center GB200. So expect to need a powerful Nvidia graphics card with plenty of VRAM (video memory) and a Linux setup with CUDA, Nvidia's software interface for GPU computing. Nvidia's related 3DGRUT tools, which convert Gaussian Splats for Isaac Sim, also require Linux, an Nvidia GPU and CUDA 11.8 or newer. Running it on a Mac or CPU-only isn't supported. The free Google Colab tier is likely too weak for it. Budget plenty of disk space for weights and generated videos, too. And keep the license in mind: the weights are cleared for research purposes only.

The bottom line

Lyra 2.0 tackles two of the most stubborn problems with generated worlds: scenes that change when you return, and errors that pile up over time. Because the end result is real 3D geometry rather than just a video, the system is practically useful for simulation and robotics. That's what sets it apart from pure world models like Google's Project Genie.

But the limits are clear. Lyra 2.0 only handles static scenes: nothing moves, and dynamic content remains an open problem. According to Wccftech, visual glitches still crop up occasionally. The benchmark results come from Nvidia itself, and the compute requirements are far beyond what most people have at home. The research license rules out commercial use of the weights for now. Turning a photo into a playable video game is still a long way off. As a tool that makes building environments easier for developers and robotics teams, though, Lyra 2.0 is a serious step forward.