Kyutai's open AURA model renders live talking-head avatars at 20.7 fps

Paris-based AI lab Kyutai has released AURA, a model that turns speech into a talking-face video in real time. According to Kyutai, the small version hits 20.7 frames per second on a single GPU. The weights and a training dataset are freely available.

Model releases Nov 20, 2026 6 min read Source: Bing News (en-US)
Illustrative image, AI-generated

What's new

Kyutai, a nonprofit AI research lab in Paris, has unveiled a new model called AURA. It takes spoken audio and generates a video of a speaking face, known as a "talking head." What sets it apart is that the video appears almost in real time, while the audio is still playing. Earlier models of this kind often needed several seconds of buffer before they could start.

AURA comes in two sizes: AURA 1.3B with 1.3 billion parameters and AURA 5B with 5 billion. Kyutai says the small version generates 576p video at 20.7 frames per second and the large one 720p at 9.7 frames per second, both on a single GPU. Kyutai is releasing the weights for both models along with a dataset. You can try a live demo at aura.kyutai.org. AI Weekly was among the outlets covering the release, and the details come from Kyutai's release on Hugging Face.

Kyutai has a track record here. The lab has already made a name for itself with speech models built for real-time conversation. AURA fits that direction, because a voice assistant with a face needs exactly this kind of model.

The problem AURA is trying to solve

Most current models that turn audio into video are diffusion models. They start from an image of pure noise and remove that noise over many small steps until a clear picture emerges, a bit like a photo slowly appearing in a developing tray. These steps are called denoising steps.

On top of that, these models are usually bidirectional. They look forward and backward in time at once and process entire video chunks several seconds long in one go. That works for a finished video. It doesn't work for a live conversation, because the model needs seconds of audio before it can even begin. The authors put it this way:

"Current audio-driven video diffusion models can generate realistic talking-head videos, but typically require dozens of denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars."

How AURA works

According to Kyutai, the speed doesn't come from a brand-new architecture. It comes from distillation: a large, slow "teacher" model trains a faster "student" model to reach similar results. Think of an experienced chef showing an apprentice shortcuts without the dish getting any worse.

The recipe has two stages:

  1. Causal student: First, a causal student model learns to imitate the teacher's denoising paths (trajectories). "Causal" means the model only looks at the past and present, never the future. To do that, it uses a chunk-wise sliding-window attention mask, so each video chunk can only attend to the chunks before it.
  2. Distribution Matching Distillation: Next, this method, combined with "Backward Simulation," closes the remaining quality gap. Put simply, the student is trained so that its outputs look like the teacher's overall, even though it uses far fewer steps.

To keep the model from forgetting what came before as it keeps generating, AURA uses a rolling KV cache. This is a kind of buffer where the model keeps the key information from previous chunks, with older entries dropping out over time. The video is built chunk by chunk, and the current audio chunk is all it needs to get started.

The numbers

Variant Resolution Frames per second Faster than teacher model
AURA 1.3B 576p 20.7 23x
AURA 5B 720p 9.7 17x

According to the paper, AURA 1.3B delivers quality comparable to TalkVerse, though at a lower resolution. TalkVerse is the reference model that also served as the teacher for the larger variant. AURA 5B is said to match its TalkVerse teacher at 720p while running 17x faster.

For context: at 20.7 frames per second, the small version comes close to smooth video (cinema runs at 24 fps). The large variant's 9.7 fps, on the other hand, will look rather choppy when played back live.

What Kyutai is releasing

Along with the weights for both models, Kyutai is providing a dataset of 30,000 synthetic videos, together with the teacher model's precomputed trajectories. That's mainly useful for researchers: if you want to reproduce the distillation method or apply it to your own models, you skip the expensive step of running the teacher model yourself. AI Weekly places AURA within a whole wave of open streaming-video releases that came out this month.

What you need to try it

The easiest option is the live demo at aura.kyutai.org. All you need is a browser. If you want to run the model yourself, you'll need an Nvidia GPU. The summary doesn't say which GPU Kyutai used to get 20.7 fps, so it may well take an expensive data center card. Our rough estimate, not Kyutai's: the 1.3B variant should start at around 8 GB of VRAM, more likely 12 GB once you factor in the extra audio and image components. The 5B variant needs noticeably more, probably 16 to 24 GB. On older or smaller cards the model will run, just not in real time. Don't expect real-time performance on Apple machines or CPU-only setups. Also budget a few gigabytes of disk space for the model and its dependencies, and a lot more for the dataset.

Our take

AURA is aimed at anyone who wants to build voice assistants, virtual presenters or customer service avatars with a face, without relying on commercial avatar services. Kyutai releasing not just the weights but also the distillation dataset is a real win for open research.

There are open questions, though. The summary names TalkVerse as the quality benchmark but publishes no benchmark numbers. It also doesn't reveal which GPU produced the 20.7 fps figure. Without that, it's hard to say how practical the model is on regular hardware. The license isn't clear from the sources either. And as with any technology that generates realistic talking faces, the potential for deepfake abuse is obvious. Whether Kyutai has any safeguards in place, and what they are, isn't clear from the available information.

Sources

#kyutai#avatar#video-generation#open-weights#real-time
Read the original source

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.