Summary
- Runway has detailed the real-time video infrastructure behind Runway Characters.
- Its roughly 57B-parameter model produces four-frame chunks to sustain 24 fps.
- Optimization reduced diffusion step time to 159.3 ms on four H100 GPUs.
- Median latency before the character’s spoken response reaches the client remains 2.13 seconds.
What does it take for an AI model to generate video as quickly as we watch it? Runway has published the engineering behind its live conversational characters.
In a research post titled “Real-Time Video at Batch Size One,” published on November 5, 2026, Runway explains how it made a roughly 57-billion-parameter video-generation model operate in real time. This is not a new product launch: it is a technical breakdown of the system already powering Runway Characters.
Unlike a conventional AI video generator, the system cannot simply process a prompt and deliver a finished clip minutes later. A digital character needs to listen, respond and keep moving throughout a live conversation.
Four new frames every 166.7 milliseconds
Runway Characters can build a digital character from a single image. Behind it is an approximately 57B-parameter autoregressive video diffusion transformer that generates video progressively rather than producing a complete sequence as one finished asset.
To maintain 24 frames per second, it must deliver a new four-frame chunk of 704 × 1088 video every 166.7 ms. If a chunk arrives late, both the picture and the voice may stall, because the stream carries synchronized video and audio.
“Batch Size One” captures the constraint: each live conversation requires its own continuous stream, without the usual efficiency gains of batching several users’ requests together.
From eight H100 GPUs to four: reducing generation time
At launch, Runway used eight NVIDIA H100 GPUs per session. With the original implementation, a step on four H100s took 240.9 ms, well over the 166.7 ms target. Following optimization, it took 159.3 ms, sustaining around 24.5 fps. An additional fused compute-and-communication kernel reduced that diffusion step to 147.4 ms.
Much of the improvement came from eliminating redundant work across the 240 attention calls per step. Runway changed how the model’s cache is maintained, preparing new data when written and using a ring buffer. Cache maintenance fell from 69.7 ms to 6.2 ms per step.
It also fused matrix multiplication with inter-GPU communication, allowing partial results to move while computation continues. Turning the generated data into actual video frames happens on a separate GPU, in parallel with the next diffusion chunk. That distinction matters: four H100s describe the diffusion stage, not necessarily the total hardware used by an entire session.
24 fps video does not mean an instant conversational reply
Smooth frame delivery and conversational response latency measure different things. Across 51 turns in 12 live sessions, Runway measured a 2.13-second median from the end of a user’s speech to the character’s voice reaching the client. The voice agent accounted for 1.42 seconds, the pipeline from the agent’s first audio to the first speaking frame took 0.49 seconds, and browser delivery added approximately 190 ms.
According to seven weeks of the company’s telemetry, average performance remained above 23.5 fps in 99.5% of sessions that streamed for at least one second. These are Runway’s production measurements rather than independent benchmarks.
Visual quality and the realities of production
Runway notes that tiny numerical changes in an autoregressive video system can cause later frames to diverge, making direct pixel comparisons misleading. Instead, it examined lip-sync measurements and repeated runs. Across 51 clips that SyncNet could score, mean differences stayed at or below 0.03, comparable to variation between reruns of the same configuration.
The paper also details a production failure. After a PyTorch and NCCL upgrade, an optimized kernel ran into memory issues because benchmarks had not covered varying prompt lengths. Runway temporarily disabled it, confined its use to steady-state workloads and re-enabled it on October 8, 2026. The episode shows how much always-on AI video depends on the serving system, not just the model.
Why video creators should pay attention
The approach is particularly relevant to interactive digital presenters, education and characters that respond dynamically to an audience. It is different from generating completed AI shots for an edit: Runway tools integrated into Premiere Pro and After Effects focus on generating and editing timeline assets, whereas Characters produces a continuous live video stream.
What we think
For those of us working in video, the most interesting change is that AI is now facing a timing constraint familiar from live production. Producing an attractive frame is not enough. The next frame has to arrive on time, or the entire conversation loses its rhythm.
What stands out is how much of the progress came from better cache and GPU utilization rather than another new model. That matters in practical terms: if each session requires less processing time, interactive productions serving real audiences become more realistic to run.
But 24 fps alone does not direct a convincing digital performance. Delivery, pauses, eye contact and timing still shape whether a character feels natural. The 2.13-second median reply latency is a reminder that the next improvements have to address the whole conversational experience, not only the frames.
Frequently asked questions
What does “Batch Size One” mean here?
Each conversation needs a continuous per-user video stream, without waiting to batch multiple requests together for efficiency, as many other AI systems do.
Does Runway Characters reply without delay?
No. Around 24 fps refers to smooth video streaming. Runway reports a median 2.13 seconds from the end of user speech until the character’s audio reaches the client.
