Early Failure Detection and Intervention
in Video Diffusion Models

Byungki Kwon¹ Sohwi Lim² Nam Hyeon-Woo¹ Moon Ye-Bin¹ Tae-Hyun Oh²

¹ Pohang University of Science and Technology (POSTECH)² Korea Advanced Institute of Science and Technology (KAIST)

NeurIPS 2026

Paper Code

Video quality metrics saturate early

Latent-to-RGB (L2R) preview

“A corgi is playing drum kit.”

Overall Consistency
Aesthetic Quality
Background Consistency

Subject Consistency
Imaging Quality

1,800-prompt mean · normalized error

Denoising step10 / 50

Drag left or right to explore

Detect early, intervene selectively

Waiting for a full video that misses the user's intent wastes both compute and time. We inspect early previews to catch likely failures, diagnose the mismatch, and intervene selectively. A lightweight single-frame probe guides semantic injection; visual feedback refines the prompt when needed. The aim is to reach a satisfactory video sooner, with less wasted generation.

Paper Figure 1: real-time inspection detects a failing bird generation from an intermediate preview, then diagnostic intervention corrects it. On CogVideoX-5B, ours improves the VBench final score from 0.778 to 0.800 with 16.55% extra time, compared with 0.793 and 43.73% for regeneration.

Latent-to-RGB (L2R) module enables early failure detection with negligible resource overhead

A tiny L2R module delivers previews ≈203× faster than the original CogVideoX decoder.

19.7 mspreview latency

0.059Mtrainable parameters

0.286 MBL2R checkpoint (FP32)

58,995 parameters · 0.286 MB L2R checkpoint for both CogVideoX and Wan.

“Iron Man flying in the sky” CogVideoX-5B

Shared latentStep 10 / 50Schematic

L2R (ours)

19.7 ms

Choose a step to decode

Original decoder

3.990 s

Choose a step to decode

Recorded previews replay the paper’s decoder latencies after loading. This is a timing illustration, not live model inference.

VBench results

Original-prompt VBench scores and additional generation time for four methods on two video diffusion models.
Method	CogVideoX-5B				Wan2.1-1.3B
Method	Final ↑	Quality ↑	Semantic ↑	Time overhead ↓	Final ↑	Quality ↑	Semantic ↑	Time overhead ↓
Original model (No intervention)	0.778	0.803	0.680	0.00%	0.789	0.811	0.700	0.00%
+ Regeneration	0.793	0.813	0.716	43.73%	0.815	0.823	0.783	51.25%
+ Ours	0.800	0.816	0.735	16.55%	0.805	0.817	0.756	20.30%
w/ Qwen3.8-27B (VLM)	0.791	0.812	0.706	16.49%	0.802	0.820	0.732	20.08%

Bold scores: best VBench result. GPT-5-mini is the default VLM; we also evaluated Qwen3.8-27B as an alternative.

VBench on detected failures

VBench + Panda70M · 872 CogVideoX / 811 Wan failures

VBench metric distributions on predicted failures: intervention shifts Overall Consistency, Subject Consistency, Background Consistency, Temporal Flickering and Motion Smoothness toward higher scores on both models.

Qualitative results

“A corgi is playing drum kit.”

Wan2.1-1.3B

Without intervention

With our intervention

More qualitative results

Future work

We aim to detect finer-grained failures earlier and develop more efficient, effective interventions. More expressive yet lightweight video evaluators could help identify subtle motion artifacts and compositional errors, reducing wasted generation compute for service providers and shortening the time to satisfactory videos for users.

BibTeX

Google Scholar ↗

@article{kwon2026earlyfailure,
  title   = {Early Failure Detection and Intervention in Video Diffusion Models},
  author  = {Kwon, Byungki and Lim, Sohwi and Nam, Hyeon-Woo
             and Moon, Ye-Bin and Oh, Tae-Hyun},
  journal = {arXiv preprint arXiv:2603.14320},
  year    = {2026}
}