Correction Training for Diffusion Models
Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noise, but at inference its inputs are constructed from its own imperfect predictions from the earlier steps, something it has never seen at training…
This article presents a method to improve the performance of diffusion models by addressing error accumulation during iterative sampling through a correction step.
What happened
The article introduces a correction step for diffusion models to address error accumulation during iterative sampling. The corrector takes the base model's predictions as input and is trained against the same ground-truth target, allowing for iterative refinement of denoising steps to prevent early artifacts from persisting in the final sample.
Why it matters
This method addresses a critical challenge in diffusion models, improving sample quality and diversity without significant additional computational costs, which is crucial for applications requiring high-quality, diverse outputs.
- Addressing exposure bias in diffusion models can significantly improve sample quality.
- Iterative correction steps during sampling help prevent early artifacts from affecting final outputs.
- Combining correction with existing techniques like CFG can enhance diversity and reduce oversaturation.
The bigger picture
The method offers a practical way to enhance diffusion model performance, making them more reliable for generating high-quality, diverse samples. It also provides a cost-effective solution by allowing distillation of corrections into the base model.
The article presents an effective correction framework for diffusion models that reduces error accumulation during sampling, leading to improved sample quality and diversity while maintaining computational efficiency.
Technical details
- Exposure bias in diffusion models due to training on real images with noise versus inference on predictions from earlier steps.
- Correction step that refines denoiser predictions iteratively during sampling.
- Training the corrector on the base model's predictions to reduce the training-inference gap.
- Distillation of refined predictions back into the base model to maintain quality without extra inference time.
FAQ
What is the main problem addressed by the correction step in diffusion models?
The main problem is error accumulation due to exposure bias, where the denoiser's inputs during inference differ from those during training, leading to degraded performance.
How does the correction step work in the proposed method?
The correction step takes the base model's predictions as input and is trained against the same ground-truth target, allowing iterative refinement of denoising steps to prevent early artifacts.
What are the benefits of using the correction step with diffusion models?
The correction step improves sample quality and diversity, achieves performance comparable to classifier-free guidance (CFG), and allows for distillation to retain quality gains without additional inference costs.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by arXiv - cs.AI on Oct 14, 2026.
Comments
Sign in to join the discussion.