(IJCNN2026) SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images

https://arxiv.org/abs/2604.09436

Diffusion models trained on noisy datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from diffusion models trained on noisy images. Leveraging the spectral bias of diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.

Transcript

1. SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images Kyushu University, Fukuoka, Japan ◦Shumpei Takezaki Yuta Matsuzaki Seiichi Uchida
2. • Diffusion models trained on noisy data can generate noisy samples Background: Degraded generation from noisy training data 1 Noisy training data → Noisy generated images [1] Ho+, NeurIPS, 2020.,[2] Daras+, NeurIPS, 2023. noisy and clean Training images Generated images noisy and clean Diffusion models[1]
3. • Our approach regenerates noisy images into cleaner images at generation. Goal: Clean generation from noisy data 2 Noisy training data → Clen generated images noisy and clean Training images Generated images Clean Proposed
4. Key Insight: Clean vs. noisy high-frequencies 3 Low High Clean image Noise Noisy image
5. Key Insight: Clean vs. noisy high-frequencies 4 Clean low-frequencies and noisy high-frequencies Clean image Noise Noisy image Low High Low High 𝑓 = High Noisy 𝑓 = Low Clean
6. Low High Our approach: Regenerate high-frequencies 5 Keep clean low-frequencies and regenerate noisy high-frequencies Clean image Noise Noisy image Low High 𝑓 = High Noisy 𝑓 = Low Clean Keep Regenerate
7. SCoRe: Spectral Cut-off Regeneration 6 Cut off noisy high-frequencies, then regenerate them Regenerate Cutoff Generated Image Cutoff Image Regenerated Image Low High No need! Cutoff high-freq. Regenerate high-freq. Keep low-freq. Training-free!
8. • Pixel space: Add noise and remove noise • Frequency: Corrupt high-freq. and restore high-freq.[3] Diffusion models restore high-frequencies 7 [3] S. Dieleman,Online, 2024. Pixel Freq. Restore
9. • SDEdit[4] regenerates an input using a diffusion model • Keep low-freq. and regenerate high-freq. SDEdit regenerates high-frequencies. 8 [4] C. Meng+,ICLR, 2022. Keep low-freq. Pixel Freq. Regenerate high-freq.
10. Pipeline of SCoRe: Diffusion models 9 𝑡 = 𝑇 𝒙𝒕 𝒙𝑻 Noisy Image 𝒙𝟎 Clean Image 𝑡 = 0
11. Pipeline of SCoRe: Cut-off high-frequencies 10 𝑡 = 𝑇 𝒙𝒕 𝒙𝑻 Noisy Image 𝒙𝟎 Clean Image 𝑡 = 0 Cutoff Image
12. Pipeline of SCoRe: Regeneration by SDEdit 11 𝑡 = 𝑇 𝒙𝒕 𝒙𝑻 Noisy Image 𝒙𝟎 Clean Image 𝑡 = 0 Cutoff Image 𝒙𝒕′ Without additional training
13. • Cutoff removes noisy high-freq. information. • Clean low-freq. cues guide high-freq. regeneration. Why can SCoRe generate clean high-freq. 12 Power 𝑓cutoff 𝑓 Clean low freq. Add noise Power 𝑓cutoff 𝑓 Regenerate
14. • Dataset: CIFAR10 (50,000 images, 32×32 pixels) • Noise setting: Gaussian • Comparison methods • Denoising filter: Bilateral[5] • Denoising model: Noise2Void[6] Experiment 1: Artificial noise settings 13 We also used Poison, Mix noise Please check the paper!! Clean 50,000 images 5000 clean images 45,000 noisy images Noisy Noise [5] Tomasi+, ICCV, 1998.,[6] Krull+, CVPR, 2019.
15. Experiment 1: Artificial noise settings 14 Bilateral N2V Proposed Diffusion model 9.8 32.3 51.7 FID↓=132.9 SCoRe generates cleaner images and achieves the best FID.
16. • Frequency analysis Experiment 1: Artificial noise settings 15 Proposed Training image Clean image SCoRe shifts the spectrum closer to clean images.
17. • Dataset: SIDD[7] (8,160 images, 128×128 pixels) • Noise settings: • Sensor noise, readout noise, etc. • Unknown statistics of noise • Comparison methods: • Same as Experiment 1 Experiment 2: Real-world noise settings 16 [7] Abdelhamed +, CVPR, 2018.
18. Experiment 2: Real-world noise settings 17 N2V Diffusion model Bilateral Proposed 16.8 27.0 30.9 FID↓=30.1 SCoRe generates cleaner images under real-world noise.
19. • Purpose: Clean generation from noisy training data • Method: Spectral cutoff + high-freq. regeneration • Results: Best FID on artificial and real-world noise Summary 18 SCoRe enables training-free clean generation via spectral regeneration