- Huggingface diffusers flavor Diffusion Posterior Sampling algorithms
- An alternative to Stability-AI/stablediffusion flavor of Diffusion Posterior Sampling
- Everything in a clean Pipeline, avoid ugly hack of Stability-AI/stablediffusion code
- Faster inference with diffusers optimized pipeline and multiple GPU support
- Seamless use all Stable Diffusion models and solvers available in Huggingface
- DPS
- Diffusion Posterior Sampling for General Noisy Inverse Problems, https://arxiv.org/abs/2209.14687
- Code: https://github.com/DPS2022/diffusion-posterior-sampling
- FreeDOM
- Training-Free Energy-Guided Conditional Diffusion Model, https://arxiv.org/abs/2303.09833
- Code: https://github.com/vvictoryuki/FreeDoM
- DSG
- Guidance with Spherical Gaussian Constraint for Conditional Diffusion, https://arxiv.org/abs/2402.03201
- Code: https://github.com/LingxiaoYang2023/DSG2024
- PSLD:
- Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models, https://arxiv.org/abs/2307.00619
- Code: https://github.com/LituRout/PSLD
- Downsampling
- Gaussian Blurring
- Motion Blurring
- Run DPS for operator srx8, 500 steps
python main.py --data ./example_imgs --out ./example_outputs/dps/srx8 --scale 4.8 --algo dps --operator srx8 --nstep 500 --model stabilityai/stable-diffusion-2-base
- Run DSG for operator gaussian deblur, 500 steps
python main.py --data ./example_imgs --out ./example_outputs/dsg/gdb --scale 0.02 --algo dsg --operator gdb --nstep 500 --model stabilityai/stable-diffusion-2-base
- Run FreeDOM for operator motion deblur, 500 steps
python main.py --data ./example_imgs --out ./example_outputs/fdm/mdb --scale 1.2 --algo fdm --operator mdb --nstep 500 --model stabilityai/stable-diffusion-2-base
- In case you are in China, use an extra environment variable for Huggingface mirror, for example
HF_ENDPOINT=https://hf-mirror.com python main.py --data ./example_imgs --out ./example_outputs/dps --scale 4.8 --algo dps --operator srx8 --nstep 500 --model stabilityai/stable-diffusion-2-base
- Multi GPU
- pretty foolish way but works. No need to worry about randomness as we fix seed.
python main.py --ngpu=2 --rank=0 ... python main.py --ngpu=2 --rank=1 ...
-
Download the first 1000 image of ImageNet validation dataset:
git lfs install git clone https://huggingface.co/datasets/xutongda/ImageNet_val1k_512
-
Current Results for stabilityai/stable-diffusion-2-base and 500 steps
- SRx8 - - Motion Deblur - PSNR LPIPS FID PSNR LPIPS FID DPS 22.33 0.4137 58.48 23.05 0.4267 59.31 PSLD 22.28 0.4163 59.08 23.06 0.4305 60.73 FreeDOM 22.64 0.3961 52.94 23.33 0.4104 54.26 DSG 23.15 0.3912 51.15 23.70 0.3977 52.01
- (have tested with) pytorch==2.1.0, diffusers==0.30.0, transformers==4.37.2
- Usually ok to install manually, but if you are lazy, use:
conda env create --name ddps --file=environments.yml
- We welcome new algorithms / operators / pipelines / test results
- We use ruff format and ruff check for code quality, make sure you pass the check before PR.