This is the official PyTorch codes for the paper
PromptRestorer: A Prompting Image Restoration Method with Degradation Perception
Cong Wang, Jinshan Pan, Wei Wang, Jiangxin Dong, Mengzhu Wang, Yakun Ju, Junyang Chen

We show that raw degradation features can effectively guide deep restoration models, providing accurate degradation priors to facilitate better restoration. While networks that do not consider them for restoration forget gradually degradation during the learning process, model capacity is severely hindered. To address this, we propose a Prompting image Restorer, termed as PromptRestorer. Specifically, PromptRestorer contains two branches: a restoration branch and a prompting branch. The former is used to restore images, while the latter perceives degradation priors to prompt the restoration branch with reliable perceived content to guide the restoration process for better recovery. To better perceive the degradation which is extracted by a pre-trained model from given degradation observations, we propose a prompting degradation perception modulator, which adequately considers the characters of the self-attention mechanism and pixel-wise modulation, to better perceive the degradation priors from global and local perspectives. To control the propagation of the perceived content for the restoration branch, we propose gated degradation perception propagation, enabling the restoration branch to adaptively learn more useful features for better recovery. Extensive experimental results show that our PromptRestorer achieves state-of-the-art results on 4 image restoration tasks, including image deraining, deblurring, dehazing, and desnowing.

(a) compares different restoration frameworks. Unlike existing approaches that are built within the architectures such as Cases 1-2, which are unable to memorize the degradation well during the learning process, we propose a prompting method (Case 3) that directly exploits raw degradation features extracted by a pre-trained model from the given degradation observations to guide restoration. In (b), we observe that both Cases 1-2 outperform our method in early iterations, as they effectively memorize degraded information. However, both Cases 1-2 experience degradation vanishing with further iterations, while our prompting method persists in guiding the restoration network with accurate degradation priors, accordingly producing better restoration quality. In (c), visual performance demonstrates that our prompting method recovers sharper images.

Overall pipeline of our PromptRestorer. PromptRestorer contains two branches: (a) the restoration branch and (b) the prompting branch. The restoration branch is used to restore images, where each block (c) in CGT is prompted by the prompting branch. The prompting branch first generates precise degradation features extracted by a pre-trained model from degradation observations, then these features prompt the restoration branch to facilitate better restoration via PromptDPM (d).

To better perceive the degradation to prompt the restoration network with more reliable perceived content from the degradation priors, we propose the PromptDPM. The PromptDPM consists of 1) Global Prompting Perceptor (G2P) and 2) Local Prompting Perceptor (L2P) to respectively perceive the degradation from global and local perspectives, enabling to generate more useful content to guide the restoration branch.

• Ubuntu >= 18.04
• CUDA >= 11.0
• Other required packages in requirements.txt

# git clone this repository
git clone https://github.com/supersupercong/PromptRestorer.git
cd UHDformer 

# create new anaconda env
conda create -n promptrestorer python=3.8
source activate promptrestorer 

# install python dependencies
pip3 install -r requirements.txt
python setup.py develop

bash train.sh

bash test.sh

@inproceedings{PromptRestorer,
  author       = {Cong Wang and
                  Jinshan Pan and
                  Wei Wang and
                  Jiangxin Dong and
                  Mengzhu Wang and
                  Yakun Ju and
                  Junyang Chen},
  title        = {PromptRestorer: {A} Prompting Image Restoration Method with Degradation
                  Perception},
  booktitle    = {NeurIPS},
  year         = {2023},
}

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Contact: Cong Wang [[email protected]]

This project is based on FeMaSR.