Differentially Private Image Classification by Learning Priors from Random Processes [arxiv]

Xinyu Tang*, Ashwinee Panda*, Vikash Sehwag, Prateek Mittal (*: equal contribution)

This version of code has been tested with Python 3.9.16 and PyTorch 1.12.1.

Set up environment via pip and anaconda

conda create -n "dprandp" python=3.9 
conda activate dprandp
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r ./requirements.txt

Note that we made small edits in opacus/data_loader.py to make sure the expected batch size of poisson sampling is the same as the hyperparameters in the code instead of approximated by 1 / len(data_loader) to be consistant with privacy accounting.

sample_rate=data_loader.batch_size/len(data_loader.dataset), #instead of 1 / len(data_loader)

We also provide the corresponding data_loader.py we used in ./opacus_utils for reference.

cp ./opacus_utils/data_loader.py $YOUR_opacus_lib_path/data_loder.py

We also make change accordingly in ./tan/src/opacus_augmented/privacy_engine_augmented.py (line 386-387).

Please make change accordingly for --max_physical_batch_size to fit your GPU memory.

For Phase I, please refer to folder ./learning_with_noise.

For Phase II and Phase III, please refer to folder ./tan.

For Table 4 in paper for DP-RandP w/o Phase III, please refer to foloder ./linear_prob.

@article{tang2023dprandp,
      title={Differentially Private Image Classification by Learning Priors from Random Processes}, 
      author={Xinyu Tang and Ashwinee Panda and Vikash Sehwag and Prateek Mittal},
      journal={arXiv preprint arXiv:2306.06076},
      year={2023}
}

This code has been built upon the code accompanying the papers

"Learning to See by Looking at Noise" [code].

"TAN Without a Burn: Scaling Laws of DP-SGD" [code].

The hyperparameter setting of the code mostly follows the papers

"Unlocking High-Accuracy Differentially Private Image Classification through Scale" [code].

"A New Linear Scaling Rule for Differentially Private Hyperparameter Optimization" [code].

If anything is unclear, please open an issue or contact Xinyu Tang ([email protected]).