This repository is the implementation of the paper Adversarial Example Detection for DNN Models: A Review and Experimental Comparison

@article{aldahdooh2022adversarial,
      title={Adversarial Example Detection for DNN Models: A Review and Experimental Comparison}, 
      author={Ahmed Aldahdooh and Wassim Hamidouche and Sid Ahmed Fezza and Olivier Deforges},
      journal={Artificial Intelligence Review},
      year={2022},
      publisher={Springer}
}

Publicly available codes for the detectors that are use in this work are customized and the original repositories/papers can be found in:

1. KD_BU
2. LID
3. NSS
4. FS
5. MagNet
6. DNR
7. SFAD
8. NIC

1. Tested on Python 3.8
2. Keras 2.3.1
3. Tensorflow 2.2
4. thundersvm for GPU-based SVM. Link

Open setup_paths.py and set the paths and other detector-related settings.

Run train_cnn_base.py -d=<dataset> -e=<nb_epochs> -b=<batch_size>. Currently, the supported datatsets are mnist, cifar, svhn, and tiny. cifar is for CIFAR-10 dataset, and tiny is for Tiny-ImageNet.

Run generate_adv.py -d=<dataset>. We use ART library. You can easily add, update, or remove adversarial attacks. DON'T forget to update the attacks arrays in setup_paths.py

To run all the detector, just execute run_detectors.py. Each detector will generate csv file that contains detection accuracy, false/true positive rate, and AUC for successful, fail, and both (all) adversarial examples.

To run a specific detector, execute detect_<detector_name>.py -d=<dataset> -[other detector-related arguments]

Execute collect_results_from_csv.py to aggregate and summarize all the results in one csv file per dataset.

Create a folder for your detector and put inside it your code and utils code.

Create a Python file detect_<detector_name>.py to run the detector for a specific dataset. We recommend to follow the code style we follow in the other detectors files. DON'T forget to add lines to generate the csv file for the result as we did in the detect_<detector_name>.py.

Add detector-related arguments in setup_paths.py.