Code for the paper Autoregressive Perturbations for Data Poisoning by Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein, David W. Jacobs. (Accepted to NeurIPS 2022)
We release our AR poisons as Zip files containing PNG images for easy viewing via Google Drive. This includes the following poisons:
| Poison | Zip Filename | Bound | Test Set Accuracy |
|---|---|---|---|
| CIFAR-10 AR Poison | ar-cifar-10.zip | ε=1 in L2 | 11.75% |
| CIFAR-100 AR Poison | ar-cifar-100.zip | ε=1 in L2 | 4.24% |
| SVHN AR Poison | ar-svhn.zip | ε=1 in L2 | 6.77% |
| STL-10 AR Poison | ar-stl.zip | ε=3 in L2 | 11.65% |
| CIFAR-10 AR Poison | linf-ar-cifar-10.zip | ε=8/255 in Linf | 20.49% |
After unzipping, these poisons can be loaded using AdversarialPoison, a subclass of torch.utils.data.Dataset. In the table, test set accuracy refers to the test set performance of a ResNet-18 which trains on the poison and is evaluated on the corresponding clean test set. A model which trains on our AR poisons is unable to generalize to the (clean) test set.
Disclaimer: While we focus on poisoning with an L2-norm bound on perturbations, we release a sample CIFAR-10 Linf AR poison. Note that the AR coeffients used in this poison are suboptimal, and we recommend reporting L2 threat model results. For details on AR poisoning in Linf, please see Appendix A.6 of our paper.
- Create a conda environment with necessary dependencies:
conda create --name arp python=3.6
conda activate arp
pip install -r requirements.txt
- Modify paths in config/base.yaml to point to your choice of dataset and storage. This config is used in train.py.
See notebooks/Generate-AR-Perturbations-from-Coefficients.ipynb for an example of how to load AR coefficients and generate an AR perturbation of a given size and norm.
In summary, after loading some AR coefficients, we can call the generate function of ARProcessPerturb3Channel:
# Load coefficients
coefficients = torch.load(os.path.join(repo_dir, 'params-classes-10-mr-10.pt'))
# Use first set of coefficients, for example
ar = ARProcessPerturb3Channel(b=coefficients[0])
# Generate a size (3, 32, 32) perturbation, after cropping a larger (36, 36) perturbation
perturbation, _ = ar.generate(size=(36,36), eps=1.0, crop=4, p=2)
The resulting perturbation can then be additively applied directly to an image of shape (3,32,32) because the perturbation is of size 1.0 in L2.
To find a set of 10 AR processes, run:
python autoregressive_param_finder.py --total=10 --required_nm_response=10 --gen_norm_upper_bound=50
This command will save a file named params-classes-10-mr-10.pt using torch.save. The format will be identical to that of RANDOM_3C_AR_PARAMS_RNMR_10 within autoregressive_params.py, a list of torch.tensor. Additional information can be found in Appendix A.3.
Before creating a poison using our script, update CIFAR_PATH (and other paths, as required) in create_ar_poisons_3channel.py with the location of your CIFAR data. Then, you can create an AR CIFAR-10 poison by calling:
python create_ar_poisons_3channel.py ${YOUR_POISON_NAME} --epsilon 1.0 --p_norm 2
By default, the code uses params from autoregressive_params.py, but you can change this behavior if you like. The script also has support for SVHN, STL, and CIFAR-100.
We provide a number of models, borrowed from the pytorch-cifar repo. To train a ResNet18 on clean CIFAR-10:
python train.py misc.project_name=${PROJECT_NAME} misc.run_name=${RUN_NAME} train.batch_size=128 train.augmentations_key="none"
To train a model on an AR CIFAR-10 poison:
python train.py misc.project_name=${PROJECT_NAME} misc.run_name=${RUN_NAME} train.adversarial_poison_path=${YOUR_POISON_PATH} train.batch_size=128 train.augmentations_key=${AUG}
Note that in this command, we specify train.adversarial_poison_path to override the config within config/base.yaml, and load a poison. You can set AUG to either "none", "cutout", "cutmix" or "mixup". Be sure to update other configs such as num_workers as necessary.
This training script uses the WandbLogger from PyTorch Lightning, so if you use Weights and Biases, you can use their online portal to analyze training curves.
To demonstrate the simplicity and separability of AR perturbations, we construct a simple CNN which can perfectly classify AR perturbations in autoregressive_perfect_model.py.
Check out our demo notebook in notebooks/Demo-of-AR-Perfect-Model.ipynb. The demo notebook demonstrates how we can take some AR coefficients, generate perturbations, and use PerfectARModel (initialized with the same AR coefficients) to perfectly classify the novel, generated AR perturbations. PerfectARModel is not trained in any way; it uses manually-specified AR filters (consisting of AR process coefficients) for a single convolution layer. More information can be found in Appendix A.2 of the paper.
Note that the code for PerfectARModel was from an earlier version of our repo where one AR process was responsible for each of the three RGB channels (as opposed to using a different set of coefficients for each of 3 channels). Early in our work, we used terms from convergent series, and manually specified them in ALL_2x2_AR_PARAMS.
If you find this work useful for your research, please cite our paper:
@inproceedings{sandovalsegura2022autoregressive,
author = {Sandoval-Segura, Pedro and Singla, Vasu and Geiping, Jonas and Goldblum, Micah and Goldstein, Tom and Jacobs, David},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {27374--27386},
publisher = {Curran Associates, Inc.},
title = {Autoregressive Perturbations for Data Poisoning},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/af66ac99716a64476c07ae8b089d59f8-Paper-Conference.pdf},
volume = {35},
year = {2022}
}