Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection

Abstract

Adversarial training methods commonly generate initial perturbations that are independent across epochs, and obtain subsequent adversarial training samples without selection. Consequently, such methods may limit thorough probing of the vicinity around the original samples and possibly lead to unnecessary or even detrimental training. In this work, a simple yet effective training framework, called Batch-in-Batch (BB), is proposed to refine adversarial training from these two perspectives. The framework jointly generates m sets of initial perturbations for each original sample, seeking to provide high quality adversarial samples by fully exploring the vicinity. Then, it incorporates a sample selection procedure to prioritize training on higher-quality adversarial samples. Through extensive experiments on three benchmark datasets with two network architectures in both single-step (Noise-Fast Gradient Sign Method, N-FGSM) and multi-step (Projected Gradient Descent, PGD) scenarios, models trained within the BB framework consistently demonstrate superior adversarial accuracy across various adversarial settings, notably achieving an improvement of more than 13% on the SVHN dataset with an attack radius of 8/255 compared to N-FGSM. The analysis further demonstrates the efficiency and mechanisms of the proposed initial perturbation design and sample selection strategies. Finally, results concerning training time indicate that the BB framework is computational-effective, even with a relatively large m.