Accurate and Robust Object Detection via Selective Adversarial Learning With Constraints

ConvNet-based object detection networks have achieved outstanding performance on clean images. However, many works have shown that these detectors perform poorly on corrupted images caused by noises, blurs, poor weather conditions and so on. With the development of security-sensitive applications, the detector’s practicability has raised increasing concerns. Existing approaches improve detector robustness via extra operations (image restoration or training on extra labeled data) or by applying adversarial training at the expense of performance degradation on clean images. In this paper, we present Selective Adversarial Learning with Constraints (SALC) as a universal detector training approach to simultaneously improve the detector’s precision and robustness. We first propose a unified formulation of adversarial samples for multitask adversarial learning, which significantly diversifies the obtained adversarial samples when integrated into the adversarial training of the detector. Next, we examine our findings on model bias against adversarial attacks of different strengths and differences in Batch Normalization (BN) statistics among clean images and different adversarial samples. On this basis, we propose a batch local comparison strategy with two BN branches to balance the detector’s accuracy and robustness. Furthermore, to avoid performance degradation caused by overwhelming subtask losses, we leverage task-aware ratio thresholds to control the influence of learning in each subtask. The proposed approach can be applied to various detectors without any extra labeled data, inference time costs, or model parameters. Extensive experiments show that our SALC achieves state-of-the-art results on both clean benchmarks (Pascal VOC and MS-COCO) and corruption benchmarks (Pascal VOC-C and MS-COCO-C).