Causal softmax for out-of-distribution generalization

Most supervised learning algorithms follow the Empirical Risk Minimization (ERM) principle, which assumes that training and test data are independently and identically distributed (IID). However, when faced with out-of-distribution (OOD) data, these models may inadvertently learn spurious correlations introduced by confounding factors in the training data. This can result in suboptimal performance on the test data, ultimately compromising the model's practical reliability. In this paper, we propose a novel causal softmax algorithm to address this challenge. First, we introduce a method to define causal and non-causal features in image classification tasks. Then, by employing a causal feature discovery module, we analyze high-level semantic activations extracted by the feature extraction network to distinguish between causal and non-causal features. Subsequently, we penalize the weights associated with non-causal features in the classifier to mitigate their influence, enabling the classifier to establish associations solely based on causal features and labels. Extensive experiments on public datasets like NICO and ImageNet-9 demonstrate the superiority of our approach.