Using Machine Learning to Catch Bogus Firms

We investigate the use of a machine learning (ML) algorithm to identify fraudulent non-existent firms that are used for tax evasion. Using a rich dataset of tax returns in an Indian state over several years, we train an ML-based model to predict fraudulent firms. We then use the model predictions to carry out field inspections of firms identified as suspicious by the ML tool. We find that the ML model is accurate in both simulated and field settings in identifying non-existent firms. Withholding a randomly selected group of firms from inspection, we estimate the causal impact of ML driven inspections. Despite the strong predictive performance, our model driven inspections do not yield a significant increase in enforcement as evidenced by the cancellation of fraudulent firm registrations and tax recovery. We provide two explanations for this discrepancy based on a close analysis of the tax department’s operating protocols: overfitting to proxy-labels, and institutional friction in integrating the model into existing administrative systems. Our study serves as a cautionary tale for the application of machine learning in public policy contexts and of relying solely on test set performance as an effectiveness indicator. Field evaluations are critical in assessing the real-world impact of predictive models.