A control-function correction for endogeneity in random coefficients models: The case of choice-based recommender systems

Abstract

Applications of discrete choice models in personalization are becoming increasingly popular among researchers and practitioners. However, in such systems, when users are presented with successive menus (or choice situations), the alternatives and attributes in each menu depend on the choices made by the user in the previous menus. This gives rise to endogeneity which can result in inconsistent estimates. Our companion paper, Danaf et al. (2020), showed that the estimates are only consistent when the entire choice history of each user is included in estimation. However, this might not be feasible because of computational constraints or data availability. In this paper, we present a control-function (CF) correction for the cases where the choice history cannot be included in estimation. Our method uses the attributes of non-personalized attributes as instruments, and applies the CF correction by including interactions between the explanatory variables and the first stage residuals. Estimation can be done either sequentially or simultaneously, however, the latter is more efficient (if the model reflects the true data generating process). This method is able to recover the population means of the distributed coefficients, especially with a long choice history. The variances are underestimated, because part of the inter-consumer variability is explained by the residuals, which are included in the systematic utility. However, the population variances can be computed from the estimation results. The modified utility equations (which include the residuals) can be used in forecasting and model application, and provide superior fit and predictions.