A Simulation Approach to Designing Digital Matching Platforms

Digital matching marketplaces are characterized by user heterogeneity, limited capacity, and dynamic market clearing. These features create spillovers between users. For example, an Airbnb listing booked by one guest cannot be booked by another guest for the same night. Spillovers limit the applicability of many experimental and observational methods for evaluating the effects of marketplace policies. In this paper, I show how to use marketplace simulations as an input into the design of user acquisition strategies and ranking algorithms. I calibrate a marketplace simulation using data on searches and transactions from Airbnb and use it to address three topics: the returns to scale in matching, the heterogeneity in returns to user acquisition, and the size of bias in experimental designs. I find that returns to scale are initially increasing due to market thickness effects and then decreasing due to availability frictions in search. Furthermore, heterogeneity in the value of listings to the platform is large - the effect of acquiring 25% more listings on bookings varies between -4.1% and 5.4% depending on the quartile of listing quality. I then measure the extent of bias in experimental treatment effects due to spillovers. The treatment effect of a better ranking algorithm on conversion rates is overstated by 53% when a quarter of users are randomized into treatment.