Web scraped data in consumer price indices

Abstract

The changes in the consumer behavior, a consumer prefers to purchase some products through internet, calls for revisiting of the traditional data collection and integration of the automated online data collection, also called data web scraping, of products’ prices for official price statistics. In this paper, we demonstrate different methods for aggregating daily web scraped price data to determine monthly prices that are normally used by National Statistical Institutes in the estimation of consumer price indices. Moreover, various economic approaches for estimating indices, which capture price dynamics of product items such as replacements and missing prices, are presented and applied to the observed web scraped data.