Response Times and Self-Reporting: Response Patterns Across Countries and World Regions Using Data From a Large Scale Computer-Based Assessment

Understanding reporting behavior in questionnaires is a key issue in enhancing cross-national data comparability and policy decisions. Computers help improve the analysis of careless or insufficient effort (C/IE) responding by logging response times and other response behavior, ensuring data quality. We introduce a response-time based approach, built on an analysis of the relationship between a survey item and a related external variable, to cross-national research. Using PISA 2015 data from 58 countries/economies, we analyze patterns of correlations between the enjoyment of science and science test scores across response time. We focus on C/IE responding towards the beginning of the response time spectrum. Results indicate rather diligent responding in Eastern Asia and a part of Northern Europe. Yet in other regions (e.g., part of Latin America and the Caribbean, and Eastern Europe), C/IE responding might be distorting the data. We provide other researchers with information regarding when and to what extent C/IE responding can occur across countries. We enhance the understanding of heterogeneity in reporting behavior across countries.