Escape From Models: How Science Publishing Went Sour (Presentation Slides)

In recent years, many published findings in science could not be reproduced. These publications turned out to be low quality despite having good p-values. Several measures to replace p-values have been proposed, including an incredibility index (Schimmack, 2012), meta-regression (Stanley & Doucouliagos, 2014). However, the real root cause of the souring of p-values is publication pressure. In that case, the newer measures will also eventually also go sour under publicaiton pressure. The problem of measuring science quality is actually just a specific case of a more general observation: Measures of behavior go sour (seem to work but actually not work) once pressure is placed on them to control people. So how do we fix science publication? There are two choices:

1. Reduce the pressure on scientists to publish. Produce fewer social science PhDs and make alternatives to tenure-track work more palatable (not adjunct hell).

2. Learn more about other systems with escape from measures. I discuss co-evolutionary hotspots and how spammers escape spam detectors.