Testing the validity of the Networked Hazard Analysis and Risk Management System (Net-HARMS)

Abstract

Testing the validity of newly developed methods is a critical component of human factors and ergonomics (HFE) practice. The Networked Hazard Analysis and Risk Management System (Net-HARMS) is a recently developed systems thinking-based risk assessment method which supports the identification of task and emergent risks across overall work systems. This article reports on a validity study of the Net-HARMS method in which outputs were compared to an expert analysis developed by the first two authors of this paper, with review by subject matter experts. The findings show that individual participant performance was poor for both groups yet when both group's analyses were pooled, validity significantly improved. Further, a subject matter expert analysis of the false alarms identified by participants showed that they may in fact represent credible risks. It is concluded that the Net-HARMS method achieved high levels of validity when participants analyses are pooled. The implications for risk assessment and the validity of HFE methods are discussed.