Development and testing of the capacity of organisations for system practices scale

Background

Systems change approaches are increasingly adopted in public health to address complex problems. It is important that measures of systems change be developed so that the effects of systems change on health outcomes can be evaluated. Organisational practices are potential levers for systems change. However, robust measures of organisational capacity to engage in these practices are lacking. Informed by the Theory of Systems Change, we developed and tested the Capacity of Organisations for System Practices (COSP) scale. The COSP scale comprises four inter-related system practices within organisations—adaptation, alignment, collaboration and evidence-driven action and learning.

Methods

We applied a three-stage process: (1) Item generation; (2) Scale pre-testing; and (3) Structural analyses. Item response theory tests and semantic review, together with factor analytic techniques, were applied to refine the item set and determine the scale structure.

Results

An initial pool of 97 items was generated and pre-tested with six content experts and four target audience representatives. Modifications resulted in 60 items. In total, 126 participants provided data for the structural analysis. A second-order hierarchical four-factor model fit the data better than the more basic correlated factor model (Δχ² = 1.758, p = .415). The fit indices for the final 31-item model were acceptable (RMSEA = .084, TLI = .819).

Conclusions

The COSP scale is ready for further testing to ensure construct validity, stability and utility.

So What?

Once validated, the Capacity of Organisations for System Practices (COSP) scale has the potential to advance the theory and practice of systems change approaches.