Objective: The purpose of this study was to evaluate how performance is defined, conceptualized, and measured in mental health and addiction service systems around the world.
Method: We conducted a systematic scoping review of English-language scientific and gray literature published from 2005 to 2015. Eligible documents (n = 222) described performance measurement systems and outlined the theory or empirical evidence for indicators. We used a structured approach for data extraction and descriptive and thematic analysis, supplemented with stakeholder consultation.
Results: We identified seven themes in the literature: similarity in performance domains across frameworks; the ability of frameworks to inform care quality at client, program/facility, and system levels; the predominance of indicators of process and outcome, over structure; the lack of evidence on the links between domains and/or indicators; common, but limited, evaluation of family/caregiver involvement; equity as a cross-cutting domain of performance; and limited attention to performance measurement in peer support services.
Conclusions: The literature on performance measurement in mental health and addictions services is vast, and a wide variety of indicators is available to those designing a measurement system. Evaluations of commonly used performance indicators have yielded mixed evidence on their ability to discriminate high- and low-performing service providers, and their sensitivity to changes in policies and practices. As performance measurement efforts grow in scope and complexity, work will be needed to ensure that indicators are fair, appropriate, and suited to support quality improvement in services of different types.
Objective: The purpose of this study was to evaluate how performance is defined, conceptualized, and measured in mental health and addiction service systems around the world.
Method: We conducted a systematic scoping review of English-language scientific and gray literature published from 2005 to 2015. Eligible documents (n = 222) described performance measurement systems and outlined the theory or empirical evidence for indicators. We used a structured approach for data extraction and descriptive and thematic analysis, supplemented with stakeholder consultation.
Results: We identified seven themes in the literature: similarity in performance domains across frameworks; the ability of frameworks to inform care quality at client, program/facility, and system levels; the predominance of indicators of process and outcome, over structure; the lack of evidence on the links between domains and/or indicators; common, but limited, evaluation of family/caregiver involvement; equity as a cross-cutting domain of performance; and limited attention to performance measurement in peer support services.
Conclusions: The literature on performance measurement in mental health and addictions services is vast, and a wide variety of indicators is available to those designing a measurement system. Evaluations of commonly used performance indicators have yielded mixed evidence on their ability to discriminate high- and low-performing service providers, and their sensitivity to changes in policies and practices. As performance measurement efforts grow in scope and complexity, work will be needed to ensure that indicators are fair, appropriate, and suited to support quality improvement in services of different types.
Objective: Substance use services and supports have traditionally been funded without the benefit of a comprehensive, quantitative planning model closely aligned with population needs. This article describes the methodology used to develop and refine key features of such a model, gives an overview of the resulting Canadian prototype, and offers examples and lessons learned in pilot work.
Method: The need for treatment was defined according to five categories of problem severity derived from national survey data and anticipated levels of help-seeking estimated from a narrative synthesis of international literature. A pan-Canadian Delphi procedure was used to allocate this help-seeking population across an agreed-upon set of treatment service categories, which included three levels each of withdrawal management, community, and residential treatment services. Projections of need and required service capacity for Canadian health planning regions were derived using synthetic estimation by age and gender. The model and gap analyses were piloted in nine regions.
Results: National distribution of need was estimated as Tier 1: 80.7%; Tier 2: 10.4%; Tier 3: 6.1%; Tier 4: 2.6%; and Tier 5: 0.2%. Pilot work of the full estimation protocol, including gap analysis, showed the results triangulated with other indicators of need and were useful for local planning.
Conclusions: Lessons learned from pilot testing were identified, including challenges with the model itself and those associated with its implementation. The process of estimation developed in this Canadian prototype, and the specifics of the model itself, can be adapted to other jurisdictions and contexts.