Measuring Patient Experience in Rheumatology Outpatient Care

In the subspecialty of Rheumatology, we are increasingly compelled to pursue and practice comprehensive healthcare, reaching beyond the foundation blocks of excellent patient-clinician interactions and gold-standard therapeutics. Nearly 10 years have passed since the World Health Organization (WHO) signaled the need for a fundamental shift towards people-centered approaches to healthcare.

To this end, objectively evaluating outcomes through collection of Patient Reported Outcome Measure (PROM) data is now integral to appraising treatment response in clinical practice, clinical trial endpoints, and standard setting. Readers will be familiar with PROMs, which collect data on symptoms, disease activity, and functional status. Examples of PROM use in rheumatology include disease-specific instruments such as the Routine Assessment of Patient Index Data (RAPID) 2–5, the Bath Ankylosing Spondylitis Disease Activity Index (BASDAI), and generic instruments such as the Health Assessment Questionnaire (HAQ) and the 36-question Short Form Health Survey (SF-36).

Conceptually similar to PROMs, but distinct, are Patient Reported Experience Measures (PREMs), which collect information directly from patients on the experience of receiving healthcare. Specifically, PREMs address the process, content, and impact of care, in both functional and relational terms [1-6]. Functional domains captured by PREMs include logistic appointment considerations, access to information about conditions and treatments, and access to inter-visit care. Relational domains captured by PREMs include individualization of care, respectful care delivery and patient empowerment. Using PREMs alongside PROMs can give comprehensive insights into how healthcare is delivered and can be improved. However, routine PREM capture in rheumatology is, to date, only sporadically described. Other contexts where PREMs are used in Australia include mental health, general practice, surgical specialties, renal medicine, and palliative care.

Endeavoring to bring a rheumatology-specific PREM to Australian patients has been the focus of a body of work over the past 3 years. Two key beliefs underpin the rationale for this work: the first is that experience-related data are essential to understanding whether healthcare services are truly patient-centered. The second is that patients are uniquely positioned to measure and evaluate the care they have received.

We conducted a systematic literature review in 2021 to ascertain the availability and frequency of use of rheumatology-specific PREMs worldwide. The review identified five validated PREMs for use with Rheumatology outpatients, with broad heterogeneity of study design and methodology and significant shortcomings in the rigor of content validity testing [7]. Additionally, no clear consensus on bringing such instruments to routine clinical implementation was evident in published literature. This represents a knowledge gap that subsequent work has endeavored to address. One of the instruments identified in clinical use was the Commissioning for Quality in Rheumatoid Arthritis (RA)-RA-PREM (CQRA-RA-PREM) [8]. This instrument was developed in the United Kingdom (UK) using the National Health Service (NHS) Patient Experience Framework (NPEF), comprising eight domains of patient experience: “respect for patient-centered values, preferences, and needs”, “coordination and integration of care”, “information, communication and education”, “physical comfort”, “emotional support”, “involvement of family and friends”, “care coordination”, and “access to care” [8, 9].

Since its development, CQRA-RA-PREM has been used in the UK to evaluate patient experience in RA, subsequently validated for use with other rheumatic conditions, and translated and validated into Dutch and Portuguese [8, 10-12]. Data derived from using CQRA-RA-PREM have been incorporated in the National Clinical Audit for Rheumatoid and Early Inflammatory Arthritis in the United Kingdom and rheumatology clinical registries in the Netherlands [8, 11, 13, 14].

Based on the appraisal of instrument development processes and published examples of external validation in subsequent generation cohorts, the CQRA-RA-PREM was deemed suitable for modification and validation with the Australian population. Justifying the use of an instrument with a new population requires consideration of instrument validity and reliability, as well as the clinical context in which it will be used. The validity of a measurement instrument is defined as the degree to which it measures the construct it intends to measure. Reliability refers to the extent to which an instrument produces consistent scores under different conditions, between different responders, and on other occasions [15-17]. Specific validation components, including face and cross-cultural validity, are essential when an instrument is modified for language or cross-cultural use. Validation should be undertaken with a sample representative of the target population to ensure relevant and meaningful data. Face validity testing of an instrument in lay terms means checking that a proposed instrument will be relevant, logical, and sufficiently inclusive to cover the issues of interest for the group that will be asked to use it. To do this with the CQRA-RA-PREM, we undertook focus groups and individual face validity interviews to establish the concerns, priorities, and experiential domains of interest for Australian rheumatology patients and healthcare professionals [18]. This work demonstrated that Australian patients prioritized coordinated, respectful and individualized care and adequacy of information transfer; healthcare professionals identified information sharing, comprehensive care and individualization as priorities for patient-centered care. Importantly, these concepts are reflected in the Domains covered by the CQRA-RA-PREM. Feedback from the face validity process with Australian stakeholders also informed minor modifications to the instrument such that it could be presented as an Australian version, the CQRA-PREM-AU. Thereafter, this modified instrument was deployed in a national patient database, the Australian Rheumatology Association Database (ARAD), to undertake the psychometric testing required to ensure it remained satisfactory for use after modification, in this patient population. Properties of structural validity, internal consistency, divergent validity and measurement invariance, and interpretability (with the definition of the minimum significant change for this instrument) were confirmed [19]. Additionally, a numerical overall summary score was tested and demonstrated to be statistically sound, which adds particular value to the practical use of the CQRA-PREM-AU in longitudinal monitoring and comparison of care.

Following this Australian validation, the CQRA-PREM-AU was deployed in an operational health service to ascertain the current care experience of patients in this region and demonstrate the feasibility of practical implementation of its use in practice. Key findings from the implementation process were the identification of patient-related factors that correlated with better care experience (such as older age and patients in contact with a specialty rheumatology nurse) and worse care experience (including diagnoses of SLE, “other” and “don't know”, and worse Patient Global Assessment). We also evaluated the extent to which patients attending our service perceive their experience of key domains such as respectful treatment, information sharing, shared decision-making, flare support, and integrated multidisciplinary care. These findings exemplify the learning gained from the routine collection of experience-related data. We encountered few barriers to this implementation exercise, with a strong survey response rate suggesting acceptability to patients, and minimal additional cost beyond person-hours for project building and analysis. In the future, we anticipate that instrument deployment could occur within existing functionality of the Electronic Medical Record, further optimizing the ease and efficiency of this process.

Locally, we have demonstrated the feasibility of deploying the instrument across a health network, and highly valuable information has been gained from respondents. The next phase of this process will be change implementation based on findings from the first measurement phase. Specifically, we will use these data to bid for expansion of access to specialty rheumatology nursing and self-management programs, highlight pathways for access to flare care and inter-visit care, and focus on patients' experiences in certain diagnostic groups.

Potential uses of the CQRA-PREM-AU, particularly the summary score, include performing analyses and comparisons between different patient groups, and summarizing and reviewing patient experience within a health service more broadly. While PROM data are evaluated individually to understand disease activity and treatment response, PREM data are most useful in aggregate form. Aggregate experience scores may build into healthcare professional and patients' understanding of how services are meeting benchmarks for care, and be useful for representing patient experience during lobbying for resource, funding and service improvements. Other applications of CQRA-PREM-AU may include its use to illustrate continuing professional development requirements, and in research settings to capture issues relevant to specific patient groups.

The CQRA-PREM-AU is quick to use, with an average completion time of under 5 min, easy to score, and can be represented using an overall summary score. Currently, the CQRA-PREM-AU is intended for routine periodic use in rheumatology outpatient services, primarily to identify areas for improvement to support personalized care, service development, and engagement with patients. We encourage readers to consider the benefits of collecting experience data alongside PROM use in their clinical practice, and welcome future endeavors to expand the use of PREMs in rheumatology to benefit our patients regionally and worldwide.

All authors contributed to design and conception of the work described. Madeleine J. Bryant was responsible for preparation of the manuscript. Rachel J. Black and Catherine L. Hill participated in critical review of themanuscript, with approval of the final version to be published.

Work described in this editorial received ethics approval from the following bodies: (1) Australian Rheumatology Australia Database: Monash University. (2) Central Adelaide Local Health Network (CALHN) Human Research Ethics Committee: project reference numbers 13846 [18], 16836 [19], 20251. The described systematic literature review was prospectively registered with PROSPERO (Registration CRD42021233819).

The authors declare no conflicts of interest.