Scores and sores: Exploring patient-reported outcomes for knee evaluation in orthopaedics, sports medicine and rehabilitation

Patient-reported outcome measures (PROMs) are used in healthcare to collect data directly from patients about their health, symptoms, functional status and overall well-being [43, 69].

PROMs were initially developed for research purposes to be used in clinical trials to incorporate the patient's perspective to assess the efficacy of treatments; however, nowadays, they are used both in research and clinical practice. In the research setting, PROMs continue to provide valuable data to studies that aim to explore the efficiency of interventions and assess the patient experience and outcome. In clinical practice, these measures help healthcare providers evaluate the effectiveness of a treatment on an individual patient.

As researchers, we constantly face the challenge of balancing different methods to assess the effectiveness of treatments, which constitutes a base for future decision-making in clinical practice. On the one hand, it is highlighted that the era in which clinicians unilaterally determined the well-being of their patients has long faded. PROMs must call the tune as they play a pivotal role in fostering a patient-centred approach in healthcare by capturing and incorporating patients’ experiences and perspectives [43, 69]. On the other hand, in light of the widespread usage of PROMs as primary outcomes in clinical studies, there are significant concerns regarding the robustness of the clinical recommendations derived from these studies [32].

Recognizing and addressing the controversies surrounding the use of PROMs is crucial for enhancing the evaluation standards in clinical studies within orthopaedics, sports medicine and rehabilitation. The KSSTA journal has recently concentrated its efforts on this important issue, demonstrating a commitment to improving research methodologies [33, 50, 54-58]. Therefore, the present editorial aims to comprehensively describe the challenges of using PROMs in orthopaedics, sports medicine and rehabilitation to evaluate knee conditions. By studying the intricacies of PROMs, including their strengths and limitations, we aim to contribute valuable insights to the healthcare community, foster a deeper understanding of their actual role in treatment decision-making and enhance patient care and overall outcomes in knee-related medical interventions.

Patient-related outcomes (PROs) are any reports provided directly by the subject, without interpretation by physicians or others, concerning the patient's overall function or feelings related to a condition and its therapy [53]. This definition is based on the feature that most distinguishes PROMs from all other methods of patient assessment: PROMs are unique in that the patients themselves report them, excluding the potential bias by the examiner. This ensures that the patient's experiences and perceptions are at the core of the evaluation. Examples of PROMs include questionnaires that assess pain levels [10, 12], fatigue or so-called perceived exertion [11, 44], depression [7, 49], anxiety [41], physical function [68] and health-related quality of life [70]. However, it is essential to remember that the definition of PROs covers any communication given to us by the patients, including statements expressed during a consultation, not only information gathered using standardized tools or instruments.

PROMs are designed or should have been designed to measure outcomes that matter to patients, such as symptoms, functioning, quality of life and treatment satisfaction. They provide insights into the real-world effects of healthcare interventions from the patient's perspective. PROMs are multidimensional, covering physical, mental, emotional and social health to comprehensively assess a patient's well-being.

Responses to PROMs are usually quantifiable, allowing for the conversion of subjective experiences that are not directly observable and called latent constructs into numerical data [13]. Most commonly, patients allocate numbers to represent the severity of different conditions. This facilitates statistical analysis, research and comparison of outcomes over time.

PROMs can dichotomously be categorized as designed to assess general aspects of health or as those specific to a disease or condition [62], body region [9] or joint [27]. Although there are PROMs designed for specific regions or joints, it is unclear whether these self-assessment measures accurately and exclusively evaluate the intended region or joint.

The primary outcome measure is the crucial variable that researchers deem most important among the various dependent variables under investigation in the study. Elevating PROs to a primary status can be viewed as prioritizing the patient's perception and experience of their condition rather than solely focusing on the disease. This approach places greater importance on understanding how the condition impacts the patient's functioning and overall quality of life. We all fully understand that the patient is the receiver of the treatment and, therefore, has an essential say in treatment outcome to balance the researcher's or clinician's perspective.

However, the designation of PROs as primary ones in orthopaedics, traumatology, sports medicine and rehabilitation has been a matter of great dispute [32]. The most significant doubts arise about using PROs as primary ones when comparing surgical treatment methods with nonsurgical treatment in the studies concerning dysfunctions of the musculoskeletal system. As emphasized in a prior KSSTA editorial, specific studies, such as those examining the treatment of acute tears of the anterior cruciate ligament (ACL), have contributed to discussions regarding the suitability of clinical recommendations derived from such studies [20, 22, 23]. This is especially pertinent because PROs may not necessarily correlate with clinician-reported or objective measures [45]. Another discussion also arises around relying on the results of studies that used inadequate PROMs, for example, PROMs that were not specifically developed for the particular studied group of patients [5, 24, 26, 31, 32, 36, 51, 60, 65].

PROMs in knee trials display considerable diversity. Studies use various PROMs to evaluate patients with the same or somewhat similar conditions. This variability shows the importance of establishing standardized PROM selection and administration guidelines to ensure consistency and comparability across knee research studies.

As an example, we can consider outcome measures used to assess patients after knee arthroplasty. Adriani et al. recently summarized the current use of outcome measures to assess physical function after knee arthroplasty in a systematic review [2]. Of the 181 articles that met the inclusion criteria, 49 outcome measures were identified. PROMs were reported as the predominant choice for evaluation. The most frequently adopted PROMs were the Knee Society Score (KSS) (43.1% of included studies), the Western Ontario and McMaster Universities Arthritis Index (WOMAC) (34.3% of included studies), the Oxford Knee Score (OKS) (28.2% of included studies) and the Knee Injury and Osteoarthritis Outcome Score (KOOS) (20% of included studies). Of note, no PROM was used in more than half of the articles examined [2]. These results align with previous systematic reviews conducted before the release of the Outcome Measures in Rheumatology Clinical Trials (OMERACT) consensus and underscore the significant variability in outcome measures used in joint replacement studies [37, 61].

Several initiatives have been taken in recent years to improve the selection of appropriate outcomes for clinical research purposes.

One such initiative is OMERACT, an international collaboration to standardize outcome measures in rheumatology research and clinical trials. OMERACT brings together researchers, clinicians, patients and other stakeholders to develop consensus-based recommendations for core outcome sets in various rheumatic conditions. The core outcome set defines the minimum outcomes that should be measured and reported in clinical trials to ensure consistency, comparability and relevance across studies. In 2017, OMERACT's interest group on total joint arthroplasty introduced a core domain set for outcomes following hip and knee joint arthroplasties using the onion model, which positioned ‘physical function’ as a fundamental domain [59, 64]. However, discussions within OMERACT and among researchers about measuring and defining physical function continue [64].

The Patient-Reported Outcomes Measurement Information System (PROMIS), developed by the National Institutes of Health, assesses various aspects of health-related quality of life from the patient's perspective. PROMIS provides standardized measures for physical, mental and social health domains, using item response theory to develop and validate item banks. These item banks are administered via computerized adaptive testing or fixed-length short forms, allowing for efficient and precise measurement of PROs across diverse populations and conditions [52]. PROMIS measures are widely used in research, clinical practice and healthcare policy to assess and monitor patient outcomes, guide treatment decisions and evaluate healthcare interventions, but there still needs to be extensive investigation into their application across different health conditions [28].

The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative focuses on enhancing the selection of outcome measurement tools in research and clinical settings. COSMIN ensures that PROMs undergo rigorous psychometric testing to uphold their quality and reliability [48]. Its goal is to advance methodology and provide resources for selecting appropriate measurement instruments [14]. Initially, COSMIN concentrated on PROMs [47].

The Core Outcome Measures in Effectiveness Trials (COMET) initiative aims to improve the consistency and quality of outcomes measured in clinical trials and other studies. COMET provides a platform for developing and disseminating standardized core outcome sets for specific areas of healthcare research. These core outcome sets are created through a collaborative process involving stakeholders, such as patients, healthcare professionals, researchers and policymakers. By promoting standardized outcome measures, COMET enhances the comparability, transparency and relevance of research findings, ultimately improving evidence-based decision-making in healthcare [15].

PROMs must be standardized with established reliability and validity for specific populations. This ensures that the data collected are consistent and accurate and enables meaningful comparisons across different subgroups within the population. However, several commonly used PROMs in orthopaedics, sports medicine and rehabilitation remain controversial concerning their correctness. As an example, we can provide the validity of the PROMs utilized in evaluating patients with meniscal injuries, which, according to the systematic review by Abram et al., was identified as poor or insufficient [1].

PROMs are intended to focus on one domain and are typically validated in a particular country or healthcare system for a disease-specific condition. Whenever any setting-related condition changes, the PROM validation needs to be extended. However, this is unfortunately seldom done, and instead, the PROMs are used in populations where their content validity may not be appropriate. There are various validation strategies, which include aspects like forward and backward translation processes with cross-cultural adaptation [6, 25]. They might also be based on expert interviews or patient involvement [38]. The different strategies have several risks of bias, being potentially correlated to the level of trust end-users might have when using PROMs. A well-designed questionnaire may refer to the classical test theory, using rigid statistical methods, including split-half consistency or internal consistency, to prove the relevance of every question to the outcome of interest, which was tested in a relevant number of stakeholders [40]. This questionnaire is likely to be more reliable and valid than questionnaires, which are only validated via a few interviews.

In recent years, significant attention has been directed towards exploring the potential of technology, mainly digital health tools and mobile applications, to enhance the collection and analysis of PROs. Technology integration fundamentally reshapes the PRO collection and analysis landscape, offering notable advantages, such as real-time data acquisition and heightened patient engagement.

Digital health tools can facilitate the real-time gathering of PROs, enabling patients to directly input their symptoms, experiences and outcomes via mobile applications. This direct input mechanism yields a more immediate and accurate reflection of their health status. Additionally, mobile applications can facilitate remote patient monitoring, eliminating the necessity for in-person visits and providing healthcare professionals with continuous updates on PROs. This remote monitoring capability is advantageous for managing chronic conditions or monitoring early postoperative recovery phases.

Furthermore, digital tools often encompass interactive features like push notifications and reminders to enhance patient engagement in reporting their outcomes. This potentially increased engagement could contribute to generating more comprehensive and timely data. Moreover, wearable sensors can autonomously capture data about physical activity, sleep patterns and other pertinent metrics [29]. By integrating this data with PROMs, a more holistic understanding of a patient's health status can be obtained.

Future developments in PROMs will likely include the integration of wearable devices, artificial intelligence, and other emerging technologies. This integration can significantly enhance the accuracy and comprehensiveness of PROs. However, many European countries have significant legal issues concerning data privacy and security. To approach these concerns, it is crucial to adhere to regulations, such as the General Data Protection Regulation, which provides a data protection and privacy framework. Ensuring patient consent, data anonymization and secure data storage are fundamental steps. Additionally, collaboration with legal experts and continuous monitoring of regulatory changes will be essential to navigate these complexities effectively. Future research should not only focus on the technological advancements of PROMs but also on the development of robust protocols to address these legal challenges.

PROs are unique as they are self-administered and reported directly by patients without involvement from healthcare professionals or other people. This approach has its pros and cons. Nearly a decade ago, El-Daly et al. conducted a study determining the readability level of frequently employed PROMs in orthopaedics, including those for knee joint conditions. They concluded that many PROMs were likely unreadable and incomprehensible to most patients [18]. Readability, defined as the ease by which text can be read and understood, depends on the text's context and presentation. El-Daly et al. assessed readability using the Flesch Reading Ease Score (FRES), with higher scores indicating that text is easily read and understood by individuals from lower school years [18]. Knee PROMs had the largest score spread among other PROMs [18]. Notably, the Lysholm Knee Score exceeded the largest FRES, indicating it is understandable by 11-year-old students at the 6th-grade level [67].

Conversely, the UCLA Activity Scale [71], Knee Outcome Survey [42], International Knee Documentation Committee (IKDC) Subjective Knee Form [27], Tegner Activity Scale [71] and Cincinnati Knee Rating System [3] obtained less than 29 FRES, corresponding to text best understood by postgraduates [18]. The other evaluated PROMs that were between the two extreme levels of readability, starting from lower (easier) readability levels, included the Hughston Clinic Knee Self-Assessment Questionnaire [21], Western Ontario Meniscal Evaluation Tool [30], Anterior Cruciate Ligament Quality of Life [46], OKS [16], Kujala [34], WOMAC [63] and KOOS [62]. Discrepancies in the readability of individual scales for assessing the knee joint based on the study of El-Daly et al. are shown in Figure 1. It is worth adding that the authors evaluated PROMs commonly used at their institution. Therefore, some widely used questionnaires were not included.

As might be expected, the methods used by El-Daly et al. in their study had various constraints. One notable limitation was their exclusive emphasis on text readability, gauged through mathematical algorithms that assessed how well the text aligned with the reader's comprehension, resulting in a numerical score. Another essential constraint lies in the study's reliance on a singular method (one formula) to evaluate text readability, as employing other formulas for the exact text might have produced different outcomes [18]. In addition, the FRES considers long polysyllabic words less readable, which may not be true for all terms. Therefore, the results cannot be read straightforwardly, but it raises a fundamental issue: whether our patients understand the PROMs we give them to complete and whether we can rely on their answers.

Returning to the definition of PROs, it is important to remember that patients must respond to PROMs without guidance from healthcare professionals or anyone else. When we measure knee range of motion, the patient does not tell us if we do it right or wrong and whether the result is correct. This is similar, but in the opposite direction, in the case of PROMs.

It seems interesting, considering the literacy skills of adults in, for example, England, the United States, and Australia [17, 35, 66], whether patients have difficulties understanding PROMs, which were written at a higher level than potential patients’ reading grade. No evidence yet links the readability of specific PROMs and patient literacy to data quality, but this issue seems overlooked in orthopaedics and sports medicine. Given the heavy reliance on PROMs in research to assess treatment efficacy and develop standards, it is crucial to investigate how well patients understand these questionnaires.

Another issue that could be underrated when using PROMs in orthopaedics and sports medicine is the readability of a document in the context of its design. This design involves various elements that contribute to the overall ease with which the reader can comprehend the content, like font and typography, organization and structure, consistent formatting, visual elements and sufficient white space.

When creating patient education materials, clarity of content and design aspects, such as layout and presentation, are crucial. Both text comprehensibility and visual elements are key to effective communication [4, 8, 39]. Patients’ preferences, including typeface style and font size, must be evaluated and addressed [19]. However, it needs to be clarified if we follow these principles when preparing PROMs. Evaluating patient preferences for form readability is essential, as there is a research gap on how typography impacts self-reported outcomes in orthopaedics and sports medicine. Addressing this in future studies is vital to ensure accurate self-reporting and build treatment standards on solid evidence.

As one limitation of their study, El-Daly et al. indicated that they did not evaluate the readability of PROMs in terms of their design. Some PROMs contain visual aids, like a series of boxes or sliding bars, which can potentially improve their level of readability to patients [18]. Additional factors that could influence the readability of PROMs encompass the complexity of conveyed concepts, the level of a patient's interest in the text and the patient's familiarity with the subject matter (disease-related terminology in the context of long-term and newly diagnosed disease) [18]. All of these factors pose challenges for future research. Clinicians and researchers are strongly advised to employ PROMs to assess patients. However, there is a lack of initiative to enhance the comprehensibility of these measures for the patients themselves.

As the definition of PROs clearly states, they must be reported without guidance from somebody else. Therefore, separate vital issues when using PROMs in orthopaedics and sports medicine can be distinguished by accessibility, precisely adapting the document to people with disabilities. This includes using appropriate colour contrast, providing alternative text for images and following accessibility guidelines.

Exploring the controversies and challenges of using PROMs to evaluate knee function in orthopaedics and sports medicine is crucial for several reasons. First, as PROMs play an essential role in broadly understood healthcare decision-making, acknowledging their limitations ensures a nuanced interpretation of patient experiences. Second, by examining controversies using PROMs, we foster a transparent dialogue about the intricacies and potential biases embedded in these assessment tools. This scrutiny contributes to refining the methodologies and highlights areas for improvement using PROMs. Lastly, exploring challenges in readability acknowledges the importance of clear communication in healthcare. It emphasizes the need for accessible and understandable PROMs to enhance patient engagement and the quality of collected data. This editorial on the controversies and challenges with using PROMs is a constructive exercise, striving for continuous enhancement and refinement in pursuing accurate and meaningful patient insights.

In summary, adopting a complementary approach to treatment evaluation is imperative by integrating subjective and objective measures to assess efficacy accurately. This comprehensive approach provides a more holistic understanding of patient outcomes, is the foundation for evidence-based medicine and informs future healthcare policies. Additionally, in light of the numerous controversies surrounding PROMs, there is an urgent need for proactive measures to address concerns and improve the reliability and validity of PROMs for use in clinical practice and research.

The authors declare no conflict of interest.

The authors have nothing to report.