Implementability and impact in clinical research and the role of clinical trial networks

Abstract

The Australian Clinical Trials Alliance (ACTA) impact and implementation reference group has published guidance on clinical trial planning, design, conduct and reporting, to optimise the impact of late-phase trials.¹ It is critical that clinical trials are designed, executed and reported on so that the generated evidence can be implemented and applied to improve practice, health systems and policy (defined as implementability).¹ This is particularly important for late-phase trials as results from these trials guide stakeholder decisions on adoption or removal of candidate interventions from practice and policy. The results can also be used to guide implementation, potentially supplemented by other study types. In some cases, decisions on candidate interventions may be based on a single trial, although more commonly, they are based on the synthesis of evidence from multiple trials. If the results of positive late-phase trials are not put into effect, the community benefit, impact and return on investment are lost. Therefore, to reduce wastage and optimise value to the community, who is both the funder and beneficiary of research, we need to maximise the impact of trials. This impact includes health and economic benefits and requires trials to be designed for implementability.^2-4

Implementability is key to trial design and delivery, independent of the results of the trial. Impact is dependent on intervention efficacy and implementability. Late-phase trials should incorporate implementability and be designed to be useful to their diverse end users (including organisations, governments, clinicians and consumers).

Characteristics that contribute to implementability include concepts around trials that are embedded in clinical care and are pragmatic, focused on informing clinical practice and policy and considering real-world investigators, recruitment, participants, intervention, delivery and outcomes. The PRECIS-2 framework (PRagmatic Explanatory Continuum Indicator Summary framework) indicates where a trial sits on the spectrum between the explanatory “Can this intervention work under ideal conditions?” to the pragmatic “Does this intervention work under usual conditions?”. This framework highlights the vital need to identify, involve and co-design or tailor trial design to the needs of end users.⁵ The Consolidated Framework for Implementation Research (CFIR) and similar tools are also useful to guide design and evaluation of implementation research.^{6, 7} In implementability, key trial design considerations include eligibility and exclusion criteria, setting, requirement for a skilled workforce or specialised equipment and extent of data collection. Similarly, trial design contributes to both embedding and implementability; with the need to recognise potential differences between strategies to promote trial efficiency, and those to promote implementability of trial findings.

The important steps for progressing interventions through to implementation into practice, health services and policy are shown in the Box.

These steps align with the Australian Academy of Health and Medical Sciences report⁸ emphasising the importance of embedding research into health care, and the vital role of workforce and infrastructure in achieving this vision. The report also highlights the important role of National Health and Medical Research Council accredited research translation centres (RTCs) in implementation. The RTCs encompass most research, education and hospital services nationally and are well positioned to support clinical trial networks and enable implementation.⁹

Trials should be co-designed with end users, including consumers, who have experience and understanding of current practice.¹⁰ It is acknowledged that in some circumstances opportunities for co-design may be limited, such as in industry-sponsored trials. Pre-trial work with end users can establish relevance and the priority of research questions and appropriate outcome measures. Formal processes can be useful¹¹ such as Delphi and nominal group techniques, although these need to be appropriate to the local context.

End points must be carefully chosen, including those of value to end users, and the minimum clinically important differences in end points considered. In addition, the minimum public health significant difference, which defines the relationship between a meaningful effect size and the public health disease burden, should be taken into account. For example, a major effect size (such as a cure for an inborn error of metabolism) in a rare disease may be just as important as minor effect sizes in common diseases (as this might cure a baby at birth and avoid lifetime costs and gain a lifetime of productivity).

The feasibility of intervention delivery should be established before a late-phase trial. This can be assessed via a systematic review of existing evidence (which may establish the merit for implementation of a particular approach to treating a health problem), as well as pilot studies. Regulatory approvals or burden should also be considered for implementation of drug or device interventions, alongside required skills and resources.

To optimise generalisability, consideration should be given to the population to which the results of the trial apply. The selected population should be as broad as possible, and entry criteria should be easily interpreted, using easily accessible information available to clinicians in routine practice, and participants in community-based trials. If the candidate intervention is targeted to a clinical setting, trial sites should be broadly representative of the centres that provide care to the key population groups in which the intervention would be implemented post-trial. For population-based studies, community-based recruitment and decentralised trial designs should be considered. If implementation will be beyond tertiary or quaternary hospitals, then the trial should be conducted to include this setting.

Inclusion criteria should be broad and exclusion criteria as limited as possible. Research burden and complexity from additional factors such as research-only or largely unavailable biomarkers should be limited. The selected population should be diverse and representative of the population that the matter in question affects. Importantly, populations who experience inequities should be appropriately included through a range of strategies, including engaging with representatives from priority populations.

Interventions should be delivered in ways similar to current or future use in routine clinical practice. If interventions are designed to align with routine clinical practice, an intervention will likely be easier to embed and adopted more quickly. If an intervention has efficacy in a trial that has considered implementability, it is likely to be broadly effective when implemented into practice. For example, trialists should optimise opportunities for clinical staff (and not researchers) to deliver interventions. As reach and penetration are key to broader effectiveness, trials that can only recruit a small proportion of a target population are not recommended. Comparators, or the current standard of care (ie, an active control) where the investigator treats control participants as they would normally, should be included in trial protocols, as this offers the opportunity to provide a definitive end of trial recommendation.

End user consultation and strategies, such as run-in periods, can assess challenges in adherence with the intervention. Protocols should include adjustments or titration that may be necessary to achieve efficacy or avoid adverse events and these should align with application in routine care. Ultimately, including diverse end users can inform embedding and pragmatic aspects related to implementability.

Other design considerations are concomitant care; strategies to increase reach (such as limited exclusion and inclusion criteria); analysis (intention-to-treat principles are recommended¹²); limiting trial burden to optimise and sustain participation (eg, by considering type of consent, requirements of participant follow-up, and use of streamlined case report forms); embedding trials in a registry;¹³ considering and capturing factors that influence reach, adoption, fidelity, maintenance and process evaluation;¹⁴ and inclusion of health economic end points. A well designed, prospective health economic analysis may be critical to optimise implementation and will inform decision makers regarding investment.^15-17 Economic analysis should also focus on cost-effectiveness of interventions across different modelled scenarios, such as for specific population subgroups. This allows for the consideration of equity and provides policy makers with more nuanced investment options than simply offering “all or nothing” choices.

Historically, a large proportion of studies go unpublished, or reporting lacks sufficient detail.¹⁸ This selective reporting increases the risk of publication bias and must be rectified. All trials must be reported in a timely manner, preferably within 12 months of their completion, and those reports must be readily accessible.¹⁹ This is vital if the trial has insufficient recruitment or is unable to deliver the intervention as it will still generate new knowledge on process, challenges and implementability, relevant to the trial and to implementation into practice. Detailed reporting on factors that affect implementation is important and consideration of an implementation protocol can optimise generated data and assist end users with implementing trial findings into practice. Interventions should be described in detail to enable replication, using tools such as the Template for Intervention Description and Replication (TIDieR) checklist.²⁰ If the comparator was protocolised, details of the comparator and delivery methods should be clear. Open access publication should be prioritised to optimise reach to end users. Affordable open access options are needed to ensure equity and access.²¹ Additional dissemination tools, channels and strategies (such as implementation guides, evidence synthesis, guidelines, policy summaries, round tables, health professional and consumer tools) are key to implementation and should be co-designed with end users. Authors should declare and maintain a record of real or perceived conflicts of interest and provide access to data where possible.

Many, but not all, clinical trials are completed by a network, with most members holding dual roles as researchers and clinicians. Many networks also include consumers, representative of end users. Consumer and community involvement (CCI) is fundamental, recommended or required by funders and considered good practice. This can include both formal or informal processes, across planning, design, conduct and reporting of trials. Early and embedded CCI in a structured, meaningful way within the network also enables appropriate implementation of trial outcomes. Networks can support quality CCI, as consumer engagement from the outset is prioritised by the NHMRC CCI statement. Tokenistic engagement with consumers should be avoided. ACTA, NHMRC and RTCs provide tools and resources to build capacity and support genuine CCI.^22-24

Networks can provide access to representative trial sites aligned to clinical sites and populations where results are applicable. Some networks, potentially with an associated registry, are ideally positioned to monitor implementation (Box). Lastly, networks can facilitate the next steps in dedicated implementation research involving the systematic study of methods, strategies and pathways that support the application of trial findings into policy and practice. It is acknowledged that implementation science requires different but overlapping skills and expertise to that of clinical trials and that capacity building is warranted in both implementability and implementation science.

To deliver on the investment and potential benefit of late-phase clinical trials, it is important that end user-informed implementability considerations are widely adopted and applied. Further information on best practice for the design, conduct and reporting of studies with a view to implementation can be found on ACTA's website (www.clinicaltrialsalliance.org.au/resource/6258/).

Open access publishing facilitated by Monash University, as part of the Wiley - Monash University agreement via the Council of Australian University Librarians.

Samantha Keogh reports monies received by her employer Queensland University of Technology from BD Medical and ITL Biomedical for educational consultancies not related to this study. Paul Cohen reports speaker's honoraria from Seqirus and AstraZeneca, stock and advisory board participation in Clinic IQ. David Johnson has received consultancy fees, research grants, speaker's honoraria and travel sponsorships from Baxter Healthcare and Fresenius Medical Care, consultancy fees from Astra Zeneca, Bayer, and AWAK, speaker's honoraria from ONO and Boehringer Ingelheim and Lilly, and travel sponsorships from ONO and Amgen. All other authors declare no relevant disclosures.

Not commissioned; externally peer reviewed.