Methodological challenges in pilot trials of herbal medicine: barriers to evidence-based practice

Abstract

Objectives

The growing popularity of herbal medicine (HM) underscores the need for high quality clinical trials to support its evidence-based integration. Pilot trials are essential for addressing methodological challenges in this field. This study evaluates the design quality, feasibility, and reporting of HM pilot trials, with a focus on their capacity to inform future full-scale studies.

Study Design and Setting

A comprehensive collection of HM pilot trials was conducted using PubMed, Web of Science, and Embase, based on predefined inclusion criteria. Data were extracted on trial characteristics, reporting quality, and progression to full-scale studies. To gather additional information on follow-up studies, authors of selected trials were contacted directly by email. Adherence to Consolidated Standards of Reporting Trials (CONSORT) guidelines for pilot trials was evaluated, and Poisson regression was applied to identify factors influencing reporting completeness.

Results

A total of 123 HM pilot trials were reviewed, predominantly from Asia (78.1%). Trials most commonly addressed respiratory (14.6%), nervous (14.6%), and reproductive systems (13.0%). Key gaps in reporting included feasibility assessments (13.1%), sample size rationale (47.2%), and randomization methods (35.8%). HM-specific details, including ingredient processing, quality control, and safety assessments, were inconsistently reported. Among the trials, 4 (3.3%) progressed to full-scale studies. Factors such as trial registration (incidence rate ratio (IRR) = 1.20, 95% CI: 1.11–1.30) and protocol publication (IRR = 1.16, 95% CI: 1.08–1.24) were positively associated with reporting completeness. Moreover, an analysis of the origin of HMs revealed that modern HM trials were 4.7 times more likely to progress to full-scale studies compared to traditional HM trials (odds ratio = 4.70, 95% CI: 0.37–252.91), although the result did not reach statistical significance (P = .300).

Conclusion

HM pilot trials, as they stand, are not yet equipped to reliably guide full-scale studies. Core issues in methodological rigor, particularly in feasibility assessment, sample size justification, and randomization processes, limit their effectiveness and integration into evidence-based practice. A dedicated checklist that merges pilot study standards with the unique needs of HM trials is essential.

Plain Language Summary

HM is increasingly used worldwide, but there are challenges in ensuring its effectiveness through clinical trials. This study aimed to evaluate the quality and reporting of pilot trials involving HM, which are smaller studies conducted before larger trials. Pilot trials are essential to identify potential issues in study design and ensure the reliability of future full-scale trials. We reviewed 123 trials from 21 countries and found that many lacked clear details on key aspects, such as sample sizes, randomization, and feasibility. These gaps make it difficult for pilot trials to reliably inform full-scale studies. Apart from this, the pilot trials reporting checklist did not include important details specific to herbal treatments, such as ingredient processing and quality control. In addition, only a few trials progressed to larger studies, often due to funding problems or other practical issues. Our study highlights the need for better reporting standards tailored to HM trials, with clear guidelines to address the unique aspects of herbal treatments. By improving the methodological rigor of pilot trials, we can ensure that HM becomes a more reliable and integral part of evidence-based health care.