Rehabilitation journal editors recognize the need for interventions targeted to improve the completeness of reporting, but there is heterogeneity in terms of strategies actually adopted: A cross-sectional web-based survey

Abstract

Reporting guidelines (RGs) have been developed by researchers, methodologists, and journal editors to enhance the quality and transparency of scientific research reporting¹ and to facilitate readers’ assessment of the internal and external validity of the studies. In 2014, a joint editorial² of 28 rehabilitation journals highlighted the need to use RGs to ensure the quality of reporting rehabilitation research, stating “that simultaneous implementation of this new reporting requirement will send a strong message to all disability and rehabilitation researchers about the need to adhere to the highest standards when performing and disseminating research.”

While there is encouraging evidence that RGs improve reporting quality,³ there are also challenges to implementation and dissemination. Rehabilitation is a growing field with increasing attention to the quality of reporting. However, our previous studies show that in rehabilitation journals authors do not frequently declare the use of RGs,⁴ and the completeness of reporting of randomized controlled trials⁵ and systematic reviews⁶ is suboptimal.

The study protocol and the full version of the survey are available on the OSF Repository.⁸ We followed the guidance by Gaur et al.⁹ for the reporting of this manuscript. We conducted a web-based survey among all editorial board members (i.e., editors-in-chief, senior editors, and associate editors) of the 68 journals indexed under the “rehabilitation” category in InCites Journal Citation Report. This study complies with the Code of Ethics for Research involving Human Participants Faculty of Science (BETHCIE), Vrije Universiteit Amsterdam; therefore, a formal ethics review is not required.¹⁰ All participants were explicitly informed that participation in the survey was anonymous and voluntary. A copy of informed consent is available in the OSF repository.⁸ Only the participants that explicitly gave their consent to be acknowledged were mentioned in the acknowledgment section. Further information on ethics and privacy are available in the published protocol.⁸

Several meetings were held among the study authors, including one editor-in-chief of a rehabilitation journal (RZP) and two members of editorial boards of high-impact rehabilitation journals (RO, AV) to discuss the survey. One investigator (TI) developed a preliminary version, which was then further reviewed and discussed to ensure that the questions adequately met the study aims (confirming face validity). Next, the clarity of the questions with appropriate response items was checked in a pilot test among the study authors (RZP, AV, AC, RO, SS, SG), and any duplications or coinciding themes were removed. The initial questionnaire was revised three times and the final questionnaire included closed, semiclosed and open-ended questions. First, journal editors were identified by visiting the editorial board web pages of the 68 journals and by searching the publicly available academic email addresses of these authors on the internet. Second, we directly asked the journals for those emails not found through this process. Third, the developed survey was sent to all available email addresses. The online survey was open from May 15 to August 1, 2022. Follow-up reminder emails were sent every 4 weeks. Only completed responses were included in the analysis. The study protocol⁸ describes further details.

We performed a descriptive analysis of the responses. Percentages were provided for binary data and count data. For Likert scale data, medians with interquartile ranges were reported. For free-text responses, two authors (TI and SG) undertook independent qualitative analyses to identify categories and themes following a thematic analysis approach.¹¹ A consensus discussion between the two authors was used to resolve any discrepancies. Preplanned subgroup analyses were conducted depending on the role of an editor in a journal (e.g., editor-in-chief, senior editor, or associate editor) and on the journal impact factor quartile. Data were analyzed and graphs were drawn using Qualtrics, Provo, UT.

From the 507 email addresses collected, 28 produced an email failure message after two attempts. Of a total of 479, we received 154 responses. Twelve responses were identified from Qualtrics Software as partial responses, which were removed from the sample, meaning 142 complete responses (29.6%) were available for analyses. Table S1 reports detailed characteristics of the sample.

Almost all participants stated they were aware of what RGs are (n = 136; 95.8%), and of the existence of separate RGs for different study designs (n = 138; 97.2%). Approximately one-third of participants (33.8%) were not aware of the Enhancing the Quality and Transparency Of health Research (EQUATOR) Network.¹²

When we asked about the importance and usefulness of the use of RGs, most of the editors (n = 83; 58.5%) believed that it is “very important” that authors follow the RGs to ensure the completeness of reporting (median rating = 5; interquartile range—IQR = 5–4).

Regarding policy and routine activity of the editors and journals, most of the journals (n = 114; 80.3%) referred to the RGs in the instructions for the authors while 82 (57.7%) required mandatory submission of the RG by the authors during the submission. The lack of time (n = 78; 54.9%) was the most common barrier to the use of RGs. Summary results are reported in Table 1.

Comments and suggestions to improve the use of RGs given by participants in response to the open question were grouped into themes (Table S2). The most suggested theme is about actions targeted to the publishers, journals, or editorial process, and making the use of RGs mandatory during submission (n = 40; 28.2%) and the need for a standard approach among all rehabilitation journals (n = 11; 7.7%).

Table S3 reports the subgroup analyses.

In summary, we found that almost all editors of rehabilitation journals are aware of the importance of the RGs and recognize the need for interventions to improve the completeness of reporting. However, our study revealed a great heterogeneity in terms of what the journals and editors do to endorse the use of RGs. Only approximately half (49%) of the editors ask to resubmit the study alongside the appropriate RG when the authors do not declare to follow any RG in their manuscripts, and most of the participants (60%) do not ask the peer-reviewers to check the RG use.

The most common perceived barrier to endorsing and adhering to RGs is the lack of time. This corroborates with previous studies,^{7, 13} that concluded that too many responsibilities fall on the shoulders of busy unpaid peer-reviewers who may not be fully equipped to carry out the role.

This study allowed us to identify several potential actions that could promote the widespread adoption of RGs (Figure S1), where publishers, journals, editors, peer-reviewers, and authors play an (active) important role. We encourage publishers and journals to adopt a standard approach to increase the attention paid to the reporting issues during the editorial and peer-review screening processes, providing explicit instruction to authors and peer-reviewers (e.g., by placing active links to the relevant RGs in the manuscript submission systems), and promoting the dialogue between editors and reviewers. Even if journals, editors, and peer-reviewers are gatekeepers to scientific publication, we firmly believe that primarily authors are responsible for complete reporting, in the same fashion as they should ensure the integrity of study design, data analysis and statistical methods, and avoiding research misconduct.

Our study has limitations. The response rate was low (29.6%). However, other surveys¹⁴ in the same field shows an even lower response rate, probably due to a gradual decrease in survey participation¹⁵ among health professionals. Moreover, participants in this survey come from journals pertaining to all quartiles, so it is reasonable to believe that it is a representative sample of the rehabilitation category.

In conclusion, rehabilitation journal editors are generally aware of the existence of RGs and recognize the need to promote the use of RGs to improve the quality and completeness of reporting. However, there is great heterogeneity in terms of what the journals and editors do to endorse the use of RGs. Moreover, we hope that journals will adopt strategies to increase the attention paid to the reporting issues during the editorial and peer-review screening processes, providing explicit instruction to peer-reviewers and promoting the dialogue between authors, editors, and reviewers.

RO, AV, and RZP are on the editorial board of The Journal of Physiotherapy, The Journal of Orthopaedic & Sports Physical Therapy, Musculoskeletal Science & Practice or The Brazilian Journal of Physical Therapy, respectively.