Research integrity and peer review最新文献

Background: Democracy and freedom of press may affect how science is prioritized, produced, communicated and disseminated. We aimed to map the production of scientific publications worldwide in terms of democracy and freedom of press ratings of countries.

Methods: This is a bibliometric study cross-linking global bibliometric data with democracy ratings and freedom of the press indices for countries around the world. Democracy ratings used the Democracy Index in 2024 and in 2006 (when first released by the Economist Intelligence Unit) and Freedom of Press ratings used the 2024 index by Reports Without Borders. The Scopus database was used for publications from each country. Fractional counts were assigned for publications co-authored by authors from different countries. Full articles, reviews, conference papers, books and book chapters were included.

Results: In 2024, countries characterized as full democracies produced only 22% (915,102/4,185,853) of the Scopus-indexed publications, versus 66% (1,157,842/1,757,310) in 2006. There was no correlation between the ratio of publications indexed in 2024 versus 2006 and the absolute or relative change in Democracy Index between 2006 and 2024 (r = 0.02 and r = 0.00, respectively). 78% of publications in 2024 (3,255,770/4,187,136) came from countries with problematic (including USA) or worse (including China) rating for freedom of press. Proportions of publications originating from countries with problematic or worse situations were 81% (n = 3,374,348), 91% (n = 3,820,811), 61% (n = 2,537,962), 62% (n = 2,608,802), and 63% (n = 2,650,819) for political, economic, legislative, sociocultural, and safety/security dimensions, respectively. Results were similar when limited to articles published in 2024 in journals with continuous annual presence in Scopus during 2006-2024. 87.1% (1,489/1,710) of the highly cited papers published in 2024 (with 150 or more Scopus citations by November 23, 2025) have at least one author from a country that is not full democracy and 98.8% (1,690/1,710) of these highly cited papers have at least one author from a country that does not have good freedom of press.

Conclusions: Most published science originates from countries struggling or suffering in democracy and/or freedom of press. The deeper causes and implications of this emerging landscape require further study.

Background: Preprints are becoming more common in the health sciences and allow for instant dissemination of research findings; however, with the risk of compromising quality and transparency. Peer review potentially improves reporting and reduces errors, although its actual impact is not known. The objective of this scoping review was to synthesize evidence comparing preprints in the health areas to their peer-reviewed versions and assess preprint publication rates.

Methods: We searched Embase, Medline OVID, Scopus, and Web of Science from inception to July 2024 for studies comparing preprints with their peer-reviewed versions and/or investigating preprint publication rates. Two reviewers independently conducted screening and extracted data on study characteristics, parameters compared, and preprint publication rates. We conducted a narrative synthesis.

Results: We included 40 studies (published 2019-2024; 92% peer-reviewed). The median number of studies analyzed per article was 356 (range: 19-73,256). 42% of preprints were eventually published among 33 studies that reported publication rates (IQR: 22%-67%). Preprint searches routinely started on January 1, 2020, with a median of 24.3 months and a median difference of 11.5 months between preprint and peer-reviewed search end dates. Commonly compared parameters were primary outcomes/endpoints (37%) and sample size (30%), with peer-reviewed articles showing improved reporting for funding (13%), conflicts of interest (13%).

Conclusion: While peer review enhances transparency and methodological reporting (e.g., funding, conflicts of interest), the content, outcomes, and conclusions of health-related preprints remain largely consistent with their peer-reviewed versions. Preprints facilitate rapid knowledge dissemination but may benefit from stricter reporting standards to improve credibility. Future efforts should focus on standardizing preprint policies to bridge quality gaps without delaying access.

Background: Despite growing calls for gender-responsive psychological research, implementation of gender-related guidelines is underresearched. The Sex and Gender Equity in Research (SAGER) guidelines recommend reporting participants' gender, presenting gender-stratified results, analyzing gender-related data, acknowledging non-binary identities, and distinguishing between biological sex and social gender. This scoping review assessed the extent to which these guidelines are followed.

Methods: We included all primary data studies on human participants published in Psychological Science from 2019 to 2024 (n = 699) and assessed their gender reporting practices according to the SAGER guidelines.

Results: While 87.8% (n = 614) of studies reported participants' gender, only 35.3% (n = 247) presented gender-stratified results, and 24.2% (n = 169) conducted gender-based analysis. Only 17.2% (n = 120) of studies reported participants' non-binary identities. Regional patterns emerged: Global North studies more frequently reported non-binary identities but less often presented gender-stratified results and conducted gender-based analysis than Global South studies. The U.S.-based studies saw a notable decline in reporting gender-stratified results, from 43.2% (n = 32) in 2022 to 28.1% (n = 16) in 2024.

Conclusion: This review reveals persistent inconsistencies in how gender is conceptualized and reported. It provides recommendations to improve gender reporting in order to facilitate the production of more accurate and socially relevant knowledge in psychological research.

Background: While conducting an umbrella review of e-cigarettes for smoking cessation, we observed that in many instances, systematic review authors reported findings favorable to the treatment, yet they declined to recommend it or recommended against it despite the evidence of its effectiveness in their own systematic reviews.

Existing literature: We searched the literature for a term or category to describe this form of reporting bias where the authors' recommendations dismiss their findings of treatment benefit. Ideally the term spin bias should apply to any conclusion or recommendation not supported by the findings of the study, but in practice spin bias is almost exclusively applied to the narrative attribution of significance or causation to statistically non-significant data or findings.

Issue under discussion: After observing that many systematic review authors dismissed their findings of effectiveness for e-cigarettes for cessation, we wondered if this form of reporting bias also occurs in the systematic reviews on other controversial treatments. We made a rapid search for recent systematic reviews on medical cannabis for pain, another controversial treatment. Here also we observed that many authors did not recommend cannabis for pain management even though their findings clearly showed treatment benefit. We tentatively offer the term reverse spin bias for the narrative discounting or dismissal of statistically significant findings. We catalogued the narrative turns that enabled reverse spin bias in 20 systematic reviews of e-cigarettes for cessation and medical cannabis for pain. We identified five mechanisms: discount the evidence base, discredit the primary studies, appeal to fear, dismiss the treatment modality a priori, and omit findings. We speculate that authors introduce reverse spin bias to improve their chances for publication or to support their position about a treatment.

Conclusion: A standard task for editors and peer reviewers is confirming that treatment recommendations are supported by the review's data, yet our examples strongly suggest that this examination for reporting bias is frequently skipped. By proposing a new term, reverse spin bias, we hope to bring stronger scrutiny to bear on these instances of reporting bias that are detrimental to evidence-informed clinical practice.

Background: Sample size calculations are critical in clinical trial design, yet hypothesised effect sizes are often overly optimistic, leading to underpowered studies. Research ethics committees (RECs) assess trial protocols, including sample size justification, but their role in mitigating optimism bias in sample size calculations is not well studied.

Methods: We descriptively analysed 50 clinical trial protocols approved by a Dutch REC (2015-2018) with available primary outcome results. We examined REC comments on sample size calculations, protocol modifications during ethics review and amendments, and discrepancies between target and observed effect sizes. For comparability, effect sizes were standardised.

Results: Nine (18%) trials received REC comments on sample size calculations, mainly addressing calculation errors (n = 5), missing parameters (n = 2), or other methodological considerations (n = 3), with only three comments (6%) requesting effect size justification. Seven (14%) trials modified their sample size calculation during ethics review, mostly in response to REC comments, and 10 (20%) trials made modifications in amendments. In total, 40 (80%) trials overestimated their target effect size. Across all trials, the target effect size was overestimated by a median of 0.22 [IQR: 0.03 - 0.41]. Changes during ethics review led to less overestimation for only one trial, which reflected the correction of a calculation error rather than a reassessment of assumptions.

Conclusions: Optimism in sample size calculations is common, but the influence of REC feedback on reducing overestimation appears limited. As this was a small, descriptive study from a single Dutch REC in 2015-2018, findings may not generalise to other settings or more recent practice. Future research should validate these findings and may help identify characteristics associated with overestimation, supporting RECs in recognising trials at risk of being underpowered.

Objectives: To test whether improving a Christmas tree meringue recipe using reporting guidelines yields more appealing and sweeter meringues.

Design: A prospective, superiority, single-blind, cluster-randomised (1:1) controlled trial.

Setting: A public participatory event in a large cultural facility in France.

Participants: Budding chefs with basic culinary skills, possessing the utensils necessary for baking Christmas tree meringues, and not having burned pasta in the past month (for safety reasons). Bunding chefs represent the cluster and meringue the unit.

Interventions: Each budding chef was randomised to a standard recipe for making Christmas tree meringues or to the same recipe written in consultation with a professional baker using the TIDieR checklist-a reporting guideline for description of complex interventions-plus a short video tutorial.

Main outcome measures: The primary outcome was reproducibility in terms of visual aspect. Secondary outcomes included colour, size, taste and survival time in the course of a sale organised as part of the public event. The visual aspect, colour and size was rated by an independent jury which compared the cooked Christmas tree meringues with the recipe picture on a scale from 1 to 10. Analyses were performed in intention-to-eat (randomization unit: budding chefs / analysis unit: Christmas trees).

Results: 60 budding chefs (30 in each group) baked a total of 845 Christmas tree meringues. There was no significant difference between the groups (mean difference = -0.1; [95%CI -0.99; 0.80]; p-value = 0.84; intra-cluster correlation, ICC = 0.77) on visual aspect. No difference was found for reproducibility in terms of colour (mean difference = -0.31; [95%CI -0.97; 0.35]; p-value = 0.35; ICC = 0.67) or size (mean difference = -0.17; [95%CI -1.07; 0.73]; p-value = 0.71; ICC = 0.74). There was no significant difference in terms of taste between the groups (mean difference = -0.55; [95%CI -1.62; 0.52]; p-value = 0.31). 400 meringues were sold during the public event with no difference in survival time between groups (hazard ratio = 1.26 [95% CI 0.75-2.09], p-value = 0.38, with values > 1 in favour of the control group). e.g. 75% survived for 134 min in intervention group and for 124 min in the control group.

Conclusions: Our study failed to demonstrate that an improved recipe using the TIDieR reporting guideline with a video tutorial improved the reproducibility in terms of visual aspect, colour, size, taste and sales for Christmas tree meringues. The best way to succeed in reproducing Christmas tree meringues alike those showcased by the recipe-and thereby to improve reproducibility of experiments-remains a mystery still to be solved by further explorations.

Trial registration: https://osf.io/dnhbx .

Background and aim: The International Committee of Medical Journal Editors (ICMJE) defines a potential conflict of interest (COI) as a situation where professional judgment could be influenced by secondary interests. Competing interests can introduce bias into the peer-review process, making it essential for all participants to declare any potential COIs. While authors are currently required to disclose their COIs, editors and editorial board members are not held to the same standard. This study aimed to evaluate the extent to which editors and editorial board members of ethics journals report their potential competing interests.

Methods: From October 23 to November 1, 2024, 82 ethics journals selected based on their impact factors were assessed, focusing on the disclosure of potential COIs by editors and editorial board members. Journal websites were examined to determine how editors and board members disclose potential COIs. Additionally, publisher websites were assessed for policies guiding these individuals in reporting COIs during peer review.

Results: Only 2% of the journals disclosed potential COIs for their editors, and 13% provided biographical information about editorial members. None of the journals employed a structured reporting approach, such as the ICMJE disclosure form, despite most claiming adherence to ICMJE and COPE guidelines. There was considerable variability in how journals and publishers guided their editors and board members in reporting their own COIs.

Conclusion: The findings indicate that disclosures of potential COIs by editors and editorial board members in leading ethics journals are often inconsistent and insufficient. Increasing transparency in this area could lead to a fairer and more trustworthy peer-review process.

Background: Research methodology education aims to equip students with the foundational knowledge of robust scientific practices, emphasizing deep understanding of scientific inquiry, integrity, and critical thinking in research practice. A literature review reveals that the observed diversity in research methods course design and instruction stems from a lack of consensus about the essential foundations required to critically engage with, design, and execute research in education. This is further compounded by a limited pedagogical innovation. However, no study has yet investigated how research methodology is taught and perceived across European universities. The objective of this study is to examine practices and attitudes regarding teaching research methodology in different European countries, across different disciplines and different training stages to identify commonalities and discrepancies.

Methods: A cross-sectional survey was designed based on the Structure of Observed Learning Outcome (SOLO) taxonomy and further developed in several rounds of expert input and feedback, ensuring comprehensive inclusion of diverse teaching formats and assessment types. The survey was distributed to research methodology and non-research methodology higher education teachers across Europe through stratified and snowball sampling methods.

Results: The survey was completed by 559 respondents across 24 countries and seven disciplinary categories. The findings identified a predominant reliance on traditional passive teaching formats, such as face-to-face or online lectures. Active methods such as flipped classroom (8.4% Bachelor, 4.8% Master, 2.3% PhD) and protocol writing (8.2% Bachelor, 6.6% Master, 3.9% PhD) were less frequently used. Written exams dominated assessment strategies at all levels. Across our stratification levels, all topics were rated very important, with hypothesis formulation, research integrity, and study design as the most necessary topics, while pre-registration, peer review, and data management plan were prioritized slightly less.

Conclusions: These findings reveal relative homogeneity in research methodology teaching across academic levels and disciplines in Europe. The persistence of passive teaching formats and the limited adoption of active methodologies reflects an untapped opportunity to improve the effectiveness of research methodology education in fostering critical thinking and ethical practices. Higher education institutions need to reevaluate research methodology curricula to better align with contemporary research demands.

Background: Gender and geographical disparities have been widely reported in the peer-review process of biomedical journals. Artificial Intelligence (AI) is increasingly transforming the publishing system; however, its potential to identify suitable reviewers, and whether it might reduce, replicate or reinforce existing biases in peer review has never been comprehensively investigated. This study sought to determine the usefulness of AI in identifying expert scientists in medicine taking into consideration gender and geographical diversity, equity and inclusion (DEI).

Methods: The title and abstract of 50 research articles published in high-impact biomedical journals between November 2023 and September 2024 were fed into a large language model software (GPT-4o), which was prompted to identify 20 distinguished scientists in the study's field. Two trials were randomly performed with and without a gender and geographical DEI prompt. Scientists were classified based on gender, geographical location, and country of affiliation income level. Furthermore, the number of peer-reviewed publications, Google Scholar-derived total citations and h-index were computed.

Results: Without a DEI prompt, GPT-4o primarily identified male scientists (68%) and those affiliated to high-income countries (95.3%). Conversely, when DEI was explicitly prompted, GPT-4o generated a gender-balanced (51% females) and geographically diverse list of scientists. Specifically, the proportion of scientists from high-income countries decreased to 42.3%, while representation from upper-middle (3.2% to 26.2%), lower-middle (1.2% to 26.1%), and low-income (0.2% to 5.4%) countries significantly increased. The number of publications (without vs. with DEI: 284 ± 237 vs. 281 ± 245, P = 0.77), citations (48,445 ± 60,270 vs. 53,792 ± 71,903, P = 0.13), and h-index (79 ± 43 vs. 76 ± 43, P = 0.15) did not differ between groups.

Conclusions: When not prompted to consider DEI, GPT-4o successfully identified expert scientists, but primarily males and those from high-income countries. However, when DEI was explicitly prompted, GPT-4o generated a gender-balanced and geographically diverse list of scientists. The academic productivity was considerably high and comparable between groups, suggesting that GPT-4o identified potentially skilled scientists who could reasonably serve as reviewers for scientific journals. These findings provide evidence that AI can be an ally in combating gender and geographical gaps in peer review, though DEI should be explicitly prompted. Conversely, AI could perpetuate existing biases if not carefully managed.

Objectives: To characterize journals that published and retracted articles retracted for having originated from paper mills and examine associations between paper mill retraction frequency and journal characteristics.

Methods: Retraction Watch database was used to identify papers retracted due to originating from paper mills and journals, between January 2020 and December 2022. Data on the total number of articles and journal characteristics were obtained from Web of Science and Journal Citation Reports. Journals were classified based on the frequency of retracted paper mill papers (1, 2-9, ≥ 10 retractions). Logistic regressions were conducted to explore associations between retraction frequency and journal characteristics.

Results: One hundred forty-two journals were identified that retracted 2,051 articles from paper mills. Among these, 71 (50%) journals had 1 retraction, 36 (25.4%) had 2-9 retractions, and 35 (24.6%) had ≥ 10 retractions; 4 (2.8%) journals had > 100 retractions. These journals, regardless of paper mill retraction number, were mainly in the second (35.2%) and third (29.6%) quartiles by impact factor. Medicine and health emerged as the predominant subject area, comprising 61.2% of all indexed journal categories. Comparing journals with one retraction to those with ten or more, the proportion of open access articles (72.6% vs. 19.2%) and median editorial times (86 vs. 116 days) differed across groups, although these differences were not statistically significant. An inverse correlation was observed between the proportion of paper mill papers and original articles (Spearman's Rho = -0.1891, 95%CI -0.370 to -0.008). Logistic regressions found no significant association between paper mill retraction number and other variables.

Conclusion: This study suggests that paper mill retractions are concentrated in a small number of journals with common characteristics: high open access rates, intermediate impact factor quartiles, a high volume of citable items, and classification in medicine and health categories. Short editorial times may indicate a higher presence of paper mill publications, but more research is needed to examine this factor in depth, as well as the possible influence of acceptance rates.