Research integrity and peer review最新文献

Background: Research on research integrity has tended to focus on frequency of research misconduct and factors that might induce someone to commit research misconduct. A definitive answer to the first question has been elusive, but it remains clear that any research misconduct is too much. Answers to the second question are so diverse, it might be productive to ask a different question: What about how research is done allows research misconduct to occur?

Methods: With that question in mind, research integrity officers (RIOs) of the 62 members of the American Association of Universities were invited to complete a brief survey about their most recent instance of a finding of research misconduct. Respondents were asked whether one or more good practices of research (e.g., openness and transparency, keeping good research records) were present in their case of research misconduct.

Results: Twenty-four (24) of the respondents (39% response rate) indicated they had dealt with at least one finding of research misconduct and answered the survey questions. Over half of these RIOs reported that their case of research misconduct had occurred in an environment in which at least nine of the ten listed good practices of research were deficient.

Conclusions: These results are not evidence for a causal effect of poor practices, but it is arguable that committing research misconduct would be more difficult if not impossible in research environments adhering to good practices of research.

Background: Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings.

Methods: In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals.

Results: Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication.

Conclusions: Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Background: Integrating a sex and gender lens is increasingly recognized as important in health research studies. Past failures to adequately consider sex in drug development, for example, led to medications that were metabolized differently, proved harmful, or ineffective, for females. Including both males and females in study populations is important but not sufficient; health, access to healthcare, and treatment provided are also influenced by gender, the socially mediated roles, responsibilities, and behaviors of boys, girls, women and men. Despite understanding the relevance of sex and gender to health research, integrating this lens into study designs can still be challenging. Identified here, are nine opportunities to address sex and gender and thereby strengthen research proposals.

Methods: Ontario investigators were invited to submit a draft of their health research proposal to the Sex and Gender Research Support Service (SGRSS) at Women's College Hospital in Toronto, Ontario. The service works to build capacity on the integration of sex, gender, and other identity factors, in health research. Using the SAGER Guidelines and the METRICS for the Study of Sex and Gender in Human Participants as guides, proposals were reviewed to enhance their sex and gender considerations. Content analysis of the feedback provided these investigators was subsequently completed.

Results: Nearly 100 hundred study proposals were reviewed and investigators provided with suggestions on how to enhance their proposal. Analyzing the feedback provided across the reviewed studies revealed commonly overlooked opportunities to elevate consideration of sex and gender. These were organized into nine suggestions to mirror the sections of a research proposal.

Conclusion: Health researchers are often challenged on how to integrate a sex and gender lens into their work. Reviews completed across a range of health research studies show there are several commonly overlooked opportunities to do better in this regard. Nine ways to improve the integration of a sex and gender lens in health research proposals have been identified.

Background: Sex and gender influence individuals' psychology, but are often overlooked in psychological science. The sex and gender equity in research (SAGER) guidelines provide instruction for addressing sex and gender within five sections of a manuscript (i.e., title/abstract, introduction, methods, results, and discussion) (Heidari et al., Res Integr Peer Rev 1:1-9, 2016).

Methods: We examined whether the 89 journals published by the American Psychological Association provide explicit instruction for authors to address sex and gender within these five sections. Both authors reviewed the journal instructions to authors for the words "sex," and "gender," and noted explicit instruction pertaining to these five sections.

Results: Only 8 journals (9.0%) instructed authors to address sex/gender within the abstract, introduction, and/or methods sections. No journals instructed authors to address sex and gender in the results or discussion sections.

Conclusion: These journals could increase sex/gender equity and improve the reproducibility of psychological science by instructing authors to follow the SAGER guidelines.

Background: Inaccurate citations are erroneous quotations or instances of paraphrasing of previously published material that mislead readers about the claims of the cited source. They are often unaddressed due to underreporting, the inability of peer reviewers and editors to detect them, and editors' reluctance to publish corrections about them. In this paper, we propose a new tool that could be used to tackle their circulation.

Methods: We provide a review of available data about inaccurate citations and analytically explore current ways of reporting and dealing with these inaccuracies. Consequently, we make a distinction between publication (i.e., first occurrence) and circulation (i.e., reuse) of inaccurate citations. Sloppy reading of published items, literature ambiguity and insufficient quality control in the editorial process are identified as factors that contribute to the publication of inaccurate citations. However, reiteration or copy-pasting without checking the validity of citations, paralleled with lack of resources/motivation to report/correct inaccurate citations contribute to their circulation.

Results and discussion: We propose the development of an online annotation tool called "MyCites" as means with which to mark and map inaccurate citations. This tool allows ORCID users to annotate citations and alert authors (of the cited and citing articles) and also editors of journals where inaccurate citations are published. Each marked citation would travel with the digital version of the document (persistent identifiers) and be visible on websites that host peer-reviewed articles (journals' websites, Pubmed, etc.). In the future development of MyCites, challenges such as the conditions of correct/incorrect-ness and parties that should adjudicate that, and, the issue of dealing with incorrect reports need to be addressed.

Background: The rigor and integrity of the published research in nutrition studies has come into serious question in recent years. Concerns focus on the use of flexible data analysis practices and selective reporting and the failure of peer review journals to identify and correct these practices. In response, it has been proposed that journals employ editorial procedures designed to improve the transparency of published research.

Objective: The present study examines the adoption of editorial procedures designed to improve the reporting of empirical studies in the field of nutrition and dietetics research.

Design: The instructions for authors of 43 journals included in Quartiles 1 and 2 of the Clarivate Analytics' 2018 Journal Citation Report category Nutrition and Dietetics were reviewed. For journals that published original research, conflict of interest disclosure, recommendation of reporting guidelines, registration of clinical trials, registration of other types of studies, encouraging data sharing, and use of the Registered Reports were assessed. For journals that only published reviews, all of the procedures except clinical trial registration were assessed.

Results: Thirty-three journals published original research and 10 published only reviews. Conflict of interest disclosure was required by all 33 original research journals. Use of guidelines, trial registration and encouragement of data sharing were mentioned by 30, 27 and 25 journals, respectively. Registration of other studies was required by eight and none offered Registered Reports as a publication option at the time of the review. All 10 review journals required conflict of interest disclosure, four recommended data sharing and three the use of guidelines. None mentioned the other two procedures.

Conclusions: While nutrition journals have adopted a number of procedures designed to improve the reporting of research findings, their limited effects likely result from the mechanisms through which they influence analytic flexibility and selective reporting and the extent to which they are properly implemented and enforced by journals.

Background: Triggered by a series of controversies and diversifying expectations of editorial practices, several innovative peer review procedures and supporting technologies have been proposed. However, adoption of these new initiatives seems slow. This raises questions about the wider conditions for peer review change and about the considerations that inform decisions to innovate. We set out to study the structure of commercial publishers' editorial process, to reveal how the benefits of peer review innovations are understood, and to describe the considerations that inform the implementation of innovations.

Methods: We carried out field visits to the editorial office of two large academic publishers housing the editorial staff of several hundreds of journals, to study their editorial process, and interviewed editors not affiliated with large publishers. Field notes were transcribed and analysed using coding software.

Results: At the publishers we analysed, the decision-making structure seems to show both clear patterns of hierarchy and layering of the different editorial practices. While information about new initiatives circulates widely, their implementation depends on assessment of stakeholder's wishes, impact on reputation, efficiency and implementation costs, with final decisions left to managers at the top of the internal hierarchy. Main tensions arise between commercial and substantial arguments. The editorial process is closely connected to commercial practices of creating business value, and the very specific terms in which business value is understood, such as reputation considerations and the urge to increase efficiency. Journals independent of large commercial publishers tend to have less hierarchically structured processes, report more flexibility to implement innovations, and to a greater extent decouple commercial and editorial perspectives.

Conclusion: Our study demonstrates that peer review innovations are partly to be understood in light of commercial considerations related to reputation, efficiency and implementations costs. These arguments extend beyond previously studied topics in publishing economics, including publishers' choice for business or publication models and reach into the very heart of the editorial and peer review process.

Background: The process of peer-review in academia has attracted criticism surrounding issues of bias, fairness, and professionalism; however, frequency of occurrence of such comments is unknown.

Methods: We evaluated 1491 sets of reviewer comments from the fields of "Ecology and Evolution" and "Behavioural Medicine," of which 920 were retrieved from the online review repository Publons and 571 were obtained from six early career investigators. Comment sets were coded for the occurrence of "unprofessional comments" and "incomplete, inaccurate or unsubstantiated critiques" using an a-prior rubric based on our published research. Results are presented as absolute numbers and percentages.

Results: Overall, 12% (179) of comment sets included at least one unprofessional comment towards the author or their work, and 41% (611) contained incomplete, inaccurate of unsubstantiated critiques (IIUC).

Conclusions: The large number of unprofessional comments, and IIUCs observed could heighten psychological distress among investigators, particularly those at an early stage in their career. We suggest that development and adherence to a universally agreed upon reviewer code of conduct is necessary to improve the quality and professional experience of peer review.

Background: The trend toward open science increases the pressure on authors to provide access to the source code and data they used to compute the results reported in their scientific papers. Since sharing materials reproducibly is challenging, several projects have developed solutions to support the release of executable analyses alongside articles.

Methods: We reviewed 11 applications that can assist researchers in adhering to reproducibility principles. The applications were found through a literature search and interactions with the reproducible research community. An application was included in our analysis if it (i) was actively maintained at the time the data for this paper was collected, (ii) supports the publication of executable code and data, (iii) is connected to the scholarly publication process. By investigating the software documentation and published articles, we compared the applications across 19 criteria, such as deployment options and features that support authors in creating and readers in studying executable papers.

Results: From the 11 applications, eight allow publishers to self-host the system for free, whereas three provide paid services. Authors can submit an executable analysis using Jupyter Notebooks or R Markdown documents (10 applications support these formats). All approaches provide features to assist readers in studying the materials, e.g., one-click reproducible results or tools for manipulating the analysis parameters. Six applications allow for modifying materials after publication.

Conclusions: The applications support authors to publish reproducible research predominantly with literate programming. Concerning readers, most applications provide user interfaces to inspect and manipulate the computational analysis. The next step is to investigate the gaps identified in this review, such as the costs publishers have to expect when hosting an application, the consideration of sensitive data, and impacts on the review process.

Background: Our aim is to highlight the benefits and limitations of open and non-anonymized peer review. Our argument is based on the literature and on responses to a survey on the reviewing process of alt.chi, a more or less open review track within the so-called Computer Human Interaction (CHI) conference, the predominant conference in the field of human-computer interaction. This track currently is the only implementation of an open peer review process in the field of human-computer interaction while, with the recent increase in interest in open scientific practices, open review is now being considered and used in other fields.

Methods: We ran an online survey with 30 responses from alt.chi authors and reviewers, collecting quantitative data using multiple-choice questions and Likert scales. Qualitative data were collected using open questions.

Results: Our main quantitative result is that respondents are more positive to open and non-anonymous reviewing for alt.chi than for other parts of the CHI conference. The qualitative data specifically highlight the benefits of open and transparent academic discussions. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

Conclusion: While the benefits are quite clear and the system is generally well-liked by alt.chi participants, they remain reluctant to see it used in other venues. This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation.