Research integrity and peer review最新文献

Background: The Health Research Council of New Zealand is the first major government funding agency to use a lottery to allocate research funding for their Explorer Grant scheme. This is a somewhat controversial approach because, despite the documented problems of peer review, many researchers believe that funding should be allocated solely using peer review, and peer review is used almost ubiquitously by funding agencies around the world. Given the rarity of alternative funding schemes, there is interest in hearing from the first cohort of researchers to ever experience a lottery. Additionally, the Health Research Council of New Zealand wanted to hear from applicants about the acceptability of the randomisation process and anonymity of applicants.

Methods: This paper presents the results of a survey of Health Research Council applicants from 2013 to 2019. The survey asked about the acceptability of using a lottery and if the lottery meant researchers took a different approach to their application.

Results: The overall response rate was 39% (126 of 325 invites), with 30% (76 of 251) from applicants in the years 2013 to 2018, and 68% (50 of 74) for those in the year 2019 who were not aware of the funding result. There was agreement that randomisation is an acceptable method for allocating Explorer Grant funds with 63% (n = 79) in favour and 25% (n = 32) against. There was less support for allocating funds randomly for other grant types with only 40% (n = 50) in favour and 37% (n = 46) against. Support for a lottery was higher amongst those that had won funding. Multiple respondents stated that they supported a lottery when ineligible applications had been excluded and outstanding applications funded, so that the remaining applications were truly equal. Most applicants reported that the lottery did not change the time they spent preparing their application.

Conclusions: The Health Research Council's experience through the Explorer Grant scheme supports further uptake of a modified lottery.

Background: Descriptive studies examining publication rates and citation counts demonstrate a geographic skew toward high-income countries (HIC), and research from low- or middle-income countries (LMICs) is generally underrepresented. This has been suggested to be due in part to reviewers' and editors' preference toward HIC sources; however, in the absence of controlled studies, it is impossible to assert whether there is bias or whether variations in the quality or relevance of the articles being reviewed explains the geographic divide. This study synthesizes the evidence from randomized and controlled studies that explore geographic bias in the peer review process.

Methods: A systematic review was conducted to identify research studies that explicitly explore the role of geographic bias in the assessment of the quality of research articles. Only randomized and controlled studies were included in the review. Five databases were searched to locate relevant articles. A narrative synthesis of included articles was performed to identify common findings.

Results: The systematic literature search yielded 3501 titles from which 12 full texts were reviewed, and a further eight were identified through searching reference lists of the full texts. Of these articles, only three were randomized and controlled studies that examined variants of geographic bias. One study found that abstracts attributed to HIC sources elicited a higher review score regarding relevance of the research and likelihood to recommend the research to a colleague, than did abstracts attributed to LIC sources. Another study found that the predicted odds of acceptance for a submission to a computer science conference were statistically significantly higher for submissions from a "Top University." Two of the studies showed the presence of geographic bias between articles from "high" or "low" prestige institutions.

Conclusions: Two of the three included studies identified that geographic bias in some form was impacting on peer review; however, further robust, experimental evidence is needed to adequately inform practice surrounding this topic. Reviewers and researchers should nonetheless be aware of whether author and institutional characteristics are interfering in their judgement of research.

Objectives: To compare changes in the number and amount of payments received by orthopedic and non-orthopedic surgeons from industry between 2014 and 2017.

Methods: Using the Centers for Medicare and Medicaid Services (CMS) Open Payment database from 2014 to 2017, we conducted a retrospective cohort study of industry payments to surgeons, including general payments and research payments.

Results: Among orthopedic surgeons, the total number of general payments decreased from 248,698 in 2014 to 241,966 in 2017, but their total value increased from $97.1 million in 2014 to $110.2 million in 2017. Among non-orthopedic surgeons, the total number decreased from 604,884 in 2014 to 582,490 in 2017, while the total value remained stable at approximately $159 million. Between 2014 and 2017, there was a differential increase in the median number of general payments received by non-orthopedic when compared to orthopedic surgeons (incidence rate ratio, 1.09; 95% CI, 1.08-1.09; p < 0.001), but a differential decline in the median value of general payments (- 8.9%; 95% CI, - 9.5%, - 8.4%; p < 0.001). Findings were consistent when stratified by nature of payment. In contrast, between 2014 and 2017, there was neither a differential change in the median number nor median value of research payments received by non-orthopedics.

Conclusion: Examination of a natural experiment of prior public disclosure of payments to orthopedic surgeons suggests that the Physician Payment Sunshine Act was associated with an increase in the number, but a decline in the value, of general payments received by non-orthopedic surgeons, but not on research payments received.

Background: Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called "spin". In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin.

Methods: Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of "spin" related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups.

Discussion: We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.

Background: The Freedom of Information Act (FOIA) provides access to unreleased government records that can be used to enhance the transparency and integrity of biomedical research. We characterized FOIA requests to Department of Health and Human Services (HHS) agencies, including request outcomes, processing times, backlogs, and costs.

Methods: Using HHS FOIA annual reports, we extracted data on the number of FOIA requests received and processed by HHS agencies between 2008 and 2017, as well as request outcomes. Processing times were reported in three time increments, < 1-20, 21-60, or 61+ days, and trends in backlog status were also described. Information about costs and fees collected were aggregated.

Results: Between 2008 and 2017, 69.6% of 530,094 HHS FOIA requests were received by the Centers for Medicare and Medicaid Services (CMS), 18.9% by the Food and Drug Administration (FDA), and 11.6% by all other HHS agencies. During this period, CMS processed 374,728 requests, FDA 114,938, and other HHS agencies 61,890. CMS and FDA reduced backlogged requests by 9396 (89.7%) and 4289 (65.3%), respectively, leaving backlogs of 1081 and 2279 requests at the end of 2017. CMS fully or partially granted 60.3% of requests whereas FDA fully or partially granted 72.4%. Of all requests to CMS, 82.0% were considered simple and 18.0% complex; 82.2% of simple requests and 54.9% of complex requests were processed in 20 days, and 5.6% and 29.9% were processed in 61+ days. In contrast, 60.2% of requests to FDA were considered simple and 39.8% complex; 28.8% of simple requests and 9.0% of complex requests were processed in 20 days, and 58.3% and 81.5% were processed in 61+ days. The costs to HHS associated with FOIA requests totaled $446.4 million ($809 per processed request), increasing from $28.1 million ($423 per request) in 2008 to $53.3 million ($1544 per request) in 2017. In total, HHS collected $8.5 million in fees (1.9% of total costs).

Conclusions: FOIA is frequently used to obtain information about HHS and its agencies. With growing costs, minimal fees collected, and lengthy processing times, HHS agencies' FOIA programs might be made more efficient through greater proactive record disclosure.

Background: Developing a comprehensive, reproducible literature search is the basis for a high-quality systematic review (SR). Librarians and information professionals, as expert searchers, can improve the quality of systematic review searches, methodology, and reporting. Likewise, journal editors and authors often seek to improve the quality of published SRs and other evidence syntheses through peer review. Health sciences librarians contribute to systematic review production but little is known about their involvement in peer reviewing SR manuscripts.

Methods: This survey aimed to assess how frequently librarians are asked to peer review systematic review manuscripts and to determine characteristics associated with those invited to review. The survey was distributed to a purposive sample through three health sciences information professional listservs.

Results: There were 291 complete survey responses. Results indicated that 22% (n = 63) of respondents had been asked by journal editors to peer review systematic review or meta-analysis manuscripts. Of the 78% (n = 228) of respondents who had not already been asked, 54% (n = 122) would peer review, and 41% (n = 93) might peer review. Only 4% (n = 9) would not review a manuscript. Respondents had peer reviewed manuscripts for 38 unique journals and believed they were asked because of their professional expertise. Of respondents who had declined to peer review (32%, n = 20), the most common explanation was "not enough time" (60%, n = 12) followed by "lack of expertise" (50%, n = 10).The vast majority of respondents (95%, n = 40) had "rejected or recommended a revision of a manuscript| after peer review. They based their decision on the "search methodology" (57%, n = 36), "search write-up" (46%, n = 29), or "entire article" (54%, n = 34). Those who selected "other" (37%, n = 23) listed a variety of reasons for rejection, including problems or errors in the PRISMA flow diagram; tables of included, excluded, and ongoing studies; data extraction; reporting; and pooling methods.

Conclusions: Despite being experts in conducting literature searches and supporting SR teams through the review process, few librarians have been asked to review SR manuscripts, or even just search strategies; yet many are willing to provide this service. Editors should involve experienced librarians with peer review and we suggest some strategies to consider.