Librarians at the Center of Peer Review Training: Increasing Collaboration Among Scholarly Communication Stakeholders

Abstract

My background in scholarly publishing and peer review brought me to the United States to teach Scholarly Communication at the School of Information Sciences of the University of Illinois Urbana-Champaign (UIUC). Offering a course dedicated to the scholarly communication process is a fairly new endeavour for iSchools. In 2022, I was entrusted with a Scholarly Communication course composed of masters students in the Library & Information Science and the PhD in Information Sciences programs. The Scholarly Communication course was created by Prof. Dr. Maria Bonn, who is one of the authors of The Scholarly Communication Notebook (https://oercommons.org/hubs/SCN), ‘an active, inclusive, empowered community of practice for teaching scholarly communication to emerging librarians’. As Professor Bonn used to do with her library science students at the University of Illinois (Bonn 2014), in one of the first class sessions the students and I spent some time browsing job advertisements for Scholarly Communication librarians published on the ALA Job List (https://joblist.ala.org/).

In these job advertisements posted from 2006 to 2014, we found positions titled ‘Scholarly Communications Librarian’, or others with a more specific focus such as ‘Copyright Librarian’, ‘Data Librarian’, or ‘Open Access and Intellectual Property Librarian’, and analysed the prevalence of scholarly communications terms, concepts, and activities, as identified by Finlay et al. (2015, 21), namely ‘instruction; digital products; outreach and liaison work; publishing; repositories; copyright, policy, and licensing; preservation; metadata, standards, and data management; and open access’. My students and I were particularly intrigued by the absence of peer review as one of the concepts requested of a scholarly communication position, since peer review is clearly represented in the schematic of the scholarly communication cycle created by the Association of College and Research Libraries (Figure 1, ACRL).

Furthermore, the definition of scholarly communication librarianship of the ACRL includes the evaluation of scholarly work in peer-reviewed. journals: ‘the system through which research and other scholarly writings are created, evaluated for quality, disseminated to the scholarly community, and preserved for future use. The system includes both formal means of communication, such as publication in peer-reviewed journals, and informal channels, such as electronic listservs’ (ACRL 2003). In addition, as Meadows (1998, preface) says ‘Communication lies at the heart of research. It is as vital for research as the actual investigation itself, for research cannot properly claim that name until it has been scrutinized and accepted by colleagues’. This statement highlights that peer review is a fundamental aspect of the communication of research in scholarly publishing. Thus, I asked my students: Why is peer review not listed as a required competency for Scholarly Communication Librarians? Do you think that one day we will have a Peer Review Librarian? What can librarians do to support education in peer review?

Motivated by these ideas, I (a peer review researcher with a PhD in Information Sciences, but no library science degree) designed and delivered an in-person and online peer review workshop curriculum for the Savvy Researcher workshop series sponsored by the University Library and supported by the School of Information Sciences at UIUC. The Savvy Researcher focuses on a variety of topics related to the lifecycle of research including advanced research and information management topics, for example, introduction to library resources, using tools to perform qualitative and quantitative research, data management, understanding copyright, and much more. All campus affiliates are invited to participate in the synchronous in-person and online sessions, including undergraduate and graduate students, campus staff, researchers, and faculty. Many of the workshops are concept-based around an assortment of scholarly communication topics, while others focus on a particular tool or library resource. The workshops are taught by librarians, graduate students in information science, and several campus partners such as the Center for Innovation in Teaching and Learning, the Writers Workshop, and the UI Press.

In this article, I introduce the six-part peer review series ‘Peer Review in Scholarly Journals: History, People, and Models’ delivered at UIUC, and the lessons learned from the experience of conducting them. Finally, my coauthors and I offer recommendations on how peer review training could be improved and discuss the fundamental role of librarians in supporting peer review training.

The six-part peer review series ‘Peer Review in Scholarly Journals: History, People, and Models’ was presented as a pilot offering during the fall semester and took place at the University Library from October 11, 2023 to November 15, 2023 (Figure 2). These new workshops were included on the Library's calendar and advertised broadly across campus throughout the fall semester. According to research programs liaison and instruction librarian and associate professor Merinda Kaye Hensley, who supported this initiative, the Peer Review Workshop Series was the first of its kind offered by the University Library.

The purpose of the series was to educate a diverse audience and raise awareness regarding the goals and challenges of the peer review process implemented by scholarly journals, including current models and the role of reviewers. Since the sessions were not designed around any specific discipline, myriad examples of peer review were highlighted. The workshops were held for a duration of 60–90 min. Please see the Appendix A for details regarding the content of the workshops.

Despite our efforts to promote the peer review workshops and be inclusive by opening registration to any member of the UIUC community, the first set of workshops had low attendance (Figure 3A). In the past 15 years, this is not an unusual occurrence for newly added workshops. Of the 26 total registrations across 6 scheduled sessions, only 10 registrants attended the workshops (Figure 3B). All attendees were graduate students with little to no experience with publishing and/or reviewing papers. The single exception was an editor that reached out directly by email to schedule a one-on-one appointment via Zoom to discuss the last workshop of the series, ‘Understanding and Implementing Open Peer Review.’

However, even with low attendance, offering workshops and sharing their descriptions across campus is a powerful tool to better inform campus stakeholders about the wide variety of topics, including complex scholarly communication issues, where librarians can help them with sharing, presenting, and archiving their scholarly work.

We expected to see increased attendance at future workshops, as word-of-mouth spread. We also conducted a second set of non-consecutive sessions in Spring and Fall 2024. These were training sessions for undergraduate students working for an undergraduate research journal. As there was limited interest in the open peer review session previously, this was not conducted. Thus, in the undergraduate sessions, we had a total of 46 attendees split between five sessions (Figure 4).

These sessions were better attended compared to the open Savvy Workshop ones. Even for an audience actively engaged as editorial staff and peer reviewers in a journal, interest waned as the specificity of themes progressed, and attendance at the last session was reduced to a third of the attendance of the introductory one.

To evaluate the workshops, an auto generated form was sent to all registrants two days after each workshop. The workshop, ‘Reviewers and Their Biases’ was the only session that received feedback from an attendee. In the form, when asked ‘Did the workshop cover the content you expected?’ the attendee replied, ‘Sort of. The workshop helped show how to identify bias, but I was looking for techniques to address bias as well. For example, specific actions or ways I can avoid it when I review papers. Overall the workshop provided a good overview of what bias is and common types of biases during the peer-review process.’ When asked about their preference for the format of workshops, this attendee answered ‘in-person’. Additionally, one faculty member sent an email to the workshop instructor stating that one of their students attended a workshop and as a faculty member they would recommend it to their students.

After this first experience in offering a peer review workshop series in an academic library, my coauthors and I reflected on how we could improve this initiative to better serve our campus stakeholders. Our improvement strategies for the pilot series are based on the identification of issues and questions that arose and how they could be addressed, namely by updating workshop content according to information needs of researchers at different career stages, providing additional opportunities for hands-on experience, offering workshops online and asynchronously, inviting scholarly communications stakeholders to speak of their perspective on the peer review process, identifying avenues to engage with undergraduate researchers, and possibly rethinking workshop titles to be more descriptive and actionable.

Librarians are also in a strategic position to teach and support learning in scholarly communication because they can interact with different campus stakeholders. Librarians can identify the information needs of campus stakeholders pertaining to the peer review process and other scholarly communication topics through community analysis. In a community analysis of the library and its environment, data should be collected on individuals, groups, agencies, and lifestyle (Grover, Greer, and Agada 2010). Based on the collected data, librarians can then offer peer review workshops at the library, and also identify key partners inside and outside the university. One example at the UIUC would be the Undergraduate Research Apprenticeship Program (URAP), which ‘offers undergraduate students with little or no research experience the opportunity to work with graduate students and postdoctoral scholars on their research projects’ (https://undergradresearch.illinois.edu/programs/urap.html). Offering these peer review workshops to undergraduates may seem out of scope, but there is increasing pressure on students in all disciplines to share their scholarly work through publication, whether through student-led publications or professional association journals. Moreover, UIUC supports seven undergraduate research journals where undergraduate student researchers can publish their work (https://ugresearchjournals.illinois.edu/). Undergraduates who are involved in student journals are more likely to graduate, and show increased knowledge of the research process (Weiner and Watkinson 2014). In addition, previous research has advocated for the inclusion of peer review in undergraduate science education, exploring how the engagement of undergraduate students in the review of preprint review can enhance scientific literacy and develop their identity as researchers (Otto et al. 2023).

Furthermore, librarians can also train faculty on how to peer review a paper. A study on co-reviewing and ghostwriting of manuscripts' peer review has questioned how much faculty are able to advise their mentees: ‘Training provided by one's PI may benefit from a personalized teaching relationship but depends on the PI's own training.’ (McDowell et al. 2019, 10). Librarians can also support faculty with resources to teach peer review in the classroom, or deliver peer review workshops following faculty syllabus' learning goals. Additionally, librarians can independently support the process of learning peer review by offering one-on-one consultations within the library.

By partnering with other scholarly communication stakeholders outside of academia, librarians are uniquely equipped to effectively create programs and services that can increase awareness and interest in scholarly publishing. In this sense, we suggest that librarians could develop partnerships with platforms such as PREreview, to provide hands-on experience in peer review to campus stakeholders, and to support the recognition of reviewers at any stage of their career. Regarding the recognition of peer reviewers, one suggestion would be to partner with recognition platforms such as Reviewer Credits. Researchers engaged with scholarly communication research should also partner with librarians to improve peer review training by including them in their research process and giving them leadership roles in peer review training. Researchers and librarians can work together to design data and evidence-based training in peer review. This partnership could help to increase diversity by reframing future workshops as unveiling the ‘hidden curriculum in higher education’ (Margolis 2001), in other words, how academic environments reproduce hierarchies of race, gender, and class that may determine who is and out and science/research.

In the sixth module of the open educational resource (OER), Librarians and Peer Review, Ford (2022) states that ‘Library workers are in an interesting position when it comes to peer review. We may be teaching students what peer review is, or we may be helping students learn techniques on how to identify peer-reviewed literature. We may be cataloging materials or running publishing programs that include peer-review literature.’ In 2013, a white paper published by ACRL has approached how information literacy and scholarly communication overlap and has pointed out that: ‘Every librarian is an academic environment is a teacher’ and ‘All roles in an academic library are impacted and altered by the changing nature of scholarly communication and the evolution of the dissemination of knowledge.’ (ACRL 2013, 4). Bonn (2014, 132) argues that ‘scholarly communication literacy has become a core competency for academic librarians’. However, although scholarly communication is a very important competency for librarians, there is lack of formal training on scholarly communication topics; thus, to include librarians in peer review education and promote future collaboration among scholarly communication stakeholders, LIS programs should include scholarly communication topics into their curriculum (Bonn, Cross, and Bolick 2020).

Through the six-part series ‘Peer Review in Scholarly Journals: History, People, and Models’ we learned that, to better serve campus stakeholders regarding education in peer review, it is important to adapt the training to the information needs of researchers and editors at different stages in their careers. Furthermore, organising effective peer review workshops requires skills in event planning, taking into consideration the best time and day to deliver the workshops, their format and duration. It is also increasingly important to make sure that instruction in an academic environment is reaching everyone it is intended for, including distance learners and students with disabilities.

Librarians have been called to advocate for open access, we hope they can be called to advocate for peer review training as well. Will we have a Peer Review Librarian one day? If so, how the role of the Peer Review Librarian will look like and how it will differ from other scholarly communication librarian positions? We will leave you with these questions. Anyway, we hope LIS programs include scholarly communication topics into their curriculum and prepare librarians to be at the centre of peer review training and other topics. We hope researchers and publishers build two-way partnerships including librarians in their research and publishing initiatives. By offering services, programs, and training in peer review based on community analysis data, survey data, scientific evidence and in collaboration with scholarly communication stakeholders outside the university, librarians have a key role in making academia and publishing more diverse, equitable, inclusive, and accessible.

Janaynne Carvalho do Amaral: conceptualization, methodology, investigation, project administration, writing original draft, writing – review and editing. Nicolene Sarich: investigation, writing – original draft, writing – review and editing. Merinda Kaye Hensley: writing – review and editing, formal analysis. Maria J. C. Machado: formal analysis, writing – review and editing, Visualization.

The authors declare no conflicts of interest.