ITEMS Corner Update: The Initial Steps in the ITEMS Development Process

Abstract

In the previous issue of Educational Measurement: Issues and Practice (EM:IP) I outlined the ten steps to authoring and producing a digital module for the Instructional Topics in Educational Measurement Series (ITEMS). In the current piece, I detail the first three steps: Step 1—Content Outline; Step 2—Content Development; and Step 3—Draft Review. After in-depth discussion of these three steps, I introduce the newest ITEMS module.

Prior to beginning the ten-step process, ITEMS module development starts with an initial meeting between myself (as editor) and the lead author(s). During this meeting, I discuss the development process in detail, showcasing what a final product looks like from the learners’ perspective in addition to a sneak-peek behind-the-scenes at what the final product looks like from the editorial perspective. After discussing the end product, the remaining conversation focuses on the 10-step process and the user-friendly templates to guide authors. The conversation concludes after coming to an agreement of the topic and general scope for the module.

Authors then independently work through a module outline template to refine the scope and sequencing of the module (Step 1). During this step, authors are encouraged to first specify their audience before setting the learning objectives of the module. Once learning objectives are set, authors are then tasked with determining the prerequisite knowledge for learners. In the next section of the template, authors outline the content and sequencing of the 4–6 sections of the module. Each section has its own learning objectives that map to the objectives of the module. One of the sections is a learner-focused interactive activity, whether it be a demonstration of software or a case study that is relevant to the content of the other sections. Once the outline is completed, the authors receive feedback to ensure adequate sequencing, feasibility of module development (e.g., covering a reasonable amount of content), and appropriateness for the audience. This is an example of the unique nature of ITEMS module development. Unlike most other publications, ITEMS module development consists of regular communication and feedback from the editor. Once the scope and outline of content have been agreed to, the authors move on to Step 2: Content Development.

For Step 2, authors are provided a slide deck template to assist in developing content consistent with the ITEMS format and brand. Using this slide deck, authors maintain creative flexibility by choosing among many slide layouts, each preprogrammed with consistent font, sizing, and color. Authors create individual slide decks for each section of the module, embedding media (e.g., pictures/figures) wherever necessary to assist learner understanding. At this stage, authors are not expected to record audio nor are they expected to put in animations. The primary focus for the authors is on the content, where the editorial team takes care of the rest. Depending on the topic, typical sections have between 10 and 15 slides, with authors planning to speak for 1 to 2 minutes during each slide. Typically, authors request feedback after developing the content for one section to confirm an appropriate balance of text, graphics, and figures. During the content development step, authors have found it valuable to write detailed notes, either via bullet points or an actual script to assist with audio recording later. The draft of each section is then reviewed by the editorial team (Step 3) where Step 2 and Step 3 become iterative until both the author(s) and editor agree on the work. After the slides are finalized, the authors may opt for an external review or choose to record audio and seek an external review later in the process. The pros and cons to seeking a review at each time is discussed with the editor.

In future issues of EM:IP, I will detail the remaining steps of the ITEMS module development process. The purposes of this exposition are to (1) familiarize readers, learners, and potential authors of the development process of this atypical publication; (2) promote the behind-the-scenes detailed work completed by the authors of these modules; and (3) garner interest in potential authors by showcasing the rigorous, yet guided development process. ITEMS modules have incredible utility to many audiences (e.g., graduate students and faculty, clients, professionals inside and outside the educational measurement field). It is through the volunteer contributions by authors, the editorial team, and reviewers that such wonderful products are available.

Finally, I am excited to announce the publication of the newest module, Digital Module 32: Understanding and Mitigating the Impact of Low Effort on Common Uses of Test and Survey Scores by Dr. James Soland. In this six-part module, Dr. Soland argues why we should worry about low examinee effort, describes how it may be defined and operationalized in testing contexts, details how to identify noneffortful responses at the item level on tests, and discusses how these methods differ in self-report contexts. He concludes the module with an activity by providing Stata syntax to identify and address rapid guessing behavior.

As always, I encourage anyone interested in authoring a module or wanting to become a part of the ITEMS team to reach out to me at [email protected].