Students perceive crossword puzzles as enjoyable. In addition to students' perceptions, crossword puzzles actually improve knowledge retention. However, crossword puzzles increased exam scores for some students but not others. Recommendations have been made for students to create puzzles for their classmates to complete with the rationale that students are encouraged to research and understand the material in order to write meaningful clues for the puzzle. While students enjoy creating their own crossword puzzles, the association between students creating crossword puzzles and knowledge retention is unknown. The purpose of this project was to determine if creating crossword puzzles and completing peers' crossword puzzles were associated with improved knowledge retention indicated by higher scores on quizzes. Students in a research course from two institutions across three semesters had the option each week to upload a blank puzzle they created prior to completing each other's puzzles and taking a quiz. Quiz scores were compared between those who did versus did not create their own puzzles and complete their peers' puzzles. Results varied by institution and programs, as well as the same program within the same institution but different semesters. Results highlight the importance of moving beyond student perceptions and towards assessing knowledge retention while taking into consideration institution, program, and semester.