Analysis of Student Grades Before and After Adopting POGIL

Abstract

From 2017-2022, our research project supported faculty at higher-ed institutions in the United States to adopt POGIL in CS1 courses. The faculty participated in summer workshops and mentoring groups during the academic year. At the end of each term, the faculty submitted a summary of their students' grades to the research team. This paper presents a Bayesian analysis of the student grades using a hierarchical ordinal logistic regression model. The data included the number of A, B, C, D, F, and W grades, disaggregated by gender and race, for all students enrolled in the course. In addition to each POGIL term, faculty submitted grades for one or two previous terms when they taught the same course without POGIL. Most faculty observed an improvement in student pass rates in the second and third term after they began teaching with POGIL. We present detailed visualizations of grade distributions from 25 faculty, along with the results of the statistical analysis. Our model suggests that CS1 faculty adopting POGIL can expect to see a modest increase of A grades and a modest decrease of DFW grades. However, the grades of Black, Hispanic, and Indigenous students decreased slightly, especially in the first term faculty taught with POGIL. The results of this study demonstrate the importance of gender and racial analysis in evaluating pedagogical approaches.