Pub Date : 2023-01-01Epub Date: 2022-07-07DOI: 10.1080/19345747.2022.2093298
Daniela Alvarez-Vargas, Sirui Wan, Lynn S Fuchs, Alice Klein, Drew H Bailey
Despite policy relevance, longer-term evaluations of educational interventions are relatively rare. A common approach to this problem has been to rely on longitudinal research to determine targets for intervention by looking at the correlation between children's early skills (e.g., preschool numeracy) and medium-term outcomes (e.g., first-grade math achievement). However, this approach has sometimes over-or under-predicted the long-term effects (e.g., 5th-grade math achievement) of successfully improving early math skills. Using a within-study comparison design, we assess various approaches to forecasting medium-term impacts of early math skill-building interventions. The most accurate forecasts were obtained when including comprehensive baseline controls and using a combination of conceptually proximal and distal short-term outcomes (in the nonexperimental longitudinal data). Researchers can use our approach to establish a set of designs and analyses to predict the impacts of their interventions up to two years post-treatment. The approach can also be applied to power analyses, model checking, and theory revisions to understand mechanisms contributing to medium-term outcomes.
{"title":"Design and Analytic Features for Reducing Biases in Skill-Building Intervention Impact Forecasts.","authors":"Daniela Alvarez-Vargas, Sirui Wan, Lynn S Fuchs, Alice Klein, Drew H Bailey","doi":"10.1080/19345747.2022.2093298","DOIUrl":"10.1080/19345747.2022.2093298","url":null,"abstract":"<p><p>Despite policy relevance, longer-term evaluations of educational interventions are relatively rare. A common approach to this problem has been to rely on longitudinal research to determine targets for intervention by looking at the correlation between children's early skills (e.g., preschool numeracy) and medium-term outcomes (e.g., first-grade math achievement). However, this approach has sometimes over-or under-predicted the long-term effects (e.g., 5th-grade math achievement) of successfully improving early math skills. Using a within-study comparison design, we assess various approaches to forecasting medium-term impacts of early math skill-building interventions. The most accurate forecasts were obtained when including comprehensive baseline controls and using a combination of conceptually proximal and distal short-term outcomes (in the nonexperimental longitudinal data). Researchers can use our approach to establish a set of designs and analyses to predict the impacts of their interventions up to two years post-treatment. The approach can also be applied to power analyses, model checking, and theory revisions to understand mechanisms contributing to medium-term outcomes.</p>","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 2","pages":"271-299"},"PeriodicalIF":1.8,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10181816/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9488266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.1080/19345747.2022.2131660
Carly Lupton-Smith, Elena Badillo Goicoechea, Megan Collins, Justin Lessler, M Kate Grabowski, Elizabeth A Stuart
The academic, socioemotional, and health impacts of school policies throughout the COVID-19 pandemic have been a source of many questions that require accurate information about the extent of onsite schooling occurring. This article investigates school operational status datasets during the pandemic, comparing (1) self-report data collected nationally on the household level through a Facebook-based survey, (2) county-level school policy data, and (3) a school-level closure status dataset based on phone GPS tracking. The percentage of any onsite instruction within states and counties are compared across datasets from December 2020 to May 2021. Sources were relatively consistent at the state level and for large counties, but key differences were revealed between units of measurement, showing differences between policy and household decisions surrounding children's schooling experiences. The consistency levels across sources support the usage of each of the school policy sources to answer questions about the educational experiences, factors, and impacts related to K-12 education across the nation during the pandemic, but it remains vital to think critically as to which unit of measurement is most relevant to targeted research questions.
{"title":"Consistency between Household and County Measures of Onsite Schooling during the COVID-19 Pandemic.","authors":"Carly Lupton-Smith, Elena Badillo Goicoechea, Megan Collins, Justin Lessler, M Kate Grabowski, Elizabeth A Stuart","doi":"10.1080/19345747.2022.2131660","DOIUrl":"https://doi.org/10.1080/19345747.2022.2131660","url":null,"abstract":"<p><p>The academic, socioemotional, and health impacts of school policies throughout the COVID-19 pandemic have been a source of many questions that require accurate information about the extent of onsite schooling occurring. This article investigates school operational status datasets during the pandemic, comparing (1) self-report data collected nationally on the household level through a Facebook-based survey, (2) county-level school policy data, and (3) a school-level closure status dataset based on phone GPS tracking. The percentage of any onsite instruction within states and counties are compared across datasets from December 2020 to May 2021. Sources were relatively consistent at the state level and for large counties, but key differences were revealed between units of measurement, showing differences between policy and household decisions surrounding children's schooling experiences. The consistency levels across sources support the usage of each of the school policy sources to answer questions about the educational experiences, factors, and impacts related to K-12 education across the nation during the pandemic, but it remains vital to think critically as to which unit of measurement is most relevant to targeted research questions.</p>","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 3","pages":"419-441"},"PeriodicalIF":1.8,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10441621/pdf/nihms-1866597.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10053232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-20DOI: 10.1080/19345747.2022.2141164
Terri J Sabol, D. McCoy, Kathryn E. Gonzalez, Emily C. Hanno, Andrea Busby, Wendy S. Wei, J. Downer
Abstract The current study examined the contexts in which a preschool teacher professional development intervention was more versus less effective in improving student outcomes. We used a fixed intercept, random coefficient (FIRC) modeling approach to explore the contextual characteristics inside and outside school walls that predicted differential effectiveness of the NCRECE teacher professional development experimental intervention on students’ language and literacy skills, executive functioning, and learning behaviors (N = 1,230 children and 317 teachers within 192 center-based preschools; the majority of preschools were publicly funded). Overall, most characteristics inside and outside school walls were not associated with treatment impact on student outcomes. The only exception is that some characteristics outside school walls were associated with treatment impact on students’ executive functioning skills, where the professional development intervention (course and/or coaching) had a more positive effect on children’s executive functioning in preschools located in neighborhoods with lower levels of institutional resources and lower structural/social processes. Findings are discussed in terms of implications for large-scale teacher professional development programs and systems.
{"title":"Contextual Characteristics Inside and Outside of School Walls as Predictors of Differential Effectiveness in Teacher Professional Development","authors":"Terri J Sabol, D. McCoy, Kathryn E. Gonzalez, Emily C. Hanno, Andrea Busby, Wendy S. Wei, J. Downer","doi":"10.1080/19345747.2022.2141164","DOIUrl":"https://doi.org/10.1080/19345747.2022.2141164","url":null,"abstract":"Abstract The current study examined the contexts in which a preschool teacher professional development intervention was more versus less effective in improving student outcomes. We used a fixed intercept, random coefficient (FIRC) modeling approach to explore the contextual characteristics inside and outside school walls that predicted differential effectiveness of the NCRECE teacher professional development experimental intervention on students’ language and literacy skills, executive functioning, and learning behaviors (N = 1,230 children and 317 teachers within 192 center-based preschools; the majority of preschools were publicly funded). Overall, most characteristics inside and outside school walls were not associated with treatment impact on student outcomes. The only exception is that some characteristics outside school walls were associated with treatment impact on students’ executive functioning skills, where the professional development intervention (course and/or coaching) had a more positive effect on children’s executive functioning in preschools located in neighborhoods with lower levels of institutional resources and lower structural/social processes. Findings are discussed in terms of implications for large-scale teacher professional development programs and systems.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"1 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41547801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-20DOI: 10.1080/19345747.2022.2151954
Elizabeth S. Park, Di Xu
Extensive literature emphasizes the promise of active learning instruction in engaging students in college classrooms. Accordingly, faculty professional development (PD) programs on active learning have become increasingly popular in postsecondary institutions; yet, quantitative evidence on the effectiveness of these programs is limited. Using administrative data and an individual fixed effects approach, we estimate the causal effect of an active learning PD program at a large public institution. Findings indicate that the training improved course performance and subsequent persistence in the same field. Further analysis of classroom observation data identifies a positive association between training and active learning teaching practices. These findings suggest that active learning PD, when implemented with high fidelity, has the potential to improve teaching quality and student outcomes.
{"title":"The Effect of Active Learning Professional Development Training on College Students’ Academic Outcomes","authors":"Elizabeth S. Park, Di Xu","doi":"10.1080/19345747.2022.2151954","DOIUrl":"https://doi.org/10.1080/19345747.2022.2151954","url":null,"abstract":"Extensive literature emphasizes the promise of active learning instruction in engaging students in college classrooms. Accordingly, faculty professional development (PD) programs on active learning have become increasingly popular in postsecondary institutions; yet, quantitative evidence on the effectiveness of these programs is limited. Using administrative data and an individual fixed effects approach, we estimate the causal effect of an active learning PD program at a large public institution. Findings indicate that the training improved course performance and subsequent persistence in the same field. Further analysis of classroom observation data identifies a positive association between training and active learning teaching practices. These findings suggest that active learning PD, when implemented with high fidelity, has the potential to improve teaching quality and student outcomes.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46033914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-19DOI: 10.1080/19345747.2022.2142177
W. Li, N. Dong, Rebecca Maynarad, Jessaca K. Spybrook, Ben Kelcey
Abstract Cluster randomized trials (CRTs) are commonly used to evaluate educational interventions, particularly their effectiveness. Recently there has been greater emphasis on using these trials to explore cost-effectiveness. However, methods for establishing the power of cluster randomized cost-effectiveness trials (CRCETs) are limited. This study develops power computation formulas and statistical software to help researchers plan two- and three-level CRCETs. We illustrate the application of our formulas and software for the designs of CRCETs and discuss the influence of sample size, nesting effects, covariates, and the covariance between cost and effectiveness measures on the statistical power of cost-effectiveness estimates.
{"title":"Experimental Design and Statistical Power for Cluster Randomized Cost-Effectiveness Trials","authors":"W. Li, N. Dong, Rebecca Maynarad, Jessaca K. Spybrook, Ben Kelcey","doi":"10.1080/19345747.2022.2142177","DOIUrl":"https://doi.org/10.1080/19345747.2022.2142177","url":null,"abstract":"Abstract Cluster randomized trials (CRTs) are commonly used to evaluate educational interventions, particularly their effectiveness. Recently there has been greater emphasis on using these trials to explore cost-effectiveness. However, methods for establishing the power of cluster randomized cost-effectiveness trials (CRCETs) are limited. This study develops power computation formulas and statistical software to help researchers plan two- and three-level CRCETs. We illustrate the application of our formulas and software for the designs of CRCETs and discuss the influence of sample size, nesting effects, covariates, and the covariance between cost and effectiveness measures on the statistical power of cost-effectiveness estimates.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47081389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-12DOI: 10.1080/19345747.2022.2142178
Timothy Lycurgus, B. Hansen, Mark White
Full Article Citation: Lycurgus, T., Hansen, B., White, M. (2022). Conjuring power from a theory of change: The PWRD method for trials with anticipated variation in effects. Journal of Research on Educational Effectiveness. DOI: https://doi.org/10.1080/19345747.2022.2142178. Many efficacy trials are conducted only after careful vetting in national funding competitions. As part of these competitions, applications must justify the intervention’s theory of change: how and why do the desired improvements in outcomes occur? In scenarios with repeated measurements on participants, some of the measurements may be more likely to manifest a treatment effect than others; the theory of change may provide guidance as to which of those observations are most likely to be affected by the treatment.
{"title":"Conjuring Power from a Theory of Change: The PWRD Method for Trials with Anticipated Variation in Effects","authors":"Timothy Lycurgus, B. Hansen, Mark White","doi":"10.1080/19345747.2022.2142178","DOIUrl":"https://doi.org/10.1080/19345747.2022.2142178","url":null,"abstract":"Full Article Citation: Lycurgus, T., Hansen, B., White, M. (2022). Conjuring power from a theory of change: The PWRD method for trials with anticipated variation in effects. Journal of Research on Educational Effectiveness. DOI: https://doi.org/10.1080/19345747.2022.2142178. Many efficacy trials are conducted only after careful vetting in national funding competitions. As part of these competitions, applications must justify the intervention’s theory of change: how and why do the desired improvements in outcomes occur? In scenarios with repeated measurements on participants, some of the measurements may be more likely to manifest a treatment effect than others; the theory of change may provide guidance as to which of those observations are most likely to be affected by the treatment.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43896924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-12DOI: 10.1080/19345747.2022.2139785
Lindsay E. Brown, Ha Yeon Kim, Carly Tubbs Dolan, Autumn Brown, Jennifer Sklar, J. Aber
Abstract Despite substantial cross-national interest in remedial programming as a way to support low-achieving students, evidence of its effectiveness is rare, particularly in low-income and/or crisis-affected contexts. In this article, we present experimental evidence of the impact of a remedial tutoring program on academic outcomes from a two-level randomized trial of two treatments in Niger: school randomization testing the impact of skill-targeted SEL activities and within-school student-level randomization testing the impact of access to remedial tutoring. We find that tutoring for 4 h per week improves students’ literacy and Math outcomes, and the addition of skill-targeted SEL activities positively impacts school grades above and beyond access to tutoring alone. These findings suggest the potential value of remedial tutoring to supplement formal schooling in low-income and/or conflict-affected contexts. They also suggest increased attention to implementation strategies, as access alone was insufficient for students to attain grade-level competencies.
{"title":"Remedial Programming and Skill-Targeted SEL in Low-Income and Crisis-Affected Contexts: Experimental Evidence From Niger","authors":"Lindsay E. Brown, Ha Yeon Kim, Carly Tubbs Dolan, Autumn Brown, Jennifer Sklar, J. Aber","doi":"10.1080/19345747.2022.2139785","DOIUrl":"https://doi.org/10.1080/19345747.2022.2139785","url":null,"abstract":"Abstract Despite substantial cross-national interest in remedial programming as a way to support low-achieving students, evidence of its effectiveness is rare, particularly in low-income and/or crisis-affected contexts. In this article, we present experimental evidence of the impact of a remedial tutoring program on academic outcomes from a two-level randomized trial of two treatments in Niger: school randomization testing the impact of skill-targeted SEL activities and within-school student-level randomization testing the impact of access to remedial tutoring. We find that tutoring for 4 h per week improves students’ literacy and Math outcomes, and the addition of skill-targeted SEL activities positively impacts school grades above and beyond access to tutoring alone. These findings suggest the potential value of remedial tutoring to supplement formal schooling in low-income and/or conflict-affected contexts. They also suggest increased attention to implementation strategies, as access alone was insufficient for students to attain grade-level competencies.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48032049","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-07DOI: 10.1080/19345747.2022.2144787
Jodi L. Davenport, Yvonne S. Kao, Kristen Johannes, C. Hornburg, Nicole M. McNeil
Abstract A vast majority of elementary students struggle with the core, pre-algebraic concept of mathematical equivalence. The Improving Children’s Understanding of Equivalence (ICUE) intervention integrates four research-based strategies to improve outcomes for second grade students: (1) introducing the equal sign before arithmetic, (2) nontraditional arithmetic practice, (3) concreteness fading exercises, and (4) comparison and explanation. In a large-scale randomized control trial in California public schools, 132 second grade teachers were randomly assigned to either use the ICUE intervention or an active control consisting of nontraditional arithmetic practice alone. Using data from 121 teachers in the analytic sample, the study found that students in the intervention group outperformed students in the active control on proximal and transfer measures of equivalence with no observable tradeoffs in computational fluency. The findings suggest that the ICUE intervention helps students construct a robust understanding of mathematical equivalence, a critical precursor to success in algebra.
{"title":"Improving Children’s Understanding of Mathematical Equivalence: An Efficacy Study","authors":"Jodi L. Davenport, Yvonne S. Kao, Kristen Johannes, C. Hornburg, Nicole M. McNeil","doi":"10.1080/19345747.2022.2144787","DOIUrl":"https://doi.org/10.1080/19345747.2022.2144787","url":null,"abstract":"Abstract A vast majority of elementary students struggle with the core, pre-algebraic concept of mathematical equivalence. The Improving Children’s Understanding of Equivalence (ICUE) intervention integrates four research-based strategies to improve outcomes for second grade students: (1) introducing the equal sign before arithmetic, (2) nontraditional arithmetic practice, (3) concreteness fading exercises, and (4) comparison and explanation. In a large-scale randomized control trial in California public schools, 132 second grade teachers were randomly assigned to either use the ICUE intervention or an active control consisting of nontraditional arithmetic practice alone. Using data from 121 teachers in the analytic sample, the study found that students in the intervention group outperformed students in the active control on proximal and transfer measures of equivalence with no observable tradeoffs in computational fluency. The findings suggest that the ICUE intervention helps students construct a robust understanding of mathematical equivalence, a critical precursor to success in algebra.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41625790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-12-01DOI: 10.1080/19345747.2022.2139312
J. Polanin, Qi Zhang, Joseph A. Taylor, R. Williams, Megha Joshi, Lauren Burr
Abstract Systematic reviews and meta-analyses are important techniques because they synthesize results from multiple primary studies on a similar topic. To influence policy, practice, and research, however, synthesis researchers must translate the results for various audiences. Ideally, the translation drives future research agendas, informs policymaking, or assists in practical decision-making. An Evidence Gap Map (EGM), a graphical or tabular visualization of systematic review and meta-analysis results, is one ideal translation technique because it provides a structured framework to assess contexts for which primary evidence is available or to determine whether the effectiveness of an intervention or a program differs across populations, conditions, and settings. To bolster the field and promote the use of EGMs, we provide an overview of what constitutes an informative EGM, detail multiple examples of EGMs using extant meta-analytic results, and present a free R Shiny application we created to easily generate EGMs from typical meta-analytic datasets. We conclude by reviewing education-based systematic reviews that included an EGM to describe the current state of the field.
{"title":"Evidence Gap Maps in Education Research","authors":"J. Polanin, Qi Zhang, Joseph A. Taylor, R. Williams, Megha Joshi, Lauren Burr","doi":"10.1080/19345747.2022.2139312","DOIUrl":"https://doi.org/10.1080/19345747.2022.2139312","url":null,"abstract":"Abstract Systematic reviews and meta-analyses are important techniques because they synthesize results from multiple primary studies on a similar topic. To influence policy, practice, and research, however, synthesis researchers must translate the results for various audiences. Ideally, the translation drives future research agendas, informs policymaking, or assists in practical decision-making. An Evidence Gap Map (EGM), a graphical or tabular visualization of systematic review and meta-analysis results, is one ideal translation technique because it provides a structured framework to assess contexts for which primary evidence is available or to determine whether the effectiveness of an intervention or a program differs across populations, conditions, and settings. To bolster the field and promote the use of EGMs, we provide an overview of what constitutes an informative EGM, detail multiple examples of EGMs using extant meta-analytic results, and present a free R Shiny application we created to easily generate EGMs from typical meta-analytic datasets. We conclude by reviewing education-based systematic reviews that included an EGM to describe the current state of the field.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"532 - 552"},"PeriodicalIF":1.8,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42808711","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-11-30DOI: 10.1080/19345747.2022.2137073
Sidrah Baloch, Thomas J. Kane, Ethan Scherer, D. Staiger
Abstract Educators must balance the needs of students who start the school year behind grade level with their obligation to teach grade-appropriate content to all students. Educational software could help educators strike this balance by targeting content to students’ differing levels of mastery. Using a regression discontinuity design and detailed software log and administrative data, we compare two versions of an online mathematics program used by students in three education agencies. We find that although students assigned the modified curriculum did progress through content objectives more quickly than students assigned the default curriculum, they did not perform better on pre- and post-objective quizzes embedded in the software, and most never progressed far enough to reach the grade-level content. Furthermore, there was no statistically significant effect of the modified curriculum on formative test scores. These findings suggest policymakers and practitioners should exercise caution when assigning exclusively remedial content to students who start the school year behind grade level, even though this is a common feature of many math educational software programs.
{"title":"The Uncertain Role of Educational Software in Remediating Student Learning: Regression Discontinuity Evidence from Three Local Education Agencies","authors":"Sidrah Baloch, Thomas J. Kane, Ethan Scherer, D. Staiger","doi":"10.1080/19345747.2022.2137073","DOIUrl":"https://doi.org/10.1080/19345747.2022.2137073","url":null,"abstract":"Abstract Educators must balance the needs of students who start the school year behind grade level with their obligation to teach grade-appropriate content to all students. Educational software could help educators strike this balance by targeting content to students’ differing levels of mastery. Using a regression discontinuity design and detailed software log and administrative data, we compare two versions of an online mathematics program used by students in three education agencies. We find that although students assigned the modified curriculum did progress through content objectives more quickly than students assigned the default curriculum, they did not perform better on pre- and post-objective quizzes embedded in the software, and most never progressed far enough to reach the grade-level content. Furthermore, there was no statistically significant effect of the modified curriculum on formative test scores. These findings suggest policymakers and practitioners should exercise caution when assigning exclusively remedial content to students who start the school year behind grade level, even though this is a common feature of many math educational software programs.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"398 - 418"},"PeriodicalIF":1.8,"publicationDate":"2022-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42713774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号