Pub Date : 2024-01-03DOI: 10.1080/19345747.2023.2284768
Margarita Olivera-Aguilar, Samuel H. Rikoon
One aspect usually overlooked in studies evaluating the efficacy of educational interventions, is the extent to which posttest responses reflect actual changes in targeted latent constructs versus ...
{"title":"Intervention Effect or Measurement Artifact? Using Invariance Models to Reveal Response-Shift Bias in Experimental Studies","authors":"Margarita Olivera-Aguilar, Samuel H. Rikoon","doi":"10.1080/19345747.2023.2284768","DOIUrl":"https://doi.org/10.1080/19345747.2023.2284768","url":null,"abstract":"One aspect usually overlooked in studies evaluating the efficacy of educational interventions, is the extent to which posttest responses reflect actual changes in targeted latent constructs versus ...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"22 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374037","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-03DOI: 10.1080/19345747.2023.2290546
Wendy Chan, Jimin Oh, Katherine J. Wilson
Over the past decade, propensity score-based methods have made an important contribution to the improvement of generalizations from educational studies. However, an important limitation is that man...
{"title":"Redefining Populations of Inference for Generalizations from Small Studies","authors":"Wendy Chan, Jimin Oh, Katherine J. Wilson","doi":"10.1080/19345747.2023.2290546","DOIUrl":"https://doi.org/10.1080/19345747.2023.2290546","url":null,"abstract":"Over the past decade, propensity score-based methods have made an important contribution to the improvement of generalizations from educational studies. However, an important limitation is that man...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"14 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-03DOI: 10.1080/19345747.2023.2287601
Joshua B. Gilbert
This simulation study examines the characteristics of the Explanatory Item Response Model (EIRM) when estimating treatment effects when compared to classical test theory (CTT) sum and mean scores a...
{"title":"Estimating Treatment Effects with the Explanatory Item Response Model","authors":"Joshua B. Gilbert","doi":"10.1080/19345747.2023.2287601","DOIUrl":"https://doi.org/10.1080/19345747.2023.2287601","url":null,"abstract":"This simulation study examines the characteristics of the Explanatory Item Response Model (EIRM) when estimating treatment effects when compared to classical test theory (CTT) sum and mean scores a...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"2 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139373885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-07DOI: 10.1080/19345747.2023.2279167
Yue Ma, Xinwu Zhang, C. Abbey, Derek Hu, Oliver Lee, Weiting Hung, Chiayuan Chang, Chyi-In Wu, Dimitris Friesen, Scott Rozelle
{"title":"Computer Assisted Learning and Academic Performance in Rural Taiwan","authors":"Yue Ma, Xinwu Zhang, C. Abbey, Derek Hu, Oliver Lee, Weiting Hung, Chiayuan Chang, Chyi-In Wu, Dimitris Friesen, Scott Rozelle","doi":"10.1080/19345747.2023.2279167","DOIUrl":"https://doi.org/10.1080/19345747.2023.2279167","url":null,"abstract":"","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"30 8","pages":""},"PeriodicalIF":1.8,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138594258","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-07DOI: 10.1080/19345747.2023.2279124
James C. DiPerna, Susan Crandall Hart, Pui-Wa Lei, Tianying Sun, Hui Zhao, Kyle Husmann, Xinyue Li
The purpose of this study was to explore the effectiveness of a universal classroom-based social-emotional learning (SEL) program for first grade students. Forty classrooms from 13 elementary schoo...
{"title":"Does Universal SEL Work under Typical Implementation Practices? Outcomes of a First Grade Effectiveness Trial","authors":"James C. DiPerna, Susan Crandall Hart, Pui-Wa Lei, Tianying Sun, Hui Zhao, Kyle Husmann, Xinyue Li","doi":"10.1080/19345747.2023.2279124","DOIUrl":"https://doi.org/10.1080/19345747.2023.2279124","url":null,"abstract":"The purpose of this study was to explore the effectiveness of a universal classroom-based social-emotional learning (SEL) program for first grade students. Forty classrooms from 13 elementary schoo...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"56 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138563042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-07DOI: 10.1080/19345747.2023.2279123
Nompumelelo Mohohlwane, Stephen Taylor, Jacobus Cilliers, Brahm Fleisch
In many countries, children need to become proficient in both their home language (L1) and an international language, such as English (L2). Governments face tradeoffs in how to prioritize these two...
{"title":"Reading Skills Transfer Best from Home Language to a Second Language: Policy Lessons from Two Field Experiments in South Africa","authors":"Nompumelelo Mohohlwane, Stephen Taylor, Jacobus Cilliers, Brahm Fleisch","doi":"10.1080/19345747.2023.2279123","DOIUrl":"https://doi.org/10.1080/19345747.2023.2279123","url":null,"abstract":"In many countries, children need to become proficient in both their home language (L1) and an international language, such as English (L2). Governments face tradeoffs in how to prioritize these two...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"248 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138562310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-29DOI: 10.1080/19345747.2023.2276740
Youngran Kim, Ron Zimmer
During the pandemic, a number of states instituted hold-harmless funding policies to protect school district financially from declining enrollments. In addition, some school choice policies have pr...
{"title":"Is Reputational Pressure Enough to Create Competitive School Choice Effects? Evidence from Seoul’s School Choice Policy","authors":"Youngran Kim, Ron Zimmer","doi":"10.1080/19345747.2023.2276740","DOIUrl":"https://doi.org/10.1080/19345747.2023.2276740","url":null,"abstract":"During the pandemic, a number of states instituted hold-harmless funding policies to protect school district financially from declining enrollments. In addition, some school choice policies have pr...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"127 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2023-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1080/19345747.2023.2269923
David Blazar, Beth Schueler
What guidance does research provide about how to improve school district performance in the United States? Despite over 30 years of inquiry on the topic of effective districts, existing frameworks ...
{"title":"Effective School District Policies and Practices: Synthesizing Theoretical Frameworks and Empirical Findings across Disciplines","authors":"David Blazar, Beth Schueler","doi":"10.1080/19345747.2023.2269923","DOIUrl":"https://doi.org/10.1080/19345747.2023.2269923","url":null,"abstract":"What guidance does research provide about how to improve school district performance in the United States? Despite over 30 years of inquiry on the topic of effective districts, existing frameworks ...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"7 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-09DOI: 10.1080/19345747.2023.2273511
Elise Chor, P. Lindsay Chase-Lansdale, Teresa Eckrich Sommer, Terri Sabol, Lauren Tighe, Jeanne Brooks-Gunn, Hirokazu Yoshikawa, Amanda Morris, Christopher King
AbstractIncreasingly, parents of young children need postsecondary credentials to compete in the labor market and meet basic family needs. This study uses a quasi-experimental design to examine the effects of CareerAdvance, a two-generation education intervention that offers postsecondary career training in healthcare for parents paired with Head Start for children. Overall, we find that CareerAdvance promotes low-income parents’ educational advancement during the first three years after program entry, with weaker evidence of benefits to career progress and psychological wellbeing, and no evidence of economic gains. The two-generation program promotes greater educational and career advancement among parents without postsecondary credentials at baseline, than for parents who began the program with postsecondary credentials. In contrast, exploratory analyses suggest that parents entering the program with postsecondary credentials experienced benefits to some individual markers of economic and psychological wellbeing within three years.Keywords: ParentsHead Starteducationpolicypropensity score Disclosure StatementNo potential conflict of interest was reported by the author(s).Open ScholarshipThis article has earned the Center for Open Science badge for Open Materials. The materials are openly accessible at https://doi.org/10.17605/OSF.IO/5NVM3.Additional informationFundingHealth Profession Opportunity Grant 90FX00100, Administration for Children and Families, U.S. Department of Health and Human Services. Health Profession Opportunity Grant-University Partnership 90PH0020, Administration for Children and Families, U.S. Department of Health and Human Services. W.K. Kellogg Foundation Grant P3020014. Foundation for Child Development Grant Northwestern 06-2014.
{"title":"Three-Year Outcomes for Low-Income Parents of Young Children in a Two-Generation Education Program","authors":"Elise Chor, P. Lindsay Chase-Lansdale, Teresa Eckrich Sommer, Terri Sabol, Lauren Tighe, Jeanne Brooks-Gunn, Hirokazu Yoshikawa, Amanda Morris, Christopher King","doi":"10.1080/19345747.2023.2273511","DOIUrl":"https://doi.org/10.1080/19345747.2023.2273511","url":null,"abstract":"AbstractIncreasingly, parents of young children need postsecondary credentials to compete in the labor market and meet basic family needs. This study uses a quasi-experimental design to examine the effects of CareerAdvance, a two-generation education intervention that offers postsecondary career training in healthcare for parents paired with Head Start for children. Overall, we find that CareerAdvance promotes low-income parents’ educational advancement during the first three years after program entry, with weaker evidence of benefits to career progress and psychological wellbeing, and no evidence of economic gains. The two-generation program promotes greater educational and career advancement among parents without postsecondary credentials at baseline, than for parents who began the program with postsecondary credentials. In contrast, exploratory analyses suggest that parents entering the program with postsecondary credentials experienced benefits to some individual markers of economic and psychological wellbeing within three years.Keywords: ParentsHead Starteducationpolicypropensity score Disclosure StatementNo potential conflict of interest was reported by the author(s).Open ScholarshipThis article has earned the Center for Open Science badge for Open Materials. The materials are openly accessible at https://doi.org/10.17605/OSF.IO/5NVM3.Additional informationFundingHealth Profession Opportunity Grant 90FX00100, Administration for Children and Families, U.S. Department of Health and Human Services. Health Profession Opportunity Grant-University Partnership 90PH0020, Administration for Children and Families, U.S. Department of Health and Human Services. W.K. Kellogg Foundation Grant P3020014. Foundation for Child Development Grant Northwestern 06-2014.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" 20","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135240736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-09DOI: 10.1080/19345747.2023.2269936
Trisha H. Borman, Geoffrey D. Borman, So Jung Park, Bo Zhu, Scott Houghton
AbstractWe present findings across three randomized trials of Descubriendo la Lectura (DLL), an intervention designed to improve literacy skills of Spanish-speaking first graders struggling with reading. DLL is a one-to-one Spanish-language literacy program lasting 12–20 weeks offered to a school’s lowest performing emerging bilingual first graders. Directly replicating procedures across three cohorts, we examined Spanish and English literacy outcomes for approximately 400 students across 30 schools, 10 districts, and five states. Analyses revealed statistically significant effects of student-level assignment to DLL on all 11 Spanish-language outcomes, with an average impact of d = 0.57. Impacts across the four English-language outcomes were positive but not statistically significant. Descriptive results suggest that the Spanish-language impacts are reliably replicated, most likely due to the intervention’s strong implementation supports.Keywords: Early literacyEnglish learnershigh-intensity tutoringrandomized controlled trialreplication Open ScholarshipThis article has earned the Center for Open Science badge for Open Data. The data are openly accessible at https://www.openicpsr.org/openicpsr/project/193586/version/V1/view.Disclosure StatementNo potential conflict of interest was reported by the author(s).Notes1 See https://ies.ed.gov/ncee/wwc.2 The WWC reviews of these studies are available at https://ies.ed.gov/ncee/wwc/study/89881 and https://ies.ed.gov/ncee/wwc/Study/90683.3 IdO is the Spanish version of RR’s Observation Survey (Clay, Citation2019), which tests six literacy tasks (letter identification, word test [i.e., reading vocabulary], concepts about print, writing vocabulary, phonemic awareness, and text reading) that, taken together, can measure emerging literacy abilities for early readers.4 The Logramos is an evidence-based Spanish-language literacy assessment that parallels the English-language Iowa Test of Basic Skills (ITBS) and is used to monitor growth and achievement in Spanish literacy skills.5 See https://charts.intensiveintervention.org/screening/tool/?id=77c5c64492268897.6 See https://www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Academic-Learning/Brief/Aprenda%3A-La-Prueba-de-Logros-en-Espa%C3%B1ol-%7C-Tercera-edici%C3%B3n/p/100000585.html.7 See https://riversideinsights.com/log_tercera.8 Specifically, we rank ordered the statistically significant findings within the domain in ascending order of the p-values, such that p1 ≤ p2 ≤ p3 ≤ p4 ≤ p5 ≤ p6 ≤ pM. For each p-value, p1-M, we computed: pi = αi/M, where i is the rank for pi, with i = 1, 2, . . . M; M is the total number of findings within the domain; and α is the target level of statistical significance.Additional informationFundingThe research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through [Grant #R305A160060] to the American Institutes for Research. The opinions expressed are
{"title":"The Spanish- and English-Language Literacy Impacts of <i>Descubriendo la Lectura</i> across Three Experimental Replications","authors":"Trisha H. Borman, Geoffrey D. Borman, So Jung Park, Bo Zhu, Scott Houghton","doi":"10.1080/19345747.2023.2269936","DOIUrl":"https://doi.org/10.1080/19345747.2023.2269936","url":null,"abstract":"AbstractWe present findings across three randomized trials of Descubriendo la Lectura (DLL), an intervention designed to improve literacy skills of Spanish-speaking first graders struggling with reading. DLL is a one-to-one Spanish-language literacy program lasting 12–20 weeks offered to a school’s lowest performing emerging bilingual first graders. Directly replicating procedures across three cohorts, we examined Spanish and English literacy outcomes for approximately 400 students across 30 schools, 10 districts, and five states. Analyses revealed statistically significant effects of student-level assignment to DLL on all 11 Spanish-language outcomes, with an average impact of d = 0.57. Impacts across the four English-language outcomes were positive but not statistically significant. Descriptive results suggest that the Spanish-language impacts are reliably replicated, most likely due to the intervention’s strong implementation supports.Keywords: Early literacyEnglish learnershigh-intensity tutoringrandomized controlled trialreplication Open ScholarshipThis article has earned the Center for Open Science badge for Open Data. The data are openly accessible at https://www.openicpsr.org/openicpsr/project/193586/version/V1/view.Disclosure StatementNo potential conflict of interest was reported by the author(s).Notes1 See https://ies.ed.gov/ncee/wwc.2 The WWC reviews of these studies are available at https://ies.ed.gov/ncee/wwc/study/89881 and https://ies.ed.gov/ncee/wwc/Study/90683.3 IdO is the Spanish version of RR’s Observation Survey (Clay, Citation2019), which tests six literacy tasks (letter identification, word test [i.e., reading vocabulary], concepts about print, writing vocabulary, phonemic awareness, and text reading) that, taken together, can measure emerging literacy abilities for early readers.4 The Logramos is an evidence-based Spanish-language literacy assessment that parallels the English-language Iowa Test of Basic Skills (ITBS) and is used to monitor growth and achievement in Spanish literacy skills.5 See https://charts.intensiveintervention.org/screening/tool/?id=77c5c64492268897.6 See https://www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Academic-Learning/Brief/Aprenda%3A-La-Prueba-de-Logros-en-Espa%C3%B1ol-%7C-Tercera-edici%C3%B3n/p/100000585.html.7 See https://riversideinsights.com/log_tercera.8 Specifically, we rank ordered the statistically significant findings within the domain in ascending order of the p-values, such that p1 ≤ p2 ≤ p3 ≤ p4 ≤ p5 ≤ p6 ≤ pM. For each p-value, p1-M, we computed: pi = αi/M, where i is the rank for pi, with i = 1, 2, . . . M; M is the total number of findings within the domain; and α is the target level of statistical significance.Additional informationFundingThe research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through [Grant #R305A160060] to the American Institutes for Research. The opinions expressed are","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" 29","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135290915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}