Pub Date : 2024-04-01Epub Date: 2023-08-03DOI: 10.1177/0193841X231193468
Jin Zheng, Yao Xiong, Yimei Zheng, Haitao Zhang, Rui Wu
Stroke is the leading cause of death and disability among people in China, and it leads to heavy burdens for patients, their families and society. An accurate prediction of the risk of stroke has important implications for early intervention and treatment. In light of recent advances in machine learning, the application of this technique in stroke prediction has achieved plentiful promising results. To detect the relationship between potential factors and the risk of stroke and examine which machine learning method significantly can enhance the prediction accuracy of stroke. We employed six machine learning methods including logistic regression, naive Bayes, decision tree, random forest, K-nearest neighbor and support vector machine, to model and predict the risk of stroke. Participants were 233 patients from Sichuan and Chongqing. Four indicators (accuracy, precision, recall and F1 metric) were examined to evaluate the predictive performance of the different models. The empirical results indicate that random forest yields the best accuracy, recall and F1 in predicting the risk of stroke, with an accuracy of .7548, precision of .7805, recall of .7619 and F1 of .7711. Additionally, the findings show that age, cerebral infarction, PM 8 (an anti-atrial fibrillation drug), and drinking are independent risk factors for stroke. Further studies should adopt a broader assortment of machine learning methods to analyze the risk of stroke, by which better accuracy can be expected. In particular, RF can successfully enhance the forecasting accuracy for stroke.
脑卒中是导致中国人死亡和残疾的主要原因,给患者、家庭和社会带来沉重负担。准确预测脑卒中风险对早期干预和治疗具有重要意义。随着机器学习技术的不断进步,该技术在脑卒中预测中的应用也取得了丰硕的成果。为了检测潜在因素与脑卒中风险之间的关系,并研究哪种机器学习方法能显著提高脑卒中预测的准确性。我们采用了六种机器学习方法,包括逻辑回归、天真贝叶斯、决策树、随机森林、K-近邻和支持向量机,对脑卒中风险进行建模和预测。研究对象为来自四川和重庆的 233 名患者。研究考察了四个指标(准确度、精确度、召回率和 F1 指标),以评估不同模型的预测性能。实证结果表明,随机森林预测脑卒中风险的准确度、召回率和 F1 值最佳,准确度为 0.7548,精确度为 0.7805,召回率为 0.7619,F1 值为 0.7711。此外,研究结果表明,年龄、脑梗塞、PM 8(一种抗心房颤动药物)和饮酒是中风的独立危险因素。进一步的研究应采用更广泛的机器学习方法来分析中风风险,从而提高准确性。尤其是射频技术可以成功提高中风的预测准确性。
{"title":"Evaluating the Stroke Risk of Patients using Machine Learning: A New Perspective from Sichuan and Chongqing.","authors":"Jin Zheng, Yao Xiong, Yimei Zheng, Haitao Zhang, Rui Wu","doi":"10.1177/0193841X231193468","DOIUrl":"10.1177/0193841X231193468","url":null,"abstract":"<p><p>Stroke is the leading cause of death and disability among people in China, and it leads to heavy burdens for patients, their families and society. An accurate prediction of the risk of stroke has important implications for early intervention and treatment. In light of recent advances in machine learning, the application of this technique in stroke prediction has achieved plentiful promising results. To detect the relationship between potential factors and the risk of stroke and examine which machine learning method significantly can enhance the prediction accuracy of stroke. We employed six machine learning methods including logistic regression, naive Bayes, decision tree, random forest, K-nearest neighbor and support vector machine, to model and predict the risk of stroke. Participants were 233 patients from Sichuan and Chongqing. Four indicators (accuracy, precision, recall and F1 metric) were examined to evaluate the predictive performance of the different models. The empirical results indicate that random forest yields the best accuracy, recall and F1 in predicting the risk of stroke, with an accuracy of .7548, precision of .7805, recall of .7619 and F1 of .7711. Additionally, the findings show that age, cerebral infarction, PM 8 (an anti-atrial fibrillation drug), and drinking are independent risk factors for stroke. Further studies should adopt a broader assortment of machine learning methods to analyze the risk of stroke, by which better accuracy can be expected. In particular, RF can successfully enhance the forecasting accuracy for stroke.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"346-369"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10302426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-01Epub Date: 2023-06-12DOI: 10.1177/0193841X231175549
David R Judkins, Gabriel Durham
In 2003, Bloom, Hill, and Riccio (BHR) published an influential paper introducing novel methods for explaining the variation in local impacts observed in multi-site randomized control trials of socio-economic interventions in terms of site-level mediators. This paper seeks to improve upon this previous work by using student-level data to measure site-level mediators and confounders. Development of asymptotic behavior backed up with simulations and an empirical example. Students and training providers. Two simulations and an empirical application to data from an evaluation of the Health Professions Opportunity Grants (HPOG) Program. This empirical analysis involved roughly 6600 participants across 37 local sites. We examine bias and mean square error of estimates of mediation coefficients as well as the true coverage of nominal 95-percent confidence intervals on the mediation coefficients. Simulations suggest that the new methods generally improve the quality of inferences even when there is no confounding. Applying this methodology to the HPOG study shows that program-average FTE months of study by month six was a significant mediator of both career progress and long-term degree/credential receipt. Evaluators can robustify their BHR-style analyses by the use of the methods proposed here.
{"title":"Using Ecometric Data to Explore Sources of Cross-Site Impact Variance in Multi-Site Trials.","authors":"David R Judkins, Gabriel Durham","doi":"10.1177/0193841X231175549","DOIUrl":"10.1177/0193841X231175549","url":null,"abstract":"<p><p>In 2003, Bloom, Hill, and Riccio (BHR) published an influential paper introducing novel methods for explaining the variation in local impacts observed in multi-site randomized control trials of socio-economic interventions in terms of site-level mediators. This paper seeks to improve upon this previous work by using student-level data to measure site-level mediators and confounders. Development of asymptotic behavior backed up with simulations and an empirical example. Students and training providers. Two simulations and an empirical application to data from an evaluation of the Health Professions Opportunity Grants (HPOG) Program. This empirical analysis involved roughly 6600 participants across 37 local sites. We examine bias and mean square error of estimates of mediation coefficients as well as the true coverage of nominal 95-percent confidence intervals on the mediation coefficients. Simulations suggest that the new methods generally improve the quality of inferences even when there is no confounding. Applying this methodology to the HPOG study shows that program-average FTE months of study by month six was a significant mediator of both career progress and long-term degree/credential receipt. Evaluators can robustify their BHR-style analyses by the use of the methods proposed here.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"274-311"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9969498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-01Epub Date: 2023-05-08DOI: 10.1177/0193841X231171965
Serena Berretta, Sara Garbin, Maria Iannario, Omar Paccagnella
Program evaluations often investigate complex or multi-dimensional constructs, such as individual opinions or attitudes, by means of ratings. A different interpretation of the same question may affect cross-country comparability, leading to the Differential Item Functioning problem. Anchoring vignettes were introduced in the literature as a way to adjust self-evaluations from this interpersonal incomparability. In this paper, we first introduce a new nonparametric solution to analyse anchoring vignette data, recoding a variable based on a rating scale to a new corrected-variable that guarantees comparability in any cross-country analysis. Then, we exploit the flexibility of a mixture model introduced to account for uncertainty in the response process (the CUP model) to test if the proposed solution is effectively able to remove this reported heterogeneity. This solution is easy to construct and has important advantages compared with the original nonparametric solution adopted with anchoring vignette data. The novel indicator is applied to investigate self-reported depression in an old population. Data that will be analysed come from the second wave of the Survey of Health, Ageing and Retirement in Europe, collected in 2006/2007. Results highlight the need of correcting for reported heterogeneity comparing individual self-evaluations. Once interpersonal incomparability resulting from the different uses of response scales is removed from the self-assessments, some estimates are reversed in magnitude and signs with respect to the analysis of the collected data.
{"title":"A Novel Indicator to Correct for Individual Reported Heterogeneity. An Application to Self-Evaluation of Later-Life Depression.","authors":"Serena Berretta, Sara Garbin, Maria Iannario, Omar Paccagnella","doi":"10.1177/0193841X231171965","DOIUrl":"10.1177/0193841X231171965","url":null,"abstract":"<p><p>Program evaluations often investigate complex or multi-dimensional constructs, such as individual opinions or attitudes, by means of ratings. A different interpretation of the same question may affect cross-country comparability, leading to the Differential Item Functioning problem. Anchoring vignettes were introduced in the literature as a way to adjust self-evaluations from this interpersonal incomparability. In this paper, we first introduce a new nonparametric solution to analyse anchoring vignette data, recoding a variable based on a rating scale to a <i>new corrected-</i>variable that guarantees comparability in any cross-country analysis. Then, we exploit the flexibility of a mixture model introduced to account for uncertainty in the response process (the CUP model) to test if the proposed solution is effectively able to remove this reported heterogeneity. This solution is easy to construct and has important advantages compared with the original nonparametric solution adopted with anchoring vignette data. The novel indicator is applied to investigate self-reported depression in an old population. Data that will be analysed come from the second wave of the Survey of Health, Ageing and Retirement in Europe, collected in 2006/2007. Results highlight the need of correcting for reported heterogeneity comparing individual self-evaluations. Once interpersonal incomparability resulting from the different uses of response scales is removed from the self-assessments, some estimates are reversed in magnitude and signs with respect to the analysis of the collected data.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"221-250"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9433536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-01Epub Date: 2023-06-23DOI: 10.1177/0193841X231182749
Shiying Wang, Jaffar Abbas, Khalid Ibrahim Al-Sulati, Syed Ale Raza Shah
Economic corridors unlock new economic opportunities and tourism development in the region to achieve sustainable development goals. Green economic growth is conducive to environmental sustainability. Economic mega-projects of CPEC promote tourism that leads to communities' well-being and better quality of life. Modern infrastructure development contributes significantly to economic growth and tourism activities. This study's objectives emphasize exploring tourism and sustainable development pursuits under OBOR economic projects that open doors to improving residents' quality of life. The growing world is an eyewitness to a continuous rise in emissions and its severe consequences for humankind. It is necessary to show off the leading factors that result in tourism and economic activities causing environmental pollution rather than blame policymakers. Undoubtedly, many studies previously focused on demonstrating the influence of socio-economic factors that lead to better environmental quality. However, the empirical literature on tourism, social well-being, foreign direct investment, and the Environment in Belt and Road developed economies needed improvement. This research applied a series of advanced estimators that help demonstrate the study's probable results. This study explores the role of Social well-being (HDI), tourism development, FDI, renewable energy, information & communication technology (ICT), and urbanization on CO2 emissions in Belt and Road (BRI) developed economies.Estimated results exhibited the significant contribution of ICT and renewable energy to sustainability. Besides, FDI contributes to emissions reduction after its threshold level. Conversely, urbanization and tourism activities contribute to environmental pollution. The study outcomes stated inverted/EKC U-shaped hypotheses related to specified economies. Finally, the analysis based on the D-H panel causality test constructs exciting results.The present study concludes that economic corridor plays a vital role in tourism development, the community's well-being, and SDGs goals (sustainable development) impact on environmental safety. The findings suggest essential and applicable policies to attain the desired sustainability level. Findings contribute to the literature on tourism, well-being, and sustainability. Further studies can use insights using this methodology.
经济走廊为本地区带来新的经济机遇和旅游业发展,以实现可持续发展目标。绿色经济增长有利于环境的可持续发展。中巴经济走廊的大型经济项目促进了旅游业的发展,提高了社区的福祉和生活质量。现代基础设施发展极大地促进了经济增长和旅游活动。本研究的目标强调在 OBOR 经济项目下探索旅游业和可持续发展,为提高居民生活质量打开大门。不断发展的世界见证了排放量的持续增长及其对人类造成的严重后果。有必要揭示旅游和经济活动造成环境污染的主导因素,而不是指责政策制定者。毋庸置疑,以前的许多研究都侧重于展示社会经济因素对改善环境质量的影响。然而,有关 "一带一路 "发达经济体的旅游业、社会福利、外国直接投资和环境的实证文献有待改进。本研究采用了一系列先进的估计方法,有助于证明研究的可能结果。本研究探讨了 "一带一路 "发达经济体的社会福利(人类发展指数)、旅游业发展、外国直接投资、可再生能源、信息通信技术(ICT)和城市化对二氧化碳排放的影响。此外,外国直接投资在达到阈值后也有助于减排。相反,城市化和旅游活动加剧了环境污染。研究结果表明了与特定经济体相关的倒/EKC U 型假设。最后,基于 D-H 面板因果检验的分析得出了令人兴奋的结果。本研究得出结论,经济走廊在旅游业发展、社区福祉和可持续发展目标(可持续发展)对环境安全的影响方面发挥着至关重要的作用。研究结果提出了达到预期可持续发展水平的基本适用政策。研究结果为有关旅游业、福祉和可持续性的文献做出了贡献。进一步的研究可以利用这种方法获得更多的见解。
{"title":"The Impact of Economic Corridor and Tourism on Local Community's Quality of Life under One Belt One Road Context.","authors":"Shiying Wang, Jaffar Abbas, Khalid Ibrahim Al-Sulati, Syed Ale Raza Shah","doi":"10.1177/0193841X231182749","DOIUrl":"10.1177/0193841X231182749","url":null,"abstract":"<p><p>Economic corridors unlock new economic opportunities and tourism development in the region to achieve sustainable development goals. Green economic growth is conducive to environmental sustainability. Economic mega-projects of CPEC promote tourism that leads to communities' well-being and better quality of life. Modern infrastructure development contributes significantly to economic growth and tourism activities. This study's objectives emphasize exploring tourism and sustainable development pursuits under OBOR economic projects that open doors to improving residents' quality of life. The growing world is an eyewitness to a continuous rise in emissions and its severe consequences for humankind. It is necessary to show off the leading factors that result in tourism and economic activities causing environmental pollution rather than blame policymakers. Undoubtedly, many studies previously focused on demonstrating the influence of socio-economic factors that lead to better environmental quality. However, the empirical literature on tourism, social well-being, foreign direct investment, and the Environment in Belt and Road developed economies needed improvement. This research applied a series of advanced estimators that help demonstrate the study's probable results. This study explores the role of Social well-being (HDI), tourism development, FDI, renewable energy, information & communication technology (ICT), and urbanization on CO2 emissions in Belt and Road (BRI) developed economies.Estimated results exhibited the significant contribution of ICT and renewable energy to sustainability. Besides, FDI contributes to emissions reduction after its threshold level. Conversely, urbanization and tourism activities contribute to environmental pollution. The study outcomes stated inverted/EKC U-shaped hypotheses related to specified economies. Finally, the analysis based on the D-H panel causality test constructs exciting results.The present study concludes that economic corridor plays a vital role in tourism development, the community's well-being, and SDGs goals (sustainable development) impact on environmental safety. The findings suggest essential and applicable policies to attain the desired sustainability level. Findings contribute to the literature on tourism, well-being, and sustainability. Further studies can use insights using this methodology.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"312-345"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10050838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1177/0193841X241241354
Steven Glazerman, Larissa Campuzano, Nancy Murray
Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.
{"title":"Education Experiments in Latin America: Empirical Evidence to Guide Evaluation Design.","authors":"Steven Glazerman, Larissa Campuzano, Nancy Murray","doi":"10.1177/0193841X241241354","DOIUrl":"https://doi.org/10.1177/0193841X241241354","url":null,"abstract":"<p><p>Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241241354"},"PeriodicalIF":0.9,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140327263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-20DOI: 10.1177/0193841X241239512
Luis Faundez, Robert Kaestner
This article describes a conceptual and empirical approach for estimating a human capital production function of child development that incorporates mother- or child-fixed effects. The use of mother- or child-fixed effects is common in this applied economics literature, but its application is often inconsistent with human capital theory. We outline the problem and demonstrate its empirical importance with an analysis of the effect of Head Start and preschool on child and adult outcomes. The empirical specification we develop has broad implications for a variety of applied microeconomic analyses beyond our specific application. Results of our analysis indicate that attending Head Start or preschool had no economically or statistically significant effect on child or adult outcomes.
{"title":"Estimating a Theoretically Consistent Human Capital Production Function With an Application to Head Start.","authors":"Luis Faundez, Robert Kaestner","doi":"10.1177/0193841X241239512","DOIUrl":"https://doi.org/10.1177/0193841X241239512","url":null,"abstract":"<p><p>This article describes a conceptual and empirical approach for estimating a human capital production function of child development that incorporates mother- or child-fixed effects. The use of mother- or child-fixed effects is common in this applied economics literature, but its application is often inconsistent with human capital theory. We outline the problem and demonstrate its empirical importance with an analysis of the effect of Head Start and preschool on child and adult outcomes. The empirical specification we develop has broad implications for a variety of applied microeconomic analyses beyond our specific application. Results of our analysis indicate that attending Head Start or preschool had no economically or statistically significant effect on child or adult outcomes.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241239512"},"PeriodicalIF":0.9,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-20DOI: 10.1177/0193841X241240639
Thuan Van Pham, Loc My Thi Nguyen, Trung Tran, Hoang Yen Thi Duong, Hoan Huu Tran, Thanh Thi Nghiem
In this paper, based on the established Vitae Researcher Development Framework (RDF), we introduce a new framework, tailor-made specifically for Vietnamese educational researchers, namely, Vietnam's Framework for Educational Researchers (VFER). VFER is expected to serve as a tool for Vietnamese educational researchers to self-evaluate their skills and support them in developing their career qualities from junior to senior career ladders. The framework includes four domains with ten subdomains and 28 indicators. To date, VFER has been implemented in some Vietnamese universities of pedagogical education. We expect that other research fields will look to VFER as a reference to build their own research capacity framework.
{"title":"Introducing a Competency Framework for Educational Researchers: The Case of Vietnam.","authors":"Thuan Van Pham, Loc My Thi Nguyen, Trung Tran, Hoang Yen Thi Duong, Hoan Huu Tran, Thanh Thi Nghiem","doi":"10.1177/0193841X241240639","DOIUrl":"https://doi.org/10.1177/0193841X241240639","url":null,"abstract":"<p><p>In this paper, based on the established Vitae Researcher Development Framework (RDF), we introduce a new framework, tailor-made specifically for Vietnamese educational researchers, namely, Vietnam's Framework for Educational Researchers (VFER). VFER is expected to serve as a tool for Vietnamese educational researchers to self-evaluate their skills and support them in developing their career qualities from junior to senior career ladders. The framework includes four domains with ten subdomains and 28 indicators. To date, VFER has been implemented in some Vietnamese universities of pedagogical education. We expect that other research fields will look to VFER as a reference to build their own research capacity framework.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241240639"},"PeriodicalIF":0.9,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-04DOI: 10.1177/0193841x241238031
Douglas J. Besharov
In a 1987 article, Peter R. Rossi promulgated “The Iron Law of Evaluation and Other Metallic Rules.” The Metallic Laws were meant as an informal (and humorous) overstatement of the weakness of contemporary evaluations of social programs. Rossi’ s underlying worry was not so much about the state of evaluation technology in the abstract, but, rather, in its inability to advance our broad understanding of social problems and what to do about them---in other words, to make evaluation policy relevant. Rossi attributed the continuing failure to develop successful “large-scale social programs” to the failure to build a strong knowledge base for this kind of “social engineering.” The qualities of studies that enable such accumulated learning are variously labeled “external validity,” “generalizability,” “applicability,” or “transferability.” This Special Issue includes five papers that seek to explore and apply this understanding.
{"title":"Program Evaluation’s Path to Greater Policy Relevance: Learning From Rossi’s Iron Laws","authors":"Douglas J. Besharov","doi":"10.1177/0193841x241238031","DOIUrl":"https://doi.org/10.1177/0193841x241238031","url":null,"abstract":"In a 1987 article, Peter R. Rossi promulgated “The Iron Law of Evaluation and Other Metallic Rules.” The Metallic Laws were meant as an informal (and humorous) overstatement of the weakness of contemporary evaluations of social programs. Rossi’ s underlying worry was not so much about the state of evaluation technology in the abstract, but, rather, in its inability to advance our broad understanding of social problems and what to do about them---in other words, to make evaluation policy relevant. Rossi attributed the continuing failure to develop successful “large-scale social programs” to the failure to build a strong knowledge base for this kind of “social engineering.” The qualities of studies that enable such accumulated learning are variously labeled “external validity,” “generalizability,” “applicability,” or “transferability.” This Special Issue includes five papers that seek to explore and apply this understanding.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"12 1","pages":""},"PeriodicalIF":0.9,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140036990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-24DOI: 10.1177/0193841X241234412
William Rhodes, Gerald Gaes, William Sabol
When individuals are released from prison, they typically enter a period of post confinement community supervision. While under community supervision, their behaviors are subject to special conditions requiring them to report to supervisors and prohibiting certain behaviors such as drug and alcohol use. Many supervisees are returned to prison because they violate those special conditions, or because they commit minor crimes that would not result in prison were they not being supervised. But others are returned to prison for serious new crimes. We distinguish the two as nuisance behaviors (the former) and pernicious behaviors (the latter). Our research applies competing events survival analysis to distinguish a structural model that accounts for nuisance behaviors from a structural model that accounts for pernicious behaviors. We demonstrate that returning offenders to prison for technical violations and minor crimes may reduce the incidence of major crimes because the occurrence of nuisance behaviors and pernicious behaviors are highly correlated. Our findings support the theory that nuisance behaviors signal the likelihood of pernicious behaviors.
{"title":"Studying Parole Revocation Practices: Accounting for Dependency Between Competing Events.","authors":"William Rhodes, Gerald Gaes, William Sabol","doi":"10.1177/0193841X241234412","DOIUrl":"https://doi.org/10.1177/0193841X241234412","url":null,"abstract":"<p><p>When individuals are released from prison, they typically enter a period of post confinement community supervision. While under community supervision, their behaviors are subject to special conditions requiring them to report to supervisors and prohibiting certain behaviors such as drug and alcohol use. Many supervisees are returned to prison because they violate those special conditions, or because they commit minor crimes that would not result in prison were they not being supervised. But others are returned to prison for serious new crimes. We distinguish the two as nuisance behaviors (the former) and pernicious behaviors (the latter). Our research applies competing events survival analysis to distinguish a structural model that accounts for nuisance behaviors from a structural model that accounts for pernicious behaviors. We demonstrate that returning offenders to prison for technical violations and minor crimes may reduce the incidence of major crimes because the occurrence of nuisance behaviors and pernicious behaviors are highly correlated. Our findings support the theory that nuisance behaviors signal the likelihood of pernicious behaviors.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241234412"},"PeriodicalIF":0.9,"publicationDate":"2024-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139944549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-20DOI: 10.1177/0193841X241229106
Gregory Chernov
Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most qualified expert. This mechanism can be seamlessly integrated into the existing replication prediction market framework with minimal implementation costs. It relies on an objective rather than subjective process and unstructured expert opinions to effectively identify various influences contributing to the replication crisis.
{"title":"The Alternative Factors Leading to Replication Crisis: Prediction and Evaluation.","authors":"Gregory Chernov","doi":"10.1177/0193841X241229106","DOIUrl":"https://doi.org/10.1177/0193841X241229106","url":null,"abstract":"<p><p>Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most qualified expert. This mechanism can be seamlessly integrated into the existing replication prediction market framework with minimal implementation costs. It relies on an objective rather than subjective process and unstructured expert opinions to effectively identify various influences contributing to the replication crisis.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241229106"},"PeriodicalIF":0.9,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139913696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}