用于评估性能预测系统的与时间相关的指标

IF 2.4 Q3 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS International Journal of Information and Learning Technology Pub Date : 2022-10-03 DOI:10.1108/ijilt-07-2022-0149
Amal Ben Soussia, Chahrazed Labba, A. Roussanaly, A. Boyer
{"title":"用于评估性能预测系统的与时间相关的指标","authors":"Amal Ben Soussia, Chahrazed Labba, A. Roussanaly, A. Boyer","doi":"10.1108/ijilt-07-2022-0149","DOIUrl":null,"url":null,"abstract":"PurposeThe goal is to assess performance prediction systems (PPS) that are used to assist at-risk learners.Design/methodology/approachThe authors propose time-dependent metrics including earliness and stability. The authors investigate the relationships between the various temporal metrics and the precision metrics in order to identify the key earliness points in the prediction process. Authors propose an algorithm for computing earliness. Furthermore, the authors propose using an earliness-stability score (ESS) to investigate the relationship between the earliness of a classifier and its stability. The ESS is used to examine the trade-off between only time-dependent metrics. The aim is to compare its use to the earliness-accuracy score (EAS).FindingsStability and accuracy are proportional when the system's accuracy increases or decreases over time. However, when the accuracy stagnates or varies slightly, the system's stability is decreasing rather than stagnating. As a result, the use of ESS and EAS is complementary and allows for a better definition of the point of earliness in time by studying the relation-ship between earliness and accuracy on the one hand and earliness and stability on the other.Originality/valueWhen evaluating the performance of PPS, the temporal dimension is an important factor that is overlooked by traditional measures current metrics are not well suited to assessing PPS’s ability to predict correctly at the earliest, as well as monitoring predictions stability and evolution over time. Thus, in this work, the authors propose time-dependent metrics, including earliness, stability and the trade-offs, with objective to assess PPS over time.","PeriodicalId":51872,"journal":{"name":"International Journal of Information and Learning Technology","volume":" ","pages":""},"PeriodicalIF":2.4000,"publicationDate":"2022-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Time-dependent metrics to assess performance prediction systems\",\"authors\":\"Amal Ben Soussia, Chahrazed Labba, A. Roussanaly, A. Boyer\",\"doi\":\"10.1108/ijilt-07-2022-0149\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"PurposeThe goal is to assess performance prediction systems (PPS) that are used to assist at-risk learners.Design/methodology/approachThe authors propose time-dependent metrics including earliness and stability. The authors investigate the relationships between the various temporal metrics and the precision metrics in order to identify the key earliness points in the prediction process. Authors propose an algorithm for computing earliness. Furthermore, the authors propose using an earliness-stability score (ESS) to investigate the relationship between the earliness of a classifier and its stability. The ESS is used to examine the trade-off between only time-dependent metrics. The aim is to compare its use to the earliness-accuracy score (EAS).FindingsStability and accuracy are proportional when the system's accuracy increases or decreases over time. However, when the accuracy stagnates or varies slightly, the system's stability is decreasing rather than stagnating. As a result, the use of ESS and EAS is complementary and allows for a better definition of the point of earliness in time by studying the relation-ship between earliness and accuracy on the one hand and earliness and stability on the other.Originality/valueWhen evaluating the performance of PPS, the temporal dimension is an important factor that is overlooked by traditional measures current metrics are not well suited to assessing PPS’s ability to predict correctly at the earliest, as well as monitoring predictions stability and evolution over time. Thus, in this work, the authors propose time-dependent metrics, including earliness, stability and the trade-offs, with objective to assess PPS over time.\",\"PeriodicalId\":51872,\"journal\":{\"name\":\"International Journal of Information and Learning Technology\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":2.4000,\"publicationDate\":\"2022-10-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Information and Learning Technology\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1108/ijilt-07-2022-0149\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Information and Learning Technology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1108/ijilt-07-2022-0149","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

目的:目的是评估用于帮助有风险的学习者的表现预测系统(PPS)。作者提出了与时间相关的指标,包括早期性和稳定性。作者研究了各种时间指标与精度指标之间的关系,以确定预测过程中的关键早期点。作者提出了一种计算早期的算法。此外,作者提出使用早期稳定性评分(ESS)来研究分类器的早期性与其稳定性之间的关系。ESS仅用于检查与时间相关的度量之间的权衡。目的是将其与早期准确性评分(EAS)进行比较。当系统的精度随时间增加或减少时,稳定性和精度成正比。然而,当精度停滞或略有变化时,系统的稳定性不是停滞而是下降。因此,ESS和EAS的使用是互补的,通过研究早期和准确性以及早期和稳定性之间的关系,可以更好地定义时间的早期点。原创性/价值在评估PPS的表现时,时间维度是一个被传统测量方法所忽视的重要因素,目前的指标并不适合于评估PPS最早正确预测的能力,也不适合监测预测的稳定性和随时间的演变。因此,在这项工作中,作者提出了与时间相关的指标,包括早期性、稳定性和权衡,目的是随着时间的推移评估PPS。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Time-dependent metrics to assess performance prediction systems
PurposeThe goal is to assess performance prediction systems (PPS) that are used to assist at-risk learners.Design/methodology/approachThe authors propose time-dependent metrics including earliness and stability. The authors investigate the relationships between the various temporal metrics and the precision metrics in order to identify the key earliness points in the prediction process. Authors propose an algorithm for computing earliness. Furthermore, the authors propose using an earliness-stability score (ESS) to investigate the relationship between the earliness of a classifier and its stability. The ESS is used to examine the trade-off between only time-dependent metrics. The aim is to compare its use to the earliness-accuracy score (EAS).FindingsStability and accuracy are proportional when the system's accuracy increases or decreases over time. However, when the accuracy stagnates or varies slightly, the system's stability is decreasing rather than stagnating. As a result, the use of ESS and EAS is complementary and allows for a better definition of the point of earliness in time by studying the relation-ship between earliness and accuracy on the one hand and earliness and stability on the other.Originality/valueWhen evaluating the performance of PPS, the temporal dimension is an important factor that is overlooked by traditional measures current metrics are not well suited to assessing PPS’s ability to predict correctly at the earliest, as well as monitoring predictions stability and evolution over time. Thus, in this work, the authors propose time-dependent metrics, including earliness, stability and the trade-offs, with objective to assess PPS over time.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
International Journal of Information and Learning Technology
International Journal of Information and Learning Technology COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS-
CiteScore
6.10
自引率
3.30%
发文量
33
期刊介绍: International Journal of Information and Learning Technology (IJILT) provides a forum for the sharing of the latest theories, applications, and services related to planning, developing, managing, using, and evaluating information technologies in administrative, academic, and library computing, as well as other educational technologies. Submissions can include research: -Illustrating and critiquing educational technologies -New uses of technology in education -Issue-or results-focused case studies detailing examples of technology applications in higher education -In-depth analyses of the latest theories, applications and services in the field The journal provides wide-ranging and independent coverage of the management, use and integration of information resources and learning technologies.
期刊最新文献
Development of an Automated Hall Effect Experimentation Method for the Electrical Characterization of Thin Films Deteksi Tingkat Kematangan Buah Pinang Menggunakan Metode Support Vector Machine Berdasarkan Warna Dan Tekstur Analisis Kinerja Mikrokomputer Raspberry Pi Pada Smart Greenhouse Berbasis Internet Of Things (IoT) Menggunakan Algoritma Naive Baye SISTEM PENDUKUNG KEPUTUSAN PENENTUAN GURU BERPRESTASI MENGGUNAKAN METODE TOPSIS (STUDI KASUS: DINAS PPO KAB. TTU) Analisis Kepuasan Pengguna Terhadap Penerapan Sistem Informasi Terpadu Layanan Prodi (SIPLO) Menggunakan End User Computing Satisfaction (EUCS)
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1