首页 > 最新文献

Proceedings of the 9th International Conference on Learning Analytics & Knowledge最新文献

英文 中文
Evaluating the Fairness of Predictive Student Models Through Slicing Analysis 通过切片分析评估预测学生模型的公平性
Josh Gardner, Christopher A. Brooks, R. Baker
Predictive modeling has been a core area of learning analytics research over the past decade, with such models currently deployed in a variety of educational contexts from MOOCs to K-12. However, analyses of the differential effectiveness of these models across demographic, identity, or other groups has been scarce. In this paper, we present a method for evaluating unfairness in predictive student models. We define this in terms of differential accuracy between subgroups, and measure it using a new metric we term the Absolute Between-ROC Area (ABROCA). We demonstrate the proposed method through a gender-based "slicing analysis" using five different models replicated from other works and a dataset of 44 unique MOOCs and over four million learners. Our results demonstrate (1) significant differences in model fairness according to (a) statistical algorithm and (b) feature set used; (2) that the gender imbalance ratio, curricular area, and specific course used for a model all display significant association with the value of the ABROCA statistic; and (3) that there is not evidence of a strict tradeoff between performance and fairness. This work provides a framework for quantifying and understanding how predictive models might inadvertently privilege, or disparately impact, different student subgroups. Furthermore, our results suggest that learning analytics researchers and practitioners can use slicing analysis to improve model fairness without necessarily sacrificing performance.1
在过去的十年中,预测建模一直是学习分析研究的核心领域,这些模型目前被部署在从mooc到K-12的各种教育环境中。然而,对这些模型在人口统计学、身份或其他群体中的差异有效性的分析很少。在本文中,我们提出了一种评估预测学生模型不公平性的方法。我们根据子组之间的差异精度来定义它,并使用我们称之为绝对roc间面积(ABROCA)的新度量来测量它。我们通过基于性别的“切片分析”来证明所提出的方法,使用了从其他作品中复制的五种不同模型和44个独特的mooc和超过400万学习者的数据集。我们的研究结果表明:(1)根据(a)统计算法和(b)所使用的特征集,模型公平性存在显著差异;(2)性别失衡比例、课程面积、模型所使用的具体课程均与ABROCA统计值呈显著相关;(3)没有证据表明在绩效和公平之间存在严格的权衡。这项工作提供了一个框架,用于量化和理解预测模型如何在不经意间特权或不同程度地影响不同的学生群体。此外,我们的结果表明,学习分析研究人员和从业者可以使用切片分析来提高模型公平性,而不必牺牲性能
{"title":"Evaluating the Fairness of Predictive Student Models Through Slicing Analysis","authors":"Josh Gardner, Christopher A. Brooks, R. Baker","doi":"10.1145/3303772.3303791","DOIUrl":"https://doi.org/10.1145/3303772.3303791","url":null,"abstract":"Predictive modeling has been a core area of learning analytics research over the past decade, with such models currently deployed in a variety of educational contexts from MOOCs to K-12. However, analyses of the differential effectiveness of these models across demographic, identity, or other groups has been scarce. In this paper, we present a method for evaluating unfairness in predictive student models. We define this in terms of differential accuracy between subgroups, and measure it using a new metric we term the Absolute Between-ROC Area (ABROCA). We demonstrate the proposed method through a gender-based \"slicing analysis\" using five different models replicated from other works and a dataset of 44 unique MOOCs and over four million learners. Our results demonstrate (1) significant differences in model fairness according to (a) statistical algorithm and (b) feature set used; (2) that the gender imbalance ratio, curricular area, and specific course used for a model all display significant association with the value of the ABROCA statistic; and (3) that there is not evidence of a strict tradeoff between performance and fairness. This work provides a framework for quantifying and understanding how predictive models might inadvertently privilege, or disparately impact, different student subgroups. Furthermore, our results suggest that learning analytics researchers and practitioners can use slicing analysis to improve model fairness without necessarily sacrificing performance.1","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127238269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 110
Technologies for automated analysis of co-located, real-life, physical learning spaces: Where are we now? 对位于同一地点、现实生活中的物理学习空间进行自动分析的技术:我们现在在哪里?
Y. H. V. Chua, J. Dauwels, S. Tan
The motivation for this paper is derived from the fact that there has been increasing interest among researchers and practitioners in developing technologies that capture, model and analyze learning and teaching experiences that take place beyond computer-based learning environments. In this paper, we review case studies of tools and technologies developed to collect and analyze data in educational settings, quantify learning and teaching processes and support assessment of learning and teaching in an automated fashion. We focus on pipelines that leverage information and data harnessed from physical spaces and/or integrates collected data across physical and digital spaces. Our review reveals a promising field of physical classroom analysis. We describe some trends and suggest potential future directions. Specifically, more research should be geared towards a) deployable and sustainable data collection set-ups in physical learning environments, b) teacher assessment, c) developing feedback and visualization systems and d) promoting inclusivity and generalizability of models across populations.
本文的动机源于这样一个事实,即研究人员和实践者对开发捕捉、建模和分析发生在计算机学习环境之外的学习和教学经验的技术越来越感兴趣。在本文中,我们回顾了一些工具和技术的案例研究,这些工具和技术用于收集和分析教育环境中的数据,量化学习和教学过程,并以自动化的方式支持学习和教学评估。我们专注于利用来自物理空间的信息和数据以及/或集成物理和数字空间收集的数据的管道。我们的综述揭示了物理课堂分析的一个有前途的领域。我们描述了一些趋势,并提出了潜在的未来方向。具体来说,更多的研究应该面向a)在物理学习环境中可部署和可持续的数据收集设置,b)教师评估,c)开发反馈和可视化系统,d)促进模型在人群中的包容性和普遍性。
{"title":"Technologies for automated analysis of co-located, real-life, physical learning spaces: Where are we now?","authors":"Y. H. V. Chua, J. Dauwels, S. Tan","doi":"10.1145/3303772.3303811","DOIUrl":"https://doi.org/10.1145/3303772.3303811","url":null,"abstract":"The motivation for this paper is derived from the fact that there has been increasing interest among researchers and practitioners in developing technologies that capture, model and analyze learning and teaching experiences that take place beyond computer-based learning environments. In this paper, we review case studies of tools and technologies developed to collect and analyze data in educational settings, quantify learning and teaching processes and support assessment of learning and teaching in an automated fashion. We focus on pipelines that leverage information and data harnessed from physical spaces and/or integrates collected data across physical and digital spaces. Our review reveals a promising field of physical classroom analysis. We describe some trends and suggest potential future directions. Specifically, more research should be geared towards a) deployable and sustainable data collection set-ups in physical learning environments, b) teacher assessment, c) developing feedback and visualization systems and d) promoting inclusivity and generalizability of models across populations.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116095288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 32
Differences in Student Trajectories via Filtered Time Series Analysis in an Immersive Virtual World 沉浸式虚拟世界中通过过滤时间序列分析的学生轨迹差异
J. Reilly, C. Dede
To scaffold students' investigations of an inquiry-based immersive virtual world for science education without undercutting the affordances an open-ended activity provides, this study explores ways time-stamped log files of groups' actions may enable the automatic generation of formative supports. Groups' logged actions in the virtual world are filtered via principal component analysis to provide a time series trajectory showing the rate of their investigative activities over time. This technique functions well in open-ended environments and examines the entire course of their experience in the virtual world instead of specific subsequences. Groups' trajectories are grouped via k-means clustering to identify different typical pathways taken through the immersive virtual world. These different approaches are then correlated with learning gains across several survey constructs (affective dimensions, ecosystem science content, understanding of causality, and experimental methods) to see how various trends are associated with different outcomes. Differences by teacher and school are explored to see how best to support inclusion and success of a diverse array of learners.
为了在不削弱开放式活动提供的支持的情况下,支持学生对科学教育中基于探究的沉浸式虚拟世界的调查,本研究探索了小组行为的时间戳日志文件可以自动生成形成性支持的方法。小组在虚拟世界中记录的行为通过主成分分析进行过滤,以提供一个时间序列轨迹,显示他们的调查活动随时间的变化速度。这一技术在开放式环境中发挥了很好的作用,并检查了他们在虚拟世界中的整个体验过程,而不是特定的子序列。群体的轨迹通过k-means聚类进行分组,以确定在沉浸式虚拟世界中采取的不同典型路径。然后将这些不同的方法与几个调查结构(情感维度、生态系统科学内容、对因果关系的理解和实验方法)中的学习收益相关联,以了解不同趋势如何与不同结果相关联。探讨了教师和学校的差异,以了解如何最好地支持各种学习者的包容和成功。
{"title":"Differences in Student Trajectories via Filtered Time Series Analysis in an Immersive Virtual World","authors":"J. Reilly, C. Dede","doi":"10.1145/3303772.3303832","DOIUrl":"https://doi.org/10.1145/3303772.3303832","url":null,"abstract":"To scaffold students' investigations of an inquiry-based immersive virtual world for science education without undercutting the affordances an open-ended activity provides, this study explores ways time-stamped log files of groups' actions may enable the automatic generation of formative supports. Groups' logged actions in the virtual world are filtered via principal component analysis to provide a time series trajectory showing the rate of their investigative activities over time. This technique functions well in open-ended environments and examines the entire course of their experience in the virtual world instead of specific subsequences. Groups' trajectories are grouped via k-means clustering to identify different typical pathways taken through the immersive virtual world. These different approaches are then correlated with learning gains across several survey constructs (affective dimensions, ecosystem science content, understanding of causality, and experimental methods) to see how various trends are associated with different outcomes. Differences by teacher and school are explored to see how best to support inclusion and success of a diverse array of learners.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"216 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123265326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Exploring Programming Semantic Analytics with Deep Learning Models 探索编程语义分析与深度学习模型
Yihan Lu, I-Han Hsiao
There are numerous studies have reported the effectiveness of example-based programming learning. However, less is explored recommending code examples with advanced Machine Learning-based models. In this work, we propose a new method to explore the semantic analytics between programming codes and the annotations. We hypothesize that these semantics analytics will capture mass amount of valuable information that can be used as features to build predictive models. We evaluated the proposed semantic analytics extraction method with multiple deep learning algorithms. Results showed that deep learning models outperformed other models and baseline in most cases. Further analysis indicated that in special cases, the proposed method outperformed deep learning models by restricting false-positive classifications.
有许多研究报告了基于示例的编程学习的有效性。然而,较少探索推荐基于高级机器学习模型的代码示例。在这项工作中,我们提出了一种新的方法来探索编程代码和注释之间的语义分析。我们假设这些语义分析将捕获大量有价值的信息,这些信息可以用作构建预测模型的特征。我们用多种深度学习算法评估了所提出的语义分析提取方法。结果表明,深度学习模型在大多数情况下优于其他模型和基线。进一步的分析表明,在特殊情况下,该方法通过限制假阳性分类而优于深度学习模型。
{"title":"Exploring Programming Semantic Analytics with Deep Learning Models","authors":"Yihan Lu, I-Han Hsiao","doi":"10.1145/3303772.3303823","DOIUrl":"https://doi.org/10.1145/3303772.3303823","url":null,"abstract":"There are numerous studies have reported the effectiveness of example-based programming learning. However, less is explored recommending code examples with advanced Machine Learning-based models. In this work, we propose a new method to explore the semantic analytics between programming codes and the annotations. We hypothesize that these semantics analytics will capture mass amount of valuable information that can be used as features to build predictive models. We evaluated the proposed semantic analytics extraction method with multiple deep learning algorithms. Results showed that deep learning models outperformed other models and baseline in most cases. Further analysis indicated that in special cases, the proposed method outperformed deep learning models by restricting false-positive classifications.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122689946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Exploring Learner Engagement Patterns in Teach-Outs Using Topic, Sentiment and On-topicness to Reflect on Pedagogy 用主题、情感和话题性探讨教学中学习者的参与模式
Wenfei Yan, Nia Dowell, Caitlin Holman, Stephen S. Welsh, Heeryung Choi, Christopher A. Brooks
MOOCs have developed into multiple learning design models with a wide range of objectives. Teach-Outs are one such example, aiming to drive meaningful discussions around topics of pressing social urgency without the use of formal assessments. Given this approach, it is crucial to evaluate learners' engagement in the discussion forum to understand their experiences. This paper presents a pilot study that applied unsupervised natural language processing techniques to understand what and how students engage in dialogue in a Teach-Out. We used topic modeling to discover the emerging topics in the discussion forums and evaluated the on-topicness of the discussions (i.e. the degree to which discussions were relevant to the Teach-Out content). We also applied content analysis to investigate the sentiments associated with the discussions. We have taken a step toward extracting structure from students' discussions to understand learning behaviors happen in the discussion forum. This is the first study to analyze discussion forums in a Teach-Out.
mooc已经发展成为具有广泛目标的多种学习设计模式。Teach-Outs就是这样一个例子,其目的是在不使用正式评估的情况下,围绕紧迫的社会紧迫性话题推动有意义的讨论。考虑到这种方法,评估学习者在论坛中的参与度以了解他们的经历是至关重要的。本文提出了一项试点研究,该研究应用无监督自然语言处理技术来了解学生在课堂教学中参与对话的内容和方式。我们使用主题建模来发现讨论论坛中出现的主题,并评估讨论的主题性(即讨论与讲授内容相关的程度)。我们还应用内容分析来调查与讨论相关的情绪。我们已经迈出了一步,从学生的讨论中提取结构,以了解论坛中发生的学习行为。这是第一个分析课堂讨论论坛的研究。
{"title":"Exploring Learner Engagement Patterns in Teach-Outs Using Topic, Sentiment and On-topicness to Reflect on Pedagogy","authors":"Wenfei Yan, Nia Dowell, Caitlin Holman, Stephen S. Welsh, Heeryung Choi, Christopher A. Brooks","doi":"10.1145/3303772.3303836","DOIUrl":"https://doi.org/10.1145/3303772.3303836","url":null,"abstract":"MOOCs have developed into multiple learning design models with a wide range of objectives. Teach-Outs are one such example, aiming to drive meaningful discussions around topics of pressing social urgency without the use of formal assessments. Given this approach, it is crucial to evaluate learners' engagement in the discussion forum to understand their experiences. This paper presents a pilot study that applied unsupervised natural language processing techniques to understand what and how students engage in dialogue in a Teach-Out. We used topic modeling to discover the emerging topics in the discussion forums and evaluated the on-topicness of the discussions (i.e. the degree to which discussions were relevant to the Teach-Out content). We also applied content analysis to investigate the sentiments associated with the discussions. We have taken a step toward extracting structure from students' discussions to understand learning behaviors happen in the discussion forum. This is the first study to analyze discussion forums in a Teach-Out.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114228821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
Predicting the Well-functioning of Learning Groups under Privacy Restrictions 预测隐私限制下学习小组的良好运作
Tobias Hecking, Dorian Doberstein, H. Hoppe
Establishing small learning groups in online courses is a possible way to foster collaborative knowledge building in an engaging and effective learning community. To enable group activities it is not enough to design collaborative tasks and to provide collaboration tools for online scenarios. Collaboration in such learning groups is prone to fail or even not to be initiated without explicit guidance. In the target situations, interventions and guiding mechanisms have to scale with a growing number of course participants. To achieve this under privacy constraints, we aim at identifying target indicators for well-functioning group work that do not rely on any kind of information about individual learners.
在在线课程中建立小型学习小组是在一个有吸引力和有效的学习社区中促进协作知识建设的一种可能方式。为了支持小组活动,仅仅为在线场景设计协作任务和提供协作工具是不够的。在这样的学习小组中,如果没有明确的指导,合作很容易失败,甚至不会开始。在目标情况下,干预措施和指导机制必须随着课程参与者人数的增加而扩大。为了在隐私限制下实现这一目标,我们的目标是确定不依赖于任何关于个体学习者的信息的功能良好的小组工作的目标指标。
{"title":"Predicting the Well-functioning of Learning Groups under Privacy Restrictions","authors":"Tobias Hecking, Dorian Doberstein, H. Hoppe","doi":"10.1145/3303772.3303826","DOIUrl":"https://doi.org/10.1145/3303772.3303826","url":null,"abstract":"Establishing small learning groups in online courses is a possible way to foster collaborative knowledge building in an engaging and effective learning community. To enable group activities it is not enough to design collaborative tasks and to provide collaboration tools for online scenarios. Collaboration in such learning groups is prone to fail or even not to be initiated without explicit guidance. In the target situations, interventions and guiding mechanisms have to scale with a growing number of course participants. To achieve this under privacy constraints, we aim at identifying target indicators for well-functioning group work that do not rely on any kind of information about individual learners.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"97 11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127999285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Semi-Automatic Generation of Intelligent Curricula to Facilitate Learning Analytics 半自动生成智能课程以促进学习分析
Angel Fiallos, X. Ochoa
Several Learning Analytics applications are limited by the cost of generating a computer understandable description of the course domain, what is called an Intelligent Curriculum. The following work contributes a novel approach to (semi-)automatically generate Intelligent Curriculum through ontologies extracted from existing learning materials such as digital books or web content. Through a series of natural language processing steps, the semi-structured information present in existing content is transformed into a concept-graph. This work also evaluates the proposed methodology by applying it to learning content for two different courses and measuring the quality of the extracted ontologies against manually generated ones. The results obtained suggest that the technique can be readily used to provide domain information to other Learning Analytics tools.
一些学习分析应用程序受到生成计算机可理解的课程域描述的成本的限制,即所谓的智能课程。下面的工作提供了一种(半)自动生成智能课程的新方法,通过从现有的学习材料(如电子书或网络内容)中提取本体。通过一系列的自然语言处理步骤,将存在于现有内容中的半结构化信息转化为概念图。这项工作还通过将所提出的方法应用于两门不同课程的学习内容,并根据手动生成的本体来衡量提取的本体的质量,从而评估所提出的方法。所获得的结果表明,该技术可以很容易地用于为其他学习分析工具提供领域信息。
{"title":"Semi-Automatic Generation of Intelligent Curricula to Facilitate Learning Analytics","authors":"Angel Fiallos, X. Ochoa","doi":"10.1145/3303772.3303834","DOIUrl":"https://doi.org/10.1145/3303772.3303834","url":null,"abstract":"Several Learning Analytics applications are limited by the cost of generating a computer understandable description of the course domain, what is called an Intelligent Curriculum. The following work contributes a novel approach to (semi-)automatically generate Intelligent Curriculum through ontologies extracted from existing learning materials such as digital books or web content. Through a series of natural language processing steps, the semi-structured information present in existing content is transformed into a concept-graph. This work also evaluates the proposed methodology by applying it to learning content for two different courses and measuring the quality of the extracted ontologies against manually generated ones. The results obtained suggest that the technique can be readily used to provide domain information to other Learning Analytics tools.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130865714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Student Centred Design of a Learning Analytics System 以学生为中心的学习分析系统设计
E. Quincey, C. Briggs, T. Kyriacou, R. Waller
Current Learning Analytics (LA) systems are primarily designed with University staff members as the target audience; very few are aimed at students, with almost none being developed with direct student involvement and undertaking a comprehensive evaluation. This paper describes a HEFCE funded project that has employed a variety of methods to engage students in the design, development and evaluation of a student facing LA dashboard. LA was integrated into the delivery of 4 undergraduate modules with 169 student sign-ups. The design of the dashboard uses a novel approach of trying to understand the reasons why students want to study at university and maps their engagement and predicted outcomes to these motivations, with weekly personalised notifications and feedback. Students are also given the choice of how to visualise the data either via a chart-based view or to be represented as themselves. A mixed-methods evaluation has shown that students' feelings of dependability and trust of the underlying analytics and data is variable. However, students were mostly positive about the usability and interface design of the system and almost all students once signed-up did interact with their LA. The majority of students could see how the LA system could support their learning and said that it would influence their behaviour. In some cases, this has had a direct impact on their levels of engagement. The main contribution of this paper is the transparent documentation of a User Centred Design approach that has produced forms of LA representation, recommendation and interaction design that go beyond those used in current similar systems and have been shown to motivate students and impact their learning behaviour.
当前的学习分析(LA)系统主要以大学教职员工为目标受众;很少有课程是针对学生的,几乎没有课程是由学生直接参与并进行全面评估的。本文描述了一个HEFCE资助的项目,该项目采用多种方法让学生参与设计、开发和评估面向LA的学生仪表板。LA被整合到4个本科模块中,有169名学生注册。仪表盘的设计采用了一种新颖的方法,试图了解学生想要在大学学习的原因,并将他们的参与度和预测结果映射到这些动机上,每周提供个性化的通知和反馈。学生还可以选择如何通过基于图表的视图或以自己的形式表示数据。一项混合方法评估表明,学生对基础分析和数据的可靠性和信任感是可变的。然而,学生们大多对系统的可用性和界面设计持积极态度,几乎所有注册的学生都与他们的LA进行了互动。大多数学生可以看到洛杉矶系统如何支持他们的学习,并表示这将影响他们的行为。在某些情况下,这对他们的参与水平产生了直接影响。本文的主要贡献是以用户为中心的设计方法的透明文档,该方法产生了超越当前类似系统中使用的LA表示,推荐和交互设计形式,并已被证明可以激励学生并影响他们的学习行为。
{"title":"Student Centred Design of a Learning Analytics System","authors":"E. Quincey, C. Briggs, T. Kyriacou, R. Waller","doi":"10.1145/3303772.3303793","DOIUrl":"https://doi.org/10.1145/3303772.3303793","url":null,"abstract":"Current Learning Analytics (LA) systems are primarily designed with University staff members as the target audience; very few are aimed at students, with almost none being developed with direct student involvement and undertaking a comprehensive evaluation. This paper describes a HEFCE funded project that has employed a variety of methods to engage students in the design, development and evaluation of a student facing LA dashboard. LA was integrated into the delivery of 4 undergraduate modules with 169 student sign-ups. The design of the dashboard uses a novel approach of trying to understand the reasons why students want to study at university and maps their engagement and predicted outcomes to these motivations, with weekly personalised notifications and feedback. Students are also given the choice of how to visualise the data either via a chart-based view or to be represented as themselves. A mixed-methods evaluation has shown that students' feelings of dependability and trust of the underlying analytics and data is variable. However, students were mostly positive about the usability and interface design of the system and almost all students once signed-up did interact with their LA. The majority of students could see how the LA system could support their learning and said that it would influence their behaviour. In some cases, this has had a direct impact on their levels of engagement. The main contribution of this paper is the transparent documentation of a User Centred Design approach that has produced forms of LA representation, recommendation and interaction design that go beyond those used in current similar systems and have been shown to motivate students and impact their learning behaviour.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"29 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123875448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 38
Knowledge Query Network for Knowledge Tracing: How Knowledge Interacts with Skills 面向知识追踪的知识查询网络:知识与技能的相互作用
Jinseok Lee, D. Yeung
Knowledge Tracing (KT) is to trace the knowledge of students as they solve a sequence of problems represented by their related skills. This involves abstract concepts of students' states of knowledge and the interactions between those states and skills. Therefore, a KT model is designed to predict whether students will give correct answers and to describe such abstract concepts. However, existing methods either give relatively low prediction accuracy or fail to explain those concepts intuitively. In this paper, we propose a new model called Knowledge Query Network (KQN) to solve these problems. KQN uses neural networks to encode student learning activities into knowledge state and skill vectors, and models the interactions between the two types of vectors with the dot product. Through this, we introduce a novel concept called probabilistic skill similarity that relates the pairwise cosine and Euclidean distances between skill vectors to the odds ratios of the corresponding skills, which makes KQN interpretable and intuitive. On four public datasets, we have carried out experiments to show the following: 1. KQN outperforms all the existing KT models based on prediction accuracy. 2. The interaction between the knowledge state and skills can be visualized for interpretation. 3. Based on probabilistic skill similarity, a skill domain can be analyzed with clustering using the distances between the skill vectors of KQN. 4. For different values of the vector space dimensionality, KQN consistently exhibits high prediction accuracy and a strong positive correlation between the distance matrices of the skill vectors.
知识追踪(KT)是指在学生解决一系列以相关技能为代表的问题时,对学生的知识进行追踪。这涉及到学生知识状态的抽象概念,以及这些状态和技能之间的相互作用。因此,设计了KT模型来预测学生是否会给出正确的答案,并描述这些抽象的概念。然而,现有的方法要么给出相对较低的预测精度,要么不能直观地解释这些概念。为了解决这些问题,本文提出了一种新的知识查询网络(KQN)模型。KQN利用神经网络将学生的学习活动编码为知识状态和技能向量,并用点积对两类向量之间的相互作用进行建模。通过这一点,我们引入了一个新的概念,称为概率技能相似性,将技能向量之间的成对余弦和欧几里得距离与相应技能的比值比联系起来,使KQN具有可解释性和直观性。在四个公共数据集上,我们进行了实验,结果表明:1。基于预测精度,KQN优于所有现有的KT模型。2. 知识状态和技能之间的相互作用可以可视化,以便解释。3.基于概率技能相似度,利用KQN的技能向量之间的距离对技能域进行聚类分析。4. 对于不同的向量空间维数,KQN均具有较高的预测精度,且技能向量距离矩阵之间具有较强的正相关关系。
{"title":"Knowledge Query Network for Knowledge Tracing: How Knowledge Interacts with Skills","authors":"Jinseok Lee, D. Yeung","doi":"10.1145/3303772.3303786","DOIUrl":"https://doi.org/10.1145/3303772.3303786","url":null,"abstract":"Knowledge Tracing (KT) is to trace the knowledge of students as they solve a sequence of problems represented by their related skills. This involves abstract concepts of students' states of knowledge and the interactions between those states and skills. Therefore, a KT model is designed to predict whether students will give correct answers and to describe such abstract concepts. However, existing methods either give relatively low prediction accuracy or fail to explain those concepts intuitively. In this paper, we propose a new model called Knowledge Query Network (KQN) to solve these problems. KQN uses neural networks to encode student learning activities into knowledge state and skill vectors, and models the interactions between the two types of vectors with the dot product. Through this, we introduce a novel concept called probabilistic skill similarity that relates the pairwise cosine and Euclidean distances between skill vectors to the odds ratios of the corresponding skills, which makes KQN interpretable and intuitive. On four public datasets, we have carried out experiments to show the following: 1. KQN outperforms all the existing KT models based on prediction accuracy. 2. The interaction between the knowledge state and skills can be visualized for interpretation. 3. Based on probabilistic skill similarity, a skill domain can be analyzed with clustering using the distances between the skill vectors of KQN. 4. For different values of the vector space dimensionality, KQN consistently exhibits high prediction accuracy and a strong positive correlation between the distance matrices of the skill vectors.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128793391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 35
Where You Are, Not What You See: The Impact of Learning Environment on Mind Wandering and Material Retention 你在哪里,而不是你看到什么:学习环境对走神和记忆的影响
Trish L. Varao-Sousa, Caitlin Mills, A. Kingstone
Online lectures are an increasingly popular tool for learning, yet research on instructor visibility during an online lecture, and students' environmental settings, has not been well-explored. The current study addresses this gap in the literature by experimentally manipulating online display format and social learning settings to understand their influence on student learning and mind-wandering experiences. Results suggest that instructor visibility within an online lecture does not impact students' MW or retention performance. However, we found some evidence that students' social setting during viewing has an impact on MW (p = .05). Specifically, students who watched the lecture in a classroom with others reported significantly more MW than students who watched the lecture alone. Finally, social setting also moderated the negative relationship between MW and material retention. Our results demonstrate that learning experiences during online lectures can vary based on where, and with whom, the lectures are watched.
在线讲座是一种越来越受欢迎的学习工具,然而,关于在线讲座中教师可见度和学生环境设置的研究还没有得到很好的探索。目前的研究通过实验操纵在线显示格式和社会学习环境来解决这一文献空白,以了解它们对学生学习和走神体验的影响。结果表明,讲师在在线讲座中的可见度不会影响学生的MW或保留性能。然而,我们发现了一些证据表明,学生在观看时的社会环境对学习能力有影响(p = 0.05)。具体来说,在教室里与其他人一起观看讲座的学生报告的MW明显高于单独观看讲座的学生。最后,社会环境也会调节记忆能力与材料记忆的负向关系。我们的研究结果表明,在线讲座的学习体验可能会因观看讲座的地点和对象而有所不同。
{"title":"Where You Are, Not What You See: The Impact of Learning Environment on Mind Wandering and Material Retention","authors":"Trish L. Varao-Sousa, Caitlin Mills, A. Kingstone","doi":"10.1145/3303772.3303824","DOIUrl":"https://doi.org/10.1145/3303772.3303824","url":null,"abstract":"Online lectures are an increasingly popular tool for learning, yet research on instructor visibility during an online lecture, and students' environmental settings, has not been well-explored. The current study addresses this gap in the literature by experimentally manipulating online display format and social learning settings to understand their influence on student learning and mind-wandering experiences. Results suggest that instructor visibility within an online lecture does not impact students' MW or retention performance. However, we found some evidence that students' social setting during viewing has an impact on MW (p = .05). Specifically, students who watched the lecture in a classroom with others reported significantly more MW than students who watched the lecture alone. Finally, social setting also moderated the negative relationship between MW and material retention. Our results demonstrate that learning experiences during online lectures can vary based on where, and with whom, the lectures are watched.","PeriodicalId":382957,"journal":{"name":"Proceedings of the 9th International Conference on Learning Analytics & Knowledge","volume":"128 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116621672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
期刊
Proceedings of the 9th International Conference on Learning Analytics & Knowledge
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1