Yukyeong Song, Wanli Xing, Xiaoyi Tian, Chenglu Li
Mathematical discussions have become a popular educational strategy to promote math literacy. While some studies have associated math literacy with linguistic factors such as verbal ability and phonological skills, no studies have examined the relationship between linguistic synchrony and math literacy. In this study, we modeled linguistic synchrony and students’ math literacy from 20,776 online mathematical discussion threads between students and facilitators. We conducted Cross-Recurrence Quantification Analysis (CRQA) to calculate linguistic synchrony within each thread. The statistical testing result comparing CRQA indices between high and low math literacy groups shows that students with high math literacy have a significantly higher Recurrence Rate (RR), Number of Recurrence Lines (NRLINE), and the average Length of lines (L), but lower Determinism (DET) and normalized Entropy (rENTR). This result implies that students with high math literacy are more likely to share common words with facilitators, but they would paraphrase them. On the other hand, students with low math literacy tend to repeat the exact same phrases from the facilitators. The findings provide a better understanding of mathematical discussions and can potentially guide teachers in promoting effective mathematical discussions.
{"title":"Are We on the Same Page? Modeling Linguistic Synchrony and Math Literacy in Mathematical Discussions","authors":"Yukyeong Song, Wanli Xing, Xiaoyi Tian, Chenglu Li","doi":"10.1145/3576050.3576082","DOIUrl":"https://doi.org/10.1145/3576050.3576082","url":null,"abstract":"Mathematical discussions have become a popular educational strategy to promote math literacy. While some studies have associated math literacy with linguistic factors such as verbal ability and phonological skills, no studies have examined the relationship between linguistic synchrony and math literacy. In this study, we modeled linguistic synchrony and students’ math literacy from 20,776 online mathematical discussion threads between students and facilitators. We conducted Cross-Recurrence Quantification Analysis (CRQA) to calculate linguistic synchrony within each thread. The statistical testing result comparing CRQA indices between high and low math literacy groups shows that students with high math literacy have a significantly higher Recurrence Rate (RR), Number of Recurrence Lines (NRLINE), and the average Length of lines (L), but lower Determinism (DET) and normalized Entropy (rENTR). This result implies that students with high math literacy are more likely to share common words with facilitators, but they would paraphrase them. On the other hand, students with low math literacy tend to repeat the exact same phrases from the facilitators. The findings provide a better understanding of mathematical discussions and can potentially guide teachers in promoting effective mathematical discussions.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130063268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
ByeongJo Kong, Erik Hemberg, Ana Bell, Una-May O’Reilly
Problem-solving approaches are an essential part of learning. Knowing how students approach solving problems can help instructors improve their instructional designs and effectively guide the learning process of students. We propose a natural language processing (NLP) driven method to capture online learners’ problem-solving approaches at scale while using Massive Open Online Courses (MOOCs) as a learning platform. We employ an online survey to gather data, NLP techniques, and existing educational theories to investigate this in the lens of both computer science and education. The paper shows how NLP techniques, i.e. preprocessing, topic modeling, and text summarization, must be tuned to extract information from a large-scale text corpus. The proposed method discovered 18 problem-solving approaches from the text data, such as using pen and paper, peer learning, trial and error, etc. We also observed topics that appear over the years, such as clarifying code logic, watching videos, etc. We observed that students heavily rely on "tools" for solving programming problems and can expect that such selection of methods can vary depending on the type of task.
{"title":"Investigating Student's Problem-solving Approaches in MOOCs using Natural Language Processing","authors":"ByeongJo Kong, Erik Hemberg, Ana Bell, Una-May O’Reilly","doi":"10.1145/3576050.3576091","DOIUrl":"https://doi.org/10.1145/3576050.3576091","url":null,"abstract":"Problem-solving approaches are an essential part of learning. Knowing how students approach solving problems can help instructors improve their instructional designs and effectively guide the learning process of students. We propose a natural language processing (NLP) driven method to capture online learners’ problem-solving approaches at scale while using Massive Open Online Courses (MOOCs) as a learning platform. We employ an online survey to gather data, NLP techniques, and existing educational theories to investigate this in the lens of both computer science and education. The paper shows how NLP techniques, i.e. preprocessing, topic modeling, and text summarization, must be tuned to extract information from a large-scale text corpus. The proposed method discovered 18 problem-solving approaches from the text data, such as using pen and paper, peer learning, trial and error, etc. We also observed topics that appear over the years, such as clarifying code logic, watching videos, etc. We observed that students heavily rely on \"tools\" for solving programming problems and can expect that such selection of methods can vary depending on the type of task.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116481908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hilário Oliveira, Rafael Ferreira Mello, Bruno Alexandre Barreiros Rosa, Mladen Raković, Pericles Miranda, T. Cordeiro, Seiji Isotani, I. Bittencourt, D. Gašević
Textual cohesion is an essential aspect of a formally written text, related to linguistic mechanisms that connect elements such as words, sentences, and paragraphs. Several studies have proposed approaches to estimate textual cohesion in essays automatically. There is limited research that aims to study the extent to which the use of machine learning approaches can predict the textual cohesion of essays written in different languages (not just English). This paper reports on the findings of a study that aimed to propose and evaluate approaches that automatically estimate the cohesion of essays in Portuguese and English. The study proposed regression-based models grounded in conventional feature-based machine learning methods and deep learning-based pre-trained language models. The study also examined the explainability of automated approaches to scrutinize their predictions. We analyzed two datasets composed of 4,570 (Portuguese) and 7,101 (English) essays. The results demonstrate that a deep learning-based model achieved the best performance on both datasets with a moderate Pearson correlation with human-rated cohesion scores. However, the explainability of the automatic cohesion estimations based on conventional machine learning models offered a stronger potential than that of the deep learning model.
{"title":"Towards explainable prediction of essay cohesion in Portuguese and English","authors":"Hilário Oliveira, Rafael Ferreira Mello, Bruno Alexandre Barreiros Rosa, Mladen Raković, Pericles Miranda, T. Cordeiro, Seiji Isotani, I. Bittencourt, D. Gašević","doi":"10.1145/3576050.3576152","DOIUrl":"https://doi.org/10.1145/3576050.3576152","url":null,"abstract":"Textual cohesion is an essential aspect of a formally written text, related to linguistic mechanisms that connect elements such as words, sentences, and paragraphs. Several studies have proposed approaches to estimate textual cohesion in essays automatically. There is limited research that aims to study the extent to which the use of machine learning approaches can predict the textual cohesion of essays written in different languages (not just English). This paper reports on the findings of a study that aimed to propose and evaluate approaches that automatically estimate the cohesion of essays in Portuguese and English. The study proposed regression-based models grounded in conventional feature-based machine learning methods and deep learning-based pre-trained language models. The study also examined the explainability of automated approaches to scrutinize their predictions. We analyzed two datasets composed of 4,570 (Portuguese) and 7,101 (English) essays. The results demonstrate that a deep learning-based model achieved the best performance on both datasets with a moderate Pearson correlation with human-rated cohesion scores. However, the explainability of the automatic cohesion estimations based on conventional machine learning models offered a stronger potential than that of the deep learning model.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125956410","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Learning analytics approaches to profiling students based on their study behaviour remain limited in how they integrate temporality and change. To advance this area of work, the current study examines profiles of change in student study behaviour in a blended undergraduate engineering course. The study is conceptualised through complex dynamical systems theory and its applications in psychological and cognitive science research. Students were profiled based on the changes in their behaviour as observed in clickstream data. Measure of entropy in the recurrence of student behaviour was used to indicate the change of a student state, consistent with the evidence from cognitive sciences. Student trajectories of weekly entropy values were clustered to identify distinct profiles. Three patterns were identified: stable weekly study, steep changes in weekly study, and moderate changes in weekly study. The students with steep changes in their weekly study activity had lower exam grades and showed destabilisation of weekly behaviour earlier in the course. The study investigated the relationships between these profiles of change, student performance, and other approaches to learner profiling, such as self-reported measures of self-regulated learning, and profiles based on the sequences of learning actions.
{"title":"Student Profiles of Change in a University Course: A Complex Dynamical Systems Perspective","authors":"Oleksandra Poquet, J. Jovanović, Abelardo Pardo","doi":"10.1145/3576050.3576077","DOIUrl":"https://doi.org/10.1145/3576050.3576077","url":null,"abstract":"Learning analytics approaches to profiling students based on their study behaviour remain limited in how they integrate temporality and change. To advance this area of work, the current study examines profiles of change in student study behaviour in a blended undergraduate engineering course. The study is conceptualised through complex dynamical systems theory and its applications in psychological and cognitive science research. Students were profiled based on the changes in their behaviour as observed in clickstream data. Measure of entropy in the recurrence of student behaviour was used to indicate the change of a student state, consistent with the evidence from cognitive sciences. Student trajectories of weekly entropy values were clustered to identify distinct profiles. Three patterns were identified: stable weekly study, steep changes in weekly study, and moderate changes in weekly study. The students with steep changes in their weekly study activity had lower exam grades and showed destabilisation of weekly behaviour earlier in the course. The study investigated the relationships between these profiles of change, student performance, and other approaches to learner profiling, such as self-reported measures of self-regulated learning, and profiles based on the sequences of learning actions.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125142247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Are there structures underlying student work that are universal across every open-ended task? We demonstrate that, across many subjects and assignment types, the probability distribution underlying student-generated open-ended work is close to Zipf’s Law. Inferring this latent structure for classroom assignments can help learning analytics researchers, instruction designers, and educators understand the landscape of various student approaches, assess the complexity of assignments, and prioritise pedagogical attention. However, typical classrooms are way too small to witness even the contour of the Zipfian pattern, and it is generally impossible to perform inference for Zipf’s law from such small number of samples. We formalise this difficult task as the Zipf Inference Challenge: (1) Infer the ordering of student-generated works by their underlying probabilities, and (2) Estimate the shape parameter of the underlying distribution in a typical-sized classroom. Our key insight in addressing this challenge is to leverage the densities of the student response landscapes represented by semantic similarity. We show that our “Semantic Density Estimation” method is able to do a much better job at inferring the latent Zipf shape and the probability-ordering of student responses for real world education datasets.
{"title":"The Student Zipf Theory: Inferring Latent Structures in Open-Ended Student Work To Help Educators","authors":"Yunsung Kim, C. Piech","doi":"10.1145/3576050.3576116","DOIUrl":"https://doi.org/10.1145/3576050.3576116","url":null,"abstract":"Are there structures underlying student work that are universal across every open-ended task? We demonstrate that, across many subjects and assignment types, the probability distribution underlying student-generated open-ended work is close to Zipf’s Law. Inferring this latent structure for classroom assignments can help learning analytics researchers, instruction designers, and educators understand the landscape of various student approaches, assess the complexity of assignments, and prioritise pedagogical attention. However, typical classrooms are way too small to witness even the contour of the Zipfian pattern, and it is generally impossible to perform inference for Zipf’s law from such small number of samples. We formalise this difficult task as the Zipf Inference Challenge: (1) Infer the ordering of student-generated works by their underlying probabilities, and (2) Estimate the shape parameter of the underlying distribution in a typical-sized classroom. Our key insight in addressing this challenge is to leverage the densities of the student response landscapes represented by semantic similarity. We show that our “Semantic Density Estimation” method is able to do a much better job at inferring the latent Zipf shape and the probability-ordering of student responses for real world education datasets.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127678237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Paul Hur, Nessrine Machaka, Christina Krist, Nigel Bosch
While classroom video data are detailed sources for mining student learning insights, their complex and unstructured nature makes them less than straightforward for researchers to analyze. In this paper, we compared the differences between the processes of expert-informed manual feature engineering and automated feature engineering using positional data for predicting student group interaction in four middle school and high school mathematics classroom videos. Our results highlighted notable differences, including improved model accuracy for the combined (manual features + automated features) models compared to the only-manual-features models (mean AUC = .778 vs. .706) at the cost of feature interpretability, increased number of features for automated feature engineering (1523 vs. 178), and engineering approach (domain-agnostic in automated vs. domain-knowledge-informed in manual). We carried out feature importance analyses and discuss the implications of the results for potentially augmenting human perspectives about qualitatively coding classroom video data by confirming and expanding views on which body areas and characteristics may be relevant to the target interaction behavior. Lastly, we discuss our study’s limitations and future work.
虽然课堂视频数据是挖掘学生学习见解的详细来源，但它们的复杂性和非结构化性质使研究人员无法直接分析它们。在本文中，我们比较了专家指导下的手动特征工程和使用位置数据的自动特征工程在四个初中和高中数学课堂视频中预测学生群体互动过程中的差异。我们的结果突出了显著的差异，包括以特征可解释性为代价，提高了组合(手动特征+自动化特征)模型的模型精度(平均AUC = .778 vs. 706)，增加了自动化特征工程的特征数量(1523 vs. 178)，以及工程方法(自动化的领域不可知vs.手动的领域知识告知)。我们进行了特征重要性分析，并讨论了结果的影响，通过确认和扩展关于哪些身体区域和特征可能与目标交互行为相关的观点，潜在地增强了人类对定性编码课堂视频数据的看法。最后，我们讨论了本研究的局限性和未来的工作。
{"title":"Informing Expert Feature Engineering through Automated Approaches: Implications for Coding Qualitative Classroom Video Data","authors":"Paul Hur, Nessrine Machaka, Christina Krist, Nigel Bosch","doi":"10.1145/3576050.3576090","DOIUrl":"https://doi.org/10.1145/3576050.3576090","url":null,"abstract":"While classroom video data are detailed sources for mining student learning insights, their complex and unstructured nature makes them less than straightforward for researchers to analyze. In this paper, we compared the differences between the processes of expert-informed manual feature engineering and automated feature engineering using positional data for predicting student group interaction in four middle school and high school mathematics classroom videos. Our results highlighted notable differences, including improved model accuracy for the combined (manual features + automated features) models compared to the only-manual-features models (mean AUC = .778 vs. .706) at the cost of feature interpretability, increased number of features for automated feature engineering (1523 vs. 178), and engineering approach (domain-agnostic in automated vs. domain-knowledge-informed in manual). We carried out feature importance analyses and discuss the implications of the results for potentially augmenting human perspectives about qualitatively coding classroom video data by confirming and expanding views on which body areas and characteristics may be relevant to the target interaction behavior. Lastly, we discuss our study’s limitations and future work.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129963916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Danielle R. Thomas, Xinyu Yang, Shivang Gupta, A. Adeniran, Elizabeth Mclaughlin, K. Koedinger
Tutoring is among the most impactful educational influences on student achievement, with perhaps the greatest promise of combating student learning loss. Due to its high impact, organizations are rapidly developing tutoring programs and discovering a common problem- a shortage of qualified, experienced tutors. This mixed methods investigation focuses on the impact of short (∼15 min.), online lessons in which tutors participate in situational judgment tests based on everyday tutoring scenarios. We developed three lessons on strategies for supporting student self-efficacy and motivation and tested them with 80 tutors from a national, online tutoring organization. Using a mixed-effects logistic regression model, we found a statistically significant learning effect indicating tutors performed about 20% higher post-instruction than pre-instruction (β = 0.811, p < 0.01). Tutors scored ∼30% better on selected compared to constructed responses at posttest with evidence that tutors are learning from selected-response questions alone. Learning analytics and qualitative feedback suggest future design modifications for larger scale deployment, such as creating more authentically challenging selected-response options, capturing common misconceptions using learnersourced data, and varying modalities of scenario delivery with the aim of maintaining learning gains while reducing time and effort for tutor participants and trainers.
辅导是对学生成绩影响最大的教育方式之一，可能最有希望消除学生的学习损失。由于它的高影响力，组织正在迅速发展辅导项目，并发现了一个共同的问题——缺乏合格的、有经验的导师。这项混合方法调查的重点是短期(~ 15分钟)在线课程的影响，在这些课程中，教师参与基于日常辅导场景的情景判断测试。我们开发了三门课程，内容是关于支持学生自我效能感和动机的策略，并由一家全国性在线辅导机构的80名导师进行了测试。使用混合效应logistic回归模型，我们发现导师在教学后的学习效果比教学前高20%左右(β = 0.811, p < 0.01)。在后测中，导师在选择回答上的得分比构建回答高30%，有证据表明导师只从选择回答问题中学习。学习分析和定性反馈建议未来进行更大规模部署的设计修改，例如创建更具挑战性的选择响应选项，使用学习者来源的数据捕获常见的误解，以及以保持学习成果为目标的不同模式的场景交付，同时减少导师参与者和培训师的时间和精力。
{"title":"When the Tutor Becomes the Student: Design and Evaluation of Efficient Scenario-based Lessons for Tutors","authors":"Danielle R. Thomas, Xinyu Yang, Shivang Gupta, A. Adeniran, Elizabeth Mclaughlin, K. Koedinger","doi":"10.1145/3576050.3576089","DOIUrl":"https://doi.org/10.1145/3576050.3576089","url":null,"abstract":"Tutoring is among the most impactful educational influences on student achievement, with perhaps the greatest promise of combating student learning loss. Due to its high impact, organizations are rapidly developing tutoring programs and discovering a common problem- a shortage of qualified, experienced tutors. This mixed methods investigation focuses on the impact of short (∼15 min.), online lessons in which tutors participate in situational judgment tests based on everyday tutoring scenarios. We developed three lessons on strategies for supporting student self-efficacy and motivation and tested them with 80 tutors from a national, online tutoring organization. Using a mixed-effects logistic regression model, we found a statistically significant learning effect indicating tutors performed about 20% higher post-instruction than pre-instruction (β = 0.811, p < 0.01). Tutors scored ∼30% better on selected compared to constructed responses at posttest with evidence that tutors are learning from selected-response questions alone. Learning analytics and qualitative feedback suggest future design modifications for larger scale deployment, such as creating more authentically challenging selected-response options, capturing common misconceptions using learnersourced data, and varying modalities of scenario delivery with the aim of maintaining learning gains while reducing time and effort for tutor participants and trainers.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121216455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stanislav Pozdniakov, Roberto Martínez-Maldonado, Yi-Shan Tsai, Vanessa Echeverría, Namrata Srivastava, D. Gašević
There is a proliferation of learning analytics (LA) dashboards aimed at supporting teachers. Yet, teachers still find it challenging to make sense of LA dashboards, thereby making informed decisions. Two main strategies to address this are emerging: i) upskilling teachers’ data literacy; ii) improving the explanatory design features of current dashboards (e.g., adding visual cues or text) to minimise the skills required by teachers to effectively use dashboards. While each approach has its own trade-offs, no previous work has explored the interplay between the dashboard design and such "data skills". In this paper, we explore how teachers with varying visualisation literacy (VL) skills use LA dashboards enhanced with (explanatory) data storytelling elements. We conducted a quasi-experimental study with 23 teachers of varied VL inspecting two versions of an authentic multichannel dashboard enhanced with data storytelling elements. We used an eye-tracking device while teachers inspected the students’ data captured from Zoom and Google Docs, followed by interviews. Results suggest that high VL teachers adopted complex exploratory strategies and were more sensitive to subtle inconsistencies in the design; while low VL teachers benefited the most from more explicit data storytelling guidance such as accompanying complex graphs with narrative and semantic colour encoding.
{"title":"How Do Teachers Use Dashboards Enhanced with Data Storytelling Elements According to their Data Visualisation Literacy Skills?","authors":"Stanislav Pozdniakov, Roberto Martínez-Maldonado, Yi-Shan Tsai, Vanessa Echeverría, Namrata Srivastava, D. Gašević","doi":"10.1145/3576050.3576063","DOIUrl":"https://doi.org/10.1145/3576050.3576063","url":null,"abstract":"There is a proliferation of learning analytics (LA) dashboards aimed at supporting teachers. Yet, teachers still find it challenging to make sense of LA dashboards, thereby making informed decisions. Two main strategies to address this are emerging: i) upskilling teachers’ data literacy; ii) improving the explanatory design features of current dashboards (e.g., adding visual cues or text) to minimise the skills required by teachers to effectively use dashboards. While each approach has its own trade-offs, no previous work has explored the interplay between the dashboard design and such \"data skills\". In this paper, we explore how teachers with varying visualisation literacy (VL) skills use LA dashboards enhanced with (explanatory) data storytelling elements. We conducted a quasi-experimental study with 23 teachers of varied VL inspecting two versions of an authentic multichannel dashboard enhanced with data storytelling elements. We used an eye-tracking device while teachers inspected the students’ data captured from Zoom and Google Docs, followed by interviews. Results suggest that high VL teachers adopted complex exploratory strategies and were more sensitive to subtle inconsistencies in the design; while low VL teachers benefited the most from more explicit data storytelling guidance such as accompanying complex graphs with narrative and semantic colour encoding.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128170602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Messages exchanged between participants in online discussion forums often contain personal names and other details that need to be redacted before the data is used for research purposes in learning analytics. However, removing the names entirely makes it harder to track the exchange of ideas between individuals within a message thread and across threads, and thereby reduces the value of this type of conversational data. In contrast, the consistent use of pseudonyms allows contributions from individuals to be tracked across messages, while also hiding the real identities of the contributors. Several factors can make it difficult to identify all instances of personal names that refer to the same individual, including spelling errors and the use of shortened forms. We developed a semi-automated approach for replacing personal names with consistent pseudonyms. We evaluated our approach on a data set of over 1,700 messages exchanged during a distance-learning course, and compared it to a general-purpose pseudonymisation tool that used deep neural networks to identify names to be redacted. We found that our tailored approach out-performed the general-purpose tool in both precision and recall, correctly identifying all but 31 substitutions out of 2,888.
{"title":"Names, Nicknames, and Spelling Errors: Protecting Participant Identity in Learning Analytics of Online Discussions","authors":"Elaine Farrow, Johanna D. Moore, D. Gašević","doi":"10.1145/3576050.3576070","DOIUrl":"https://doi.org/10.1145/3576050.3576070","url":null,"abstract":"Messages exchanged between participants in online discussion forums often contain personal names and other details that need to be redacted before the data is used for research purposes in learning analytics. However, removing the names entirely makes it harder to track the exchange of ideas between individuals within a message thread and across threads, and thereby reduces the value of this type of conversational data. In contrast, the consistent use of pseudonyms allows contributions from individuals to be tracked across messages, while also hiding the real identities of the contributors. Several factors can make it difficult to identify all instances of personal names that refer to the same individual, including spelling errors and the use of shortened forms. We developed a semi-automated approach for replacing personal names with consistent pseudonyms. We evaluated our approach on a data set of over 1,700 messages exchanged during a distance-learning course, and compared it to a general-purpose pseudonymisation tool that used deep neural networks to identify names to be redacted. We found that our tailored approach out-performed the general-purpose tool in both precision and recall, correctly identifying all but 31 substitutions out of 2,888.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121572312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Luiz Barboza, Rafael Mello, M. Modell, E. Teixeira
Interest in data science has been growing across industries - both STEM and non-STEM. Non-STEM students often have difficulties with programming and data analysis tools. These entry barriers can be minimized, and these concepts can be easily absorbed when using visual tools. Thus, for this specific audience, the use of visual tools has been essential for teaching data science. Several of these tools are available, but they all have limitations. This work presents Blockly-DS: a new tool capable of assisting in teaching data science to a non-STEM audience. The Blockly-DS tool is being tested in two Brazilian higher education institutions, one, IBMEC, a business undergraduate university, and the other, FIAP, a STEM school that offers an MBA as well as corporate and undergraduate courses. The preliminary results presented in this article refers to a validation with two groups of training sessions for junior financial analysts of a major Brazilian bank in partnership with FIAP.
{"title":"Blockly-DS: Blocks Programming for Data Science with Visual, Statistical, Descriptive and Predictive Analysis","authors":"Luiz Barboza, Rafael Mello, M. Modell, E. Teixeira","doi":"10.1145/3576050.3576097","DOIUrl":"https://doi.org/10.1145/3576050.3576097","url":null,"abstract":"Interest in data science has been growing across industries - both STEM and non-STEM. Non-STEM students often have difficulties with programming and data analysis tools. These entry barriers can be minimized, and these concepts can be easily absorbed when using visual tools. Thus, for this specific audience, the use of visual tools has been essential for teaching data science. Several of these tools are available, but they all have limitations. This work presents Blockly-DS: a new tool capable of assisting in teaching data science to a non-STEM audience. The Blockly-DS tool is being tested in two Brazilian higher education institutions, one, IBMEC, a business undergraduate university, and the other, FIAP, a STEM school that offers an MBA as well as corporate and undergraduate courses. The preliminary results presented in this article refers to a validation with two groups of training sessions for junior financial analysts of a major Brazilian bank in partnership with FIAP.","PeriodicalId":394433,"journal":{"name":"LAK23: 13th International Learning Analytics and Knowledge Conference","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114652912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：