Digital competence (DC) is a broad set of skills, attitudes, and knowledge for confident, critical and responsible use of digital technologies in every aspect of life. DC proves essential in the contemporary digital landscape, yet its diffusion is hindered by biases, misunderstandings, and limited awareness. Teaching Informatics in the educational curriculum is increasingly supported by the institutions but faces serious challenges, such as teacher upskilling and support. In response, grassroots movements promoting computing literacy in an informal setting have grown, including EU Code Week, whose vision is to develop computing skills while promoting diversity and raising awareness of the importance of digital skills. This study extensively analyses EU Code Week editions spanning 2014 to 2021 across European Union member states, pursuing three primary objectives: firstly, to evaluate the teacher engagement in the campaign in terms of penetration, retention, and spatial distribution; secondly, to characterise the multifaceted audience and themes embraced by these initiatives; and lastly, to investigate the influence of socio-economic factors on engagement. The investigation uncovers the underlying mechanisms fostering Code Week’s engagement, providing insights to campaign organisers for strategic planning and resource allocation in future editions. Moreover, the analysis reveals that the most engaged areas are characterised by lower income, as well as lower digital literacy, restricted access to technology, and a less established computer education, suggesting that Code Week thrives precisely where its impact is most needed.
{"title":"Investigating Participation Mechanisms in EU Code Week","authors":"Christel Sirocchi, Annika Ostergren Pofantis, Alessandro Bogliolo","doi":"10.1145/3632531","DOIUrl":"https://doi.org/10.1145/3632531","url":null,"abstract":"Digital competence (DC) is a broad set of skills, attitudes, and knowledge for confident, critical and responsible use of digital technologies in every aspect of life. DC proves essential in the contemporary digital landscape, yet its diffusion is hindered by biases, misunderstandings, and limited awareness. Teaching Informatics in the educational curriculum is increasingly supported by the institutions but faces serious challenges, such as teacher upskilling and support. In response, grassroots movements promoting computing literacy in an informal setting have grown, including EU Code Week, whose vision is to develop computing skills while promoting diversity and raising awareness of the importance of digital skills. This study extensively analyses EU Code Week editions spanning 2014 to 2021 across European Union member states, pursuing three primary objectives: firstly, to evaluate the teacher engagement in the campaign in terms of penetration, retention, and spatial distribution; secondly, to characterise the multifaceted audience and themes embraced by these initiatives; and lastly, to investigate the influence of socio-economic factors on engagement. The investigation uncovers the underlying mechanisms fostering Code Week’s engagement, providing insights to campaign organisers for strategic planning and resource allocation in future editions. Moreover, the analysis reveals that the most engaged areas are characterised by lower income, as well as lower digital literacy, restricted access to technology, and a less established computer education, suggesting that Code Week thrives precisely where its impact is most needed.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"55 15","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136347894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Those working towards equitable K12 computing education in the United States have always had their work cut out for them: understanding how best to teach, developing robust curricula, designing novel tools, building teacher capacity, and supporting systemic change in schools to bring equitable computing education to young people. Collectively, these areas represent an ambitious and complex set of problems to solve. But current changes to the educational landscape in the United States—where teaching basic ideas about how to critically engage with the world has become deeply politicized, and where forces working towards privatization of education are on the rise—require us to broaden the scope of our work. Our efforts must shift to not just include the aforementioned problems but also include ones that are more explicitly political: engaging in solidarity with the larger project of public education, and supporting teachers to have the freedom to teach what students will need to contribute to our society.
{"title":"Their Fight is Our Fight: Why Computing Education Advocates Must Be in Solidarity with Public Schools","authors":"Rafi Santo, Aman Yadav, David Phelps","doi":"10.1145/3632296","DOIUrl":"https://doi.org/10.1145/3632296","url":null,"abstract":"Those working towards equitable K12 computing education in the United States have always had their work cut out for them: understanding how best to teach, developing robust curricula, designing novel tools, building teacher capacity, and supporting systemic change in schools to bring equitable computing education to young people. Collectively, these areas represent an ambitious and complex set of problems to solve. But current changes to the educational landscape in the United States—where teaching basic ideas about how to critically engage with the world has become deeply politicized, and where forces working towards privatization of education are on the rise—require us to broaden the scope of our work. Our efforts must shift to not just include the aforementioned problems but also include ones that are more explicitly political: engaging in solidarity with the larger project of public education, and supporting teachers to have the freedom to teach what students will need to contribute to our society.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"112 45","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135137613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Heather Killen, Merijke Coenraad, Virginia Byrne, Lautaro Cabrera, Kelly Mills, Diane Jass Ketelhut, Jandelyn D. Plane
Computational thinking (CT) is playing an increasingly relevant role within disciplinary teaching in elementary school, particularly in science. However, many teachers are unfamiliar with CT, either because their education occurred before the popularization of CT or because CT instruction was not included in their pre-service coursework. For these teachers, CT professional development (PD) becomes a primary mechanism to close their CT knowledge gap. While CT PD has demonstrated success at increasing teacher's CT understanding, researchers have reported varied outcomes in supporting teachers to write CT-integrated lesson plans. To explore how we might support teachers to integrate CT into elementary science, we employed design-based research (DBR) in a dual-track design of in-class CT instruction for pre-service undergraduates within an elementary science methods class paired with a collaborative, multi-month PD opportunity for pre- and in-service teachers. In this article, we reflect on our 5-year period of DBR and present our design insights and implications for CT instruction and curriculum design from each iteration. Our findings on best practices will inform both teacher educators and PD providers within CT education. Our work will also be of interest to researchers considering DBR for technology-based educational projects.
{"title":"Teacher Education to Integrate Computational Thinking into Elementary Science: A Design-Based Research Study","authors":"Heather Killen, Merijke Coenraad, Virginia Byrne, Lautaro Cabrera, Kelly Mills, Diane Jass Ketelhut, Jandelyn D. Plane","doi":"10.1145/3618115","DOIUrl":"https://doi.org/10.1145/3618115","url":null,"abstract":"Computational thinking (CT) is playing an increasingly relevant role within disciplinary teaching in elementary school, particularly in science. However, many teachers are unfamiliar with CT, either because their education occurred before the popularization of CT or because CT instruction was not included in their pre-service coursework. For these teachers, CT professional development (PD) becomes a primary mechanism to close their CT knowledge gap. While CT PD has demonstrated success at increasing teacher's CT understanding, researchers have reported varied outcomes in supporting teachers to write CT-integrated lesson plans. To explore how we might support teachers to integrate CT into elementary science, we employed design-based research (DBR) in a dual-track design of in-class CT instruction for pre-service undergraduates within an elementary science methods class paired with a collaborative, multi-month PD opportunity for pre- and in-service teachers. In this article, we reflect on our 5-year period of DBR and present our design insights and implications for CT instruction and curriculum design from each iteration. Our findings on best practices will inform both teacher educators and PD providers within CT education. Our work will also be of interest to researchers considering DBR for technology-based educational projects.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":" 8","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland, Ja'Afaru Musa, Negar Mohammadhassan, Raul Vincent Lumapas
Objectives . Communication skills are crucial for effective software development teams, but those skills are difficult to teach. The goal of our project is to evaluate the effectiveness of teaching face-to-face communication skills using AVW-Space, a platform for video-based learning which provides personalized nudges to support student's engagement during video watching. Participants . The participants in our study are second-year software engineering students. The study was conducted over three years, with students enrolled in a semester-long project course. Study Method . We performed a quasi-experimental study over three years to teach face-to-face communication using AVW-Space, a video-based learning platform. We present the instance of AVW-Space we developed to teach face-to-face communication. Participants watched and commented on ten videos, and later commented on the recording of their own team meeting. In 2020, the participants (n = 50) did not receive nudges, and we use the data collected that year as control. In 2021 (n = 49) and 2022 (n = 48), nudges were provided adaptively, to encourage students to write more and higher-quality comments. Findings . The findings from the study show the effectiveness of nudges. We found significant differences in engagement when nudges were provided. Furthermore, there is a causal effect of nudges on the interaction time, the total number of comments written and the number of high-quality comments, as well as on learning. Finally, participants exposed to nudges reported higher perceived learning. Conclusions . Our research shows the effect of nudges on student engagement and learning while using the instance of AVW-Space for teaching face-to-face communication skills. Future work will explore other soft skills, as well as providing explanations for the decisions made by AVW-Space.
{"title":"Effectiveness of Video-based Training for Face-to-Face Communication Skills of Software Engineers: Evidence from a Three-Year Study","authors":"Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland, Ja'Afaru Musa, Negar Mohammadhassan, Raul Vincent Lumapas","doi":"10.1145/3631532","DOIUrl":"https://doi.org/10.1145/3631532","url":null,"abstract":"Objectives . Communication skills are crucial for effective software development teams, but those skills are difficult to teach. The goal of our project is to evaluate the effectiveness of teaching face-to-face communication skills using AVW-Space, a platform for video-based learning which provides personalized nudges to support student's engagement during video watching. Participants . The participants in our study are second-year software engineering students. The study was conducted over three years, with students enrolled in a semester-long project course. Study Method . We performed a quasi-experimental study over three years to teach face-to-face communication using AVW-Space, a video-based learning platform. We present the instance of AVW-Space we developed to teach face-to-face communication. Participants watched and commented on ten videos, and later commented on the recording of their own team meeting. In 2020, the participants (n = 50) did not receive nudges, and we use the data collected that year as control. In 2021 (n = 49) and 2022 (n = 48), nudges were provided adaptively, to encourage students to write more and higher-quality comments. Findings . The findings from the study show the effectiveness of nudges. We found significant differences in engagement when nudges were provided. Furthermore, there is a causal effect of nudges on the interaction time, the total number of comments written and the number of high-quality comments, as well as on learning. Finally, participants exposed to nudges reported higher perceived learning. Conclusions . Our research shows the effect of nudges on student engagement and learning while using the instance of AVW-Space for teaching face-to-face communication skills. Future work will explore other soft skills, as well as providing explanations for the decisions made by AVW-Space.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"41 15","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135818708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Arif Rachmatullah, Jessica Vandenberg, Sein Shin, Eric Wiebe
Objectives The lack of gender diversity in the computer science (CS) field and workforce is a well-documented challenge that many, but not all, countries face. Such a challenge may be tied to sociocultural and psychological issues that have impacted K–12 CS education, eventually creating a gender gap in CS attitudes and interests. The current study compared American, Korean, and Indonesian middle and high school students’ CS attitudes in two different studies (Study 1 and Study 2). Concurrently, this study also examined whether the items in the CS attitudes scale exhibit country and gender measurement biases. Participants In Study 1, we gathered data on CS attitudes from 886 middle school students (aged 11–14 years old) in the US, Korea, and Indonesia. In Study 2, we collected data on CS attitudes from Indonesian ( n = 427) and Korean ( n = 682) high school students. Study Methods The participating students took the same (translated) previously validated CS attitudes scale. We ran a unidimensional IRT and differential item functioning (DIF). We also ran a two-way analysis of variance (ANOVA) and the Kruskal-Wallis H test. Findings In Study 1, despite the valid instrument, we found it inappropriate as is for international comparison studies because students from different countries interpreted some items differently. After removing these biased items, we then compared gender-based differences in CS attitudes across countries. We found no significant differences between males and females in the Indonesian middle school data, whereas male students had significantly higher CS attitudes than female students in both American and Korean student data. In Study 2, we found the same pattern in gender differences in CS attitudes scores as in Study 1. The results revealed no significant difference in CS attitudes based on gender in the Indonesian high school data; however, Korean male students had significantly higher CS attitudes than female students. Conclusions These findings underscore the importance of a country’s sociocultural context in influencing gap and diversity in secondary school students’ CS attitudes.
{"title":"Cross-Country Variation in (Binary) Gender Differences in Secondary School Students’ CS Attitudes: Re-Validating and Generalizing a CS Attitudes Scale","authors":"Arif Rachmatullah, Jessica Vandenberg, Sein Shin, Eric Wiebe","doi":"10.1145/3630014","DOIUrl":"https://doi.org/10.1145/3630014","url":null,"abstract":"Objectives The lack of gender diversity in the computer science (CS) field and workforce is a well-documented challenge that many, but not all, countries face. Such a challenge may be tied to sociocultural and psychological issues that have impacted K–12 CS education, eventually creating a gender gap in CS attitudes and interests. The current study compared American, Korean, and Indonesian middle and high school students’ CS attitudes in two different studies (Study 1 and Study 2). Concurrently, this study also examined whether the items in the CS attitudes scale exhibit country and gender measurement biases. Participants In Study 1, we gathered data on CS attitudes from 886 middle school students (aged 11–14 years old) in the US, Korea, and Indonesia. In Study 2, we collected data on CS attitudes from Indonesian ( n = 427) and Korean ( n = 682) high school students. Study Methods The participating students took the same (translated) previously validated CS attitudes scale. We ran a unidimensional IRT and differential item functioning (DIF). We also ran a two-way analysis of variance (ANOVA) and the Kruskal-Wallis H test. Findings In Study 1, despite the valid instrument, we found it inappropriate as is for international comparison studies because students from different countries interpreted some items differently. After removing these biased items, we then compared gender-based differences in CS attitudes across countries. We found no significant differences between males and females in the Indonesian middle school data, whereas male students had significantly higher CS attitudes than female students in both American and Korean student data. In Study 2, we found the same pattern in gender differences in CS attitudes scores as in Study 1. The results revealed no significant difference in CS attitudes based on gender in the Indonesian high school data; however, Korean male students had significantly higher CS attitudes than female students. Conclusions These findings underscore the importance of a country’s sociocultural context in influencing gap and diversity in secondary school students’ CS attitudes.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"35 11","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135366383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background and Context. Students’ programming projects are often assessed on the basis of their tests as well as their implementations, most commonly using test adequacy criteria like branch coverage, or, in some cases, mutation analysis. As a result, students are implicitly encouraged to use these tools during their development process (i.e., so they have awareness of the strength of their own test suites). Objectives. Little is known about how students choose test cases for their software while being guided by these feedback mechanisms. We aim to explore the interaction between students and commonly used testing feedback mechanisms (in this case, branch coverage and mutation-based feedback). Method. We use grounded theory to explore this interaction. We conducted 12 think-aloud interviews with students as they were asked to complete a series of software testing tasks, each of which involved a different feedback mechanism. Interviews were recorded and transcripts were analyzed, and we present the overarching themes that emerged from our analysis. Findings. Our findings are organized into a process model describing how students completed software testing tasks while being guided by a test adequacy criterion. Program comprehension strategies were commonly employed to reason about feedback and devise test cases. Mutation-based feedback tended to be cognitively overwhelming for students, and they resorted to weaker heuristics in order to address this feedback. Implications. In the presence of testing feedback, students did not appear to consider problem coverage as a testing goal so much as program coverage . While test adequacy criteria can be useful for assessment of software tests, we must consider whether they represent good goals for testing, and if our current methods of practice and assessment are encouraging poor testing habits.
{"title":"A Model of How Students Engineer Test Cases With Feedback","authors":"Austin M. Shin, Ayaan M. Kazerouni","doi":"10.1145/3628604","DOIUrl":"https://doi.org/10.1145/3628604","url":null,"abstract":"Background and Context. Students’ programming projects are often assessed on the basis of their tests as well as their implementations, most commonly using test adequacy criteria like branch coverage, or, in some cases, mutation analysis. As a result, students are implicitly encouraged to use these tools during their development process (i.e., so they have awareness of the strength of their own test suites). Objectives. Little is known about how students choose test cases for their software while being guided by these feedback mechanisms. We aim to explore the interaction between students and commonly used testing feedback mechanisms (in this case, branch coverage and mutation-based feedback). Method. We use grounded theory to explore this interaction. We conducted 12 think-aloud interviews with students as they were asked to complete a series of software testing tasks, each of which involved a different feedback mechanism. Interviews were recorded and transcripts were analyzed, and we present the overarching themes that emerged from our analysis. Findings. Our findings are organized into a process model describing how students completed software testing tasks while being guided by a test adequacy criterion. Program comprehension strategies were commonly employed to reason about feedback and devise test cases. Mutation-based feedback tended to be cognitively overwhelming for students, and they resorted to weaker heuristics in order to address this feedback. Implications. In the presence of testing feedback, students did not appear to consider problem coverage as a testing goal so much as program coverage . While test adequacy criteria can be useful for assessment of software tests, we must consider whether they represent good goals for testing, and if our current methods of practice and assessment are encouraging poor testing habits.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"31 7","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135567625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In learning to program and understanding how a programming language controls a computer, learners develop both insights and misconceptions whilst their mental models are gradually refined. It is important that the learner is able to distinguish the different elements and roles of a computer (e.g. compiler, interpreter, memory, etc.), which novice programmers may find difficult to comprehend. Forming accurate mental models is one of the potential sources of difficulty inextricably linked to mastering computing concepts and processes, and for learning computer programming. It is common to use some form of representation (e.g. an abstract machine or a computational agent) to support technical or pedagogic explanations. The Notional Machine (NM) is a pedagogical device that entails one or more computational concepts, originally described as an idealised computer operating with the constructs of a particular programming language. It can be used to support specific or general learning goals and will typically have some concrete representation that can be referred to. Computational Thinking (CT), which is defined as a way of thinking that is used for [computational] problem solving, is often presented as using a Computational Agent (CA) to carry out information processing presented by a solution. In CT, where the typical goal is to produce an algorithm or a computer program, the CA seemingly serves a purpose very similar to an NM. Although it changes through the different stages of development (of the learner and of the curriculum), the roles of CAs and NMs can be seen as versatile tools that connect a learner’s mental model with the conceptual model of a program. In this article we look at this relationship between CAs and NMs, and indicate how they would look at different stages of learning. We traverse the range of definitions and usages of these concepts, and articulate models that clarify how these are viewed in the literature. This includes exploring the nature of machines and agents, and how historical views of these relate to modern pedagogy for computation. We argue that the CA can be seen as an abstract, simplified variant of an NM that provides a useful perspective to the learner to support them to form robust mental models of NMs more efficiently and effectively. We propose that teaching programming should make use of the idea of a CA at different stages of learning, as a link that connects a learner’s mental model to a full NM.
{"title":"Computational Thinking and Notional Machines: The Missing Link","authors":"Bhagya Munasinghe, Tim Bell, Anthony Robins","doi":"10.1145/3627829","DOIUrl":"https://doi.org/10.1145/3627829","url":null,"abstract":"In learning to program and understanding how a programming language controls a computer, learners develop both insights and misconceptions whilst their mental models are gradually refined. It is important that the learner is able to distinguish the different elements and roles of a computer (e.g. compiler, interpreter, memory, etc.), which novice programmers may find difficult to comprehend. Forming accurate mental models is one of the potential sources of difficulty inextricably linked to mastering computing concepts and processes, and for learning computer programming. It is common to use some form of representation (e.g. an abstract machine or a computational agent) to support technical or pedagogic explanations. The Notional Machine (NM) is a pedagogical device that entails one or more computational concepts, originally described as an idealised computer operating with the constructs of a particular programming language. It can be used to support specific or general learning goals and will typically have some concrete representation that can be referred to. Computational Thinking (CT), which is defined as a way of thinking that is used for [computational] problem solving, is often presented as using a Computational Agent (CA) to carry out information processing presented by a solution. In CT, where the typical goal is to produce an algorithm or a computer program, the CA seemingly serves a purpose very similar to an NM. Although it changes through the different stages of development (of the learner and of the curriculum), the roles of CAs and NMs can be seen as versatile tools that connect a learner’s mental model with the conceptual model of a program. In this article we look at this relationship between CAs and NMs, and indicate how they would look at different stages of learning. We traverse the range of definitions and usages of these concepts, and articulate models that clarify how these are viewed in the literature. This includes exploring the nature of machines and agents, and how historical views of these relate to modern pedagogy for computation. We argue that the CA can be seen as an abstract, simplified variant of an NM that provides a useful perspective to the learner to support them to form robust mental models of NMs more efficiently and effectively. We propose that teaching programming should make use of the idea of a CA at different stages of learning, as a link that connects a learner’s mental model to a full NM.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"240 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135825496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the immense interest in ChatGPT worldwide, education has seen a mix of both excitement and skepticism. To properly evaluate its impact on education, it is crucial to understand how far it can help students without prior knowledge answer assessment questions. This study aims to address this question as well as the impact of the question type. We conducted multiple experiments with computer engineering students (experiment group: n = 41 to 56), who were asked to use ChatGPT to answer previous test questions before learning about the related topics. Their scores were then compared with the scores of previous-term students who answered the same questions in a quiz or exam setting (control group: n = 24 to 61). The results showed a wide range of effect sizes, from -2.55 to 1.23, depending on the question type and content. The experiment group performed best answering code analysis and conceptual questions but struggled with code completion and questions that involved images. On the other hand, the performance in code generation tasks was inconsistent. Overall, the ChatGPT group’s answers lagged slightly behind the control group’s answers with an effect size of − 0.16. We conclude that ChatGPT, at least in the field of this study, is not yet ready to rely on by students who don’t have sufficient background to evaluate generated answers. We suggest that educators try using ChatGPT and educate students on effective questioning techniques and how to assess the generated responses. This study provides insights into the capabilities and limitations of ChatGPT in education and informs future research and development.
{"title":"Can students without prior knowledge use ChatGPT to answer test questions? An empirical study","authors":"Abdulhadi Shoufan","doi":"10.1145/3628162","DOIUrl":"https://doi.org/10.1145/3628162","url":null,"abstract":"With the immense interest in ChatGPT worldwide, education has seen a mix of both excitement and skepticism. To properly evaluate its impact on education, it is crucial to understand how far it can help students without prior knowledge answer assessment questions. This study aims to address this question as well as the impact of the question type. We conducted multiple experiments with computer engineering students (experiment group: n = 41 to 56), who were asked to use ChatGPT to answer previous test questions before learning about the related topics. Their scores were then compared with the scores of previous-term students who answered the same questions in a quiz or exam setting (control group: n = 24 to 61). The results showed a wide range of effect sizes, from -2.55 to 1.23, depending on the question type and content. The experiment group performed best answering code analysis and conceptual questions but struggled with code completion and questions that involved images. On the other hand, the performance in code generation tasks was inconsistent. Overall, the ChatGPT group’s answers lagged slightly behind the control group’s answers with an effect size of − 0.16. We conclude that ChatGPT, at least in the field of this study, is not yet ready to rely on by students who don’t have sufficient background to evaluate generated answers. We suggest that educators try using ChatGPT and educate students on effective questioning techniques and how to assess the generated responses. This study provides insights into the capabilities and limitations of ChatGPT in education and informs future research and development.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"162 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135825264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maral Kargarmoakhar, Monique Ross, Zahra Hazari, Stephen Secules, Mark Allen Weiss, Michael Georgiopoulos, Kenneth Christensen, Tiana Solis
While computing programs in the U.S. are experiencing growth in enrollment trends, they are still grappling with matters related to retention and persistence of computing undergraduates. One construct identified by scholars as having an impact on persistence in computing is computing identity which is shaped by constructs such as recognition, performance/competence beliefs, sense of belonging, and interest. Likewise, participation in what scholars call communities of practice (CoP) can aid in the development of their computing identity. To help foster computing identity development, an initiative was designed at three large public universities named Flit-Path (Florida IT Pathways to Success). Flit-Path was established using the principles inherent to communities of practice with the goal of recruiting and retaining computing students. The Flit-Path program leveraged curricular and co-curricular support to engage academically talented students with financial need in computing disciplines (e.g., computer engineering, computer science, and information technology) as well as provided financial assistance via scholarships. The guiding research question for this study was - What is the impact of a computing community of practice (the Flit-Path program) on students’ computing identity, specifically the constructs of recognition, performance/competence, sense of belonging, and interest? In order to address this question, a validated survey instrument was used to compare 64 computing students who enrolled in the Flit-Path program with students from the same universities with matched years in college, computing GPA, race/ ethnicity, gender, home/ environment support, and work hours outside the home. For comparing the two groups, the research team used multivariate matching methods in R. The results of the study revealed students in the Flit-Path program demonstrated substantially higher computing identities. Students who participated in the Flit-Path program experienced higher recognition, performance/competence, and sense of belonging in the computing field than their non-Flit-Path counterparts. There was also a borderline positive effect for interest. Together, the results indicate that well-designed CoP interventions in computing programs can have a significant effect on students’ identification with computing and ultimately their persistence.
{"title":"The Impact of a Community of Practice Scholarship Program on Students’ Computing Identity","authors":"Maral Kargarmoakhar, Monique Ross, Zahra Hazari, Stephen Secules, Mark Allen Weiss, Michael Georgiopoulos, Kenneth Christensen, Tiana Solis","doi":"10.1145/3623615","DOIUrl":"https://doi.org/10.1145/3623615","url":null,"abstract":"While computing programs in the U.S. are experiencing growth in enrollment trends, they are still grappling with matters related to retention and persistence of computing undergraduates. One construct identified by scholars as having an impact on persistence in computing is computing identity which is shaped by constructs such as recognition, performance/competence beliefs, sense of belonging, and interest. Likewise, participation in what scholars call communities of practice (CoP) can aid in the development of their computing identity. To help foster computing identity development, an initiative was designed at three large public universities named Flit-Path (Florida IT Pathways to Success). Flit-Path was established using the principles inherent to communities of practice with the goal of recruiting and retaining computing students. The Flit-Path program leveraged curricular and co-curricular support to engage academically talented students with financial need in computing disciplines (e.g., computer engineering, computer science, and information technology) as well as provided financial assistance via scholarships. The guiding research question for this study was - What is the impact of a computing community of practice (the Flit-Path program) on students’ computing identity, specifically the constructs of recognition, performance/competence, sense of belonging, and interest? In order to address this question, a validated survey instrument was used to compare 64 computing students who enrolled in the Flit-Path program with students from the same universities with matched years in college, computing GPA, race/ ethnicity, gender, home/ environment support, and work hours outside the home. For comparing the two groups, the research team used multivariate matching methods in R. The results of the study revealed students in the Flit-Path program demonstrated substantially higher computing identities. Students who participated in the Flit-Path program experienced higher recognition, performance/competence, and sense of belonging in the computing field than their non-Flit-Path counterparts. There was also a borderline positive effect for interest. Together, the results indicate that well-designed CoP interventions in computing programs can have a significant effect on students’ identification with computing and ultimately their persistence.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135397582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alvine B. Belle, Callum Sutherland, O. Adesina, Sègla Kpodjedo, Nathanael Ojong, Lisa Cole
Background: People who are racialized, gendered, or otherwise minoritized are underrepresented in computing professions in North America. This is reflected in undergraduate computer science (CS) programs, in which students from marginalized backgrounds continue to experience inequities that do not typically affect White cis-men. This is especially true for Black students in general, and Black women in particular, whose experience of systemic, anti-Black racism compromises their ability to persist and thrive in CS education contexts. Objectives: This systematic mapping study endeavours to (1) determine the quantity of existing non-deficit-based studies concerned with the persistence of Black students in undergraduate CS; (2) summarize the findings and recommendations in those studies; and (3) identify areas in which additional studies may be required. We aim to accomplish these objectives by way of two research questions: (RQ1) What factors are associated with Black students’ persistence in undergraduate CS programs?; and (RQ2) What recommendations have been made to further bolster Black students’ persistence in undergraduate CS education programs? Methods: This systematic mapping study was conducted in accordance with PRISMA 2020 and SEGRESS guidelines. Studies were identified by conducting keyword searches in seven databases. Inclusion and exclusion criteria were designed to capture studies illuminating persistence factors for Black students in undergraduate CS programs. To ensure the completeness of our search results, we engaged in snowballing and an expert-based search to identify additional studies of interest. Finally, data were collected from each study to address the research questions outlined above. Results: Using the methods outlined above, we identified 16 empirical studies, including qualitative, quantitative, and mixed-methods studies informed by a range of theoretical frameworks. Based on data collected from the primary studies in our sample, we identified 13 persistence factors across four categories: (I) social capital, networking, & support; (II) career & professional development; (III) pedagogical & programmatic interventions; and (IV) exposure & access. This data-collection process also yielded 26 recommendations across six stakeholder groups: (i) researchers; (ii) colleges and universities; (iii) the computing industry; (iv) K-12 systems and schools; (v) governments; and (vi) parents. Conclusion: This systematic mapping study resulted in the identification of numerous persistence factors for Black students in CS. Crucially, however, these persistence factors allow Black students to persist, but not thrive, in CS. Accordingly, we contend that more needs to be done to address the systemic inequities faced by Black people in general, and Black women in particular, in computing programs and professions. As evidenced by the relatively small number of primary studies captured by this systematic mapping study, there exists an urgent
{"title":"Bolstering the Persistence of Black Students in Undergraduate Computer Science Programs: A Systematic Mapping Study","authors":"Alvine B. Belle, Callum Sutherland, O. Adesina, Sègla Kpodjedo, Nathanael Ojong, Lisa Cole","doi":"10.1145/3617896","DOIUrl":"https://doi.org/10.1145/3617896","url":null,"abstract":"Background: People who are racialized, gendered, or otherwise minoritized are underrepresented in computing professions in North America. This is reflected in undergraduate computer science (CS) programs, in which students from marginalized backgrounds continue to experience inequities that do not typically affect White cis-men. This is especially true for Black students in general, and Black women in particular, whose experience of systemic, anti-Black racism compromises their ability to persist and thrive in CS education contexts. Objectives: This systematic mapping study endeavours to (1) determine the quantity of existing non-deficit-based studies concerned with the persistence of Black students in undergraduate CS; (2) summarize the findings and recommendations in those studies; and (3) identify areas in which additional studies may be required. We aim to accomplish these objectives by way of two research questions: (RQ1) What factors are associated with Black students’ persistence in undergraduate CS programs?; and (RQ2) What recommendations have been made to further bolster Black students’ persistence in undergraduate CS education programs? Methods: This systematic mapping study was conducted in accordance with PRISMA 2020 and SEGRESS guidelines. Studies were identified by conducting keyword searches in seven databases. Inclusion and exclusion criteria were designed to capture studies illuminating persistence factors for Black students in undergraduate CS programs. To ensure the completeness of our search results, we engaged in snowballing and an expert-based search to identify additional studies of interest. Finally, data were collected from each study to address the research questions outlined above. Results: Using the methods outlined above, we identified 16 empirical studies, including qualitative, quantitative, and mixed-methods studies informed by a range of theoretical frameworks. Based on data collected from the primary studies in our sample, we identified 13 persistence factors across four categories: (I) social capital, networking, & support; (II) career & professional development; (III) pedagogical & programmatic interventions; and (IV) exposure & access. This data-collection process also yielded 26 recommendations across six stakeholder groups: (i) researchers; (ii) colleges and universities; (iii) the computing industry; (iv) K-12 systems and schools; (v) governments; and (vi) parents. Conclusion: This systematic mapping study resulted in the identification of numerous persistence factors for Black students in CS. Crucially, however, these persistence factors allow Black students to persist, but not thrive, in CS. Accordingly, we contend that more needs to be done to address the systemic inequities faced by Black people in general, and Black women in particular, in computing programs and professions. As evidenced by the relatively small number of primary studies captured by this systematic mapping study, there exists an urgent","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"1 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41570662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：