Learners of database courses usually encounter difficulties in building entity-relationship (ER) models and relational models for database problems. These difficulties may arise because of semantic gaps between the stages of database design. To investigate this issue, we employed semantic network theory—particularly the concept of semantic distance—as an analytical framework.
Hypothesis
We hypothesized that (i) the transition from business requirements to ER models involves a greater semantic distance than the ER-to-relational conversion, and (ii) this greater distance results in imprecise semantic elaboration, leading to lower learning performance.
Method
An experiment was designed in which participants, drawn from a database course, completed two sequential translation tasks: (a) business requirements to ER models, and (b) ER models to relational models. Participants' performances were assessed through problem-solving effectiveness and efficiency measures.
Results
The problem-solving effectiveness and efficiency of task (b) surpassed those of task (a), suggesting that task (a) entailed greater semantic distance and less precise elaboration of semantic relationships than task (b).
Conclusion
Our findings suggest that semantic distance and semantic elaboration are critical factors in database design education, aligning with semantic network theory. Instruction should reduce semantic distance and pay more attention to the teaching of the business requirement to ER model transition. This may prompt scholars to develop more effective teaching methods for database design learning from the perspective of semantic distance.
{"title":"Difficulties in understanding the transition between database design stages: An experiment from a semantic distance perspective","authors":"Shin-Shing Shin, Yu-Shan Lin, Yi-Cheng Chen, Wei-Ru Chiou","doi":"10.1002/jee.70038","DOIUrl":"https://doi.org/10.1002/jee.70038","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Learners of database courses usually encounter difficulties in building entity-relationship (ER) models and relational models for database problems. These difficulties may arise because of semantic gaps between the stages of database design. To investigate this issue, we employed semantic network theory—particularly the concept of semantic distance—as an analytical framework.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Hypothesis</h3>\u0000 \u0000 <p>We hypothesized that (i) the transition from business requirements to ER models involves a greater semantic distance than the ER-to-relational conversion, and (ii) this greater distance results in imprecise semantic elaboration, leading to lower learning performance.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Method</h3>\u0000 \u0000 <p>An experiment was designed in which participants, drawn from a database course, completed two sequential translation tasks: (a) business requirements to ER models, and (b) ER models to relational models. Participants' performances were assessed through problem-solving effectiveness and efficiency measures.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The problem-solving effectiveness and efficiency of task (b) surpassed those of task (a), suggesting that task (a) entailed greater semantic distance and less precise elaboration of semantic relationships than task (b).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Our findings suggest that semantic distance and semantic elaboration are critical factors in database design education, aligning with semantic network theory. Instruction should reduce semantic distance and pay more attention to the teaching of the business requirement to ER model transition. This may prompt scholars to develop more effective teaching methods for database design learning from the perspective of semantic distance.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145366569","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There is a growing emphasis on integrating engineering design into K-12 science, technology, engineering, and mathematics (STEM) education. Prior studies have primarily examined the impact of engineering design on student learning outcomes or employed frequency-based static statistical analyses, rather than focusing on student learning processes—that is, what students actually did and thought during engineering design courses.
Purpose/Hypotheses
This study explored the effects of two prominent engineering design approaches, forward and reverse, on learning outcomes and learning processes.
Design/Method
A mixed-methods quasi-experiment was conducted with a purposive sample of 52 11th-grade students enrolled in a mechatronics engineering design course. One class engaged in the forward engineering teaching condition (F-class; n = 28) and the other in the reverse engineering teaching condition (R-class; n = 24).
Results
Mann–Whitney U tests indicated that the two conditions exhibited no significant differences in overall design solutions. However, the R-class surpassed the F-class in understandability and organic qualities, using the creative product analysis matrix. The Mann–Whitney U test showed that the R-class scored significantly higher on mechatronics content knowledge than the F-class. Lag sequential analysis and epistemic network analysis revealed distinct differences in engineering design behavior, engineering design reflection, and cognitive action between the conditions.
Conclusions
These findings underscore the importance of applying multiple analytical lenses to examine outcomes and processes in engineering design learning. They offer practical implications for developing secondary-level engineering design curricula that support students' design solutions, conceptual understanding, design behavior, design reflection, and higher order cognitive action.
{"title":"Comparing forward and reverse engineering design approaches: Lag sequential and epistemic network analyses of student learning","authors":"Yu-Hung Chien, Chia-Yu Liu, Cheng-Shiun Tsai","doi":"10.1002/jee.70039","DOIUrl":"https://doi.org/10.1002/jee.70039","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>There is a growing emphasis on integrating engineering design into K-12 science, technology, engineering, and mathematics (STEM) education. Prior studies have primarily examined the impact of engineering design on student learning outcomes or employed frequency-based static statistical analyses, rather than focusing on student learning processes—that is, what students actually did and thought during engineering design courses.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose/Hypotheses</h3>\u0000 \u0000 <p>This study explored the effects of two prominent engineering design approaches, forward and reverse, on learning outcomes and learning processes.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Design/Method</h3>\u0000 \u0000 <p>A mixed-methods quasi-experiment was conducted with a purposive sample of 52 11th-grade students enrolled in a mechatronics engineering design course. One class engaged in the forward engineering teaching condition (F-class; <i>n</i> = 28) and the other in the reverse engineering teaching condition (R-class; <i>n</i> = 24).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Mann–Whitney <i>U</i> tests indicated that the two conditions exhibited no significant differences in overall design solutions. However, the R-class surpassed the F-class in understandability and organic qualities, using the creative product analysis matrix. The Mann–Whitney U test showed that the R-class scored significantly higher on mechatronics content knowledge than the F-class. Lag sequential analysis and epistemic network analysis revealed distinct differences in engineering design behavior, engineering design reflection, and cognitive action between the conditions.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>These findings underscore the importance of applying multiple analytical lenses to examine outcomes and processes in engineering design learning. They offer practical implications for developing secondary-level engineering design curricula that support students' design solutions, conceptual understanding, design behavior, design reflection, and higher order cognitive action.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145317626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computer science faces a persistent attrition problem, with people leaving the field at a rate that exceeds new entrants. Given the increasing demand for computing jobs, it is essential to focus on reducing the number of individuals exiting the field.
Purpose
This study investigates why individuals leave the computer science field across various stages and contexts, addressing two questions: (1) What are the reasons for leaving? (2) What external factors influence these decisions?
Method
This large-scale qualitative study collected over 10,000 Reddit posts using keyword-based scraping. Using generative AI, we refined the dataset, filtering it down to 263 relevant posts. Generative AI was then used for thematic analysis on this subset of posts, utilizing the established GATOS method. We extend this approach by integrating a human-in-the-loop process to contextualize the identified themes within social cognitive career theory.
Results
Findings reveal diverse reasons for leaving, including job dissatisfaction, interests in other fields, psychological factors, academic challenges, health concerns, and industry issues. Influential factors include background, transition requirements, alternative field characteristics, and personal circumstances. Although the extent varied, all of these reasons and factors were observed at every departure stage.
Conclusions
These findings provide important insights that can help inform industry and academic policies and practices. Additionally, we contribute to the development of more efficient, scalable workflows for future qualitative research using generative AI.
{"title":"Using generative AI for large-scale qualitative analysis of social media posts to understand why people leave computer science","authors":"Amanda Ross, Andrew Katz","doi":"10.1002/jee.70036","DOIUrl":"https://doi.org/10.1002/jee.70036","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Computer science faces a persistent attrition problem, with people leaving the field at a rate that exceeds new entrants. Given the increasing demand for computing jobs, it is essential to focus on reducing the number of individuals exiting the field.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>This study investigates why individuals leave the computer science field across various stages and contexts, addressing two questions: (1) What are the reasons for leaving? (2) What external factors influence these decisions?</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Method</h3>\u0000 \u0000 <p>This large-scale qualitative study collected over 10,000 Reddit posts using keyword-based scraping. Using generative AI, we refined the dataset, filtering it down to 263 relevant posts. Generative AI was then used for thematic analysis on this subset of posts, utilizing the established GATOS method. We extend this approach by integrating a human-in-the-loop process to contextualize the identified themes within social cognitive career theory.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Findings reveal diverse reasons for leaving, including job dissatisfaction, interests in other fields, psychological factors, academic challenges, health concerns, and industry issues. Influential factors include background, transition requirements, alternative field characteristics, and personal circumstances. Although the extent varied, all of these reasons and factors were observed at every departure stage.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>These findings provide important insights that can help inform industry and academic policies and practices. Additionally, we contribute to the development of more efficient, scalable workflows for future qualitative research using generative AI.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/jee.70036","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145224466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ibukun Osunbunmi, Taiwo Feyijimi, Stephanie Cutler, Yashin Brijmohan, Lexy Arinze, Viyon Dansu, Bolaji Bamidele, Jennifer Wu, Robert Rabb
Background
Attrition of engineering students continues to be a concern in higher education. Despite indications that students who opt to leave engineering programs may go on to make meaningful contributions in other fields more aligned to their interests, it remains essential to support those who choose to stay in engineering with the necessary resources, mentorship, and enabling environments to thrive.
Purpose
This study explores predictors of persistence to graduation for students in a College of Engineering (CoE), examining pre-college preparation (SAT scores), academic performance in core courses, demographic factors, and engagement in co-curricular activities.
Methods
We analyzed a 10-year dataset (fall 2007 to fall 2016) from a US R1 university's CoE, comprising 16,292 observations. Machine learning techniques, including dimensionality reduction (forward, backward, and unidirectional stepwise regression), explainable artificial intelligence, and predictive modeling (K-nearest neighbors, logistic regression, decision trees, artificial neural networks, and gradient boosting), were applied to identify significant predictors of persistence.
Results
Key predictors of persistence included students' GPAs in their first two years and SAT math. Additional factors, although not consistently ranked highly by all models, include performance in PHYS 211, CHM 110, and MAT 140 (Physics 1, Chemistry 1, and Calculus 1, respectively). Demographics and engaging in co-curricular activities also contribute to persistence, although not as significantly as academic factors.
Conclusion
Findings from the machine learning models extend Tinto's theory of persistence, and identify key factors that predict engineering students' persistence to graduation. We recommend that institutions engage in strategic planning and policymaking as part of their collective effort to reduce engineering student attrition.
{"title":"Artificial intelligence in engineering education research: Using machine learning models to predict undergraduate engineering students' persistence to graduation","authors":"Ibukun Osunbunmi, Taiwo Feyijimi, Stephanie Cutler, Yashin Brijmohan, Lexy Arinze, Viyon Dansu, Bolaji Bamidele, Jennifer Wu, Robert Rabb","doi":"10.1002/jee.70034","DOIUrl":"https://doi.org/10.1002/jee.70034","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Attrition of engineering students continues to be a concern in higher education. Despite indications that students who opt to leave engineering programs may go on to make meaningful contributions in other fields more aligned to their interests, it remains essential to support those who choose to stay in engineering with the necessary resources, mentorship, and enabling environments to thrive.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>This study explores predictors of persistence to graduation for students in a College of Engineering (CoE), examining pre-college preparation (SAT scores), academic performance in core courses, demographic factors, and engagement in co-curricular activities.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We analyzed a 10-year dataset (fall 2007 to fall 2016) from a US R1 university's CoE, comprising 16,292 observations. Machine learning techniques, including dimensionality reduction (forward, backward, and unidirectional stepwise regression), explainable artificial intelligence, and predictive modeling (K-nearest neighbors, logistic regression, decision trees, artificial neural networks, and gradient boosting), were applied to identify significant predictors of persistence.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Key predictors of persistence included students' GPAs in their first two years and SAT math. Additional factors, although not consistently ranked highly by all models, include performance in PHYS 211, CHM 110, and MAT 140 (Physics 1, Chemistry 1, and Calculus 1, respectively). Demographics and engaging in co-curricular activities also contribute to persistence, although not as significantly as academic factors.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Findings from the machine learning models extend Tinto's theory of persistence, and identify key factors that predict engineering students' persistence to graduation. We recommend that institutions engage in strategic planning and policymaking as part of their collective effort to reduce engineering student attrition.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/jee.70034","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145146571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Metacognitive processes have been linked to the development of conceptual knowledge in STEM courses, but previous work has centered on the regulatory aspects of metacognition.
Purpose
We interrogated the relationship between epistemic metacognition and conceptual knowledge in engineering statics courses across six universities by asking students a difficult concept question with concurrent reflection prompts that elicited their metacognitive thinking.
Method
We used a mixed-methods design containing an embedded phase followed by an explanatory phase. This design allowed us to both prompt and measure student epistemic metacognition within the learning context. The embedded phase consisted of quantitative and qualitative analyses of student responses. The explanatory phase consisted of an analysis of six instructor interviews.
Results
Analysis of 267 student responses showed greater variation in students' epistemic metacognition than in their ability to answer correctly. Students used different kinds of epistemic metacognitive resources about the nature and origin of knowledge, epistemological forms, epistemological activities, and stances toward knowledge. These resources generally assembled into one of two frames: a constructed knowledge framing valuing conceptual knowledge and sense-making, and an authoritative knowledge framing foregrounding numerical, algorithmic problem-solving. All six instructors interviewed described resources that align with both frames, and none explicitly considered student epistemic metacognition.
Conclusions
Instructors' explicit attention to epistemic metacognition can potentially shift students to more productive frames for engineering learning. Findings here also inform two broader issues in STEM instruction: student resistance to active learning, and the direct instruction versus inquiry-based learning debate.
{"title":"“More conceptual than actual”: Epistemic metacognition in response to a non-numerical statics question","authors":"Lorena S. Grundy, Milo D. Koretsky","doi":"10.1002/jee.70035","DOIUrl":"https://doi.org/10.1002/jee.70035","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Metacognitive processes have been linked to the development of conceptual knowledge in STEM courses, but previous work has centered on the regulatory aspects of metacognition.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>We interrogated the relationship between epistemic metacognition and conceptual knowledge in engineering statics courses across six universities by asking students a difficult concept question with concurrent reflection prompts that elicited their metacognitive thinking.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Method</h3>\u0000 \u0000 <p>We used a mixed-methods design containing an embedded phase followed by an explanatory phase. This design allowed us to both prompt and measure student epistemic metacognition within the learning context. The embedded phase consisted of quantitative and qualitative analyses of student responses. The explanatory phase consisted of an analysis of six instructor interviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Analysis of 267 student responses showed greater variation in students' epistemic metacognition than in their ability to answer correctly. Students used different kinds of epistemic metacognitive resources about the nature and origin of knowledge, epistemological forms, epistemological activities, and stances toward knowledge. These resources generally assembled into one of two frames: a <i>constructed knowledge framing</i> valuing conceptual knowledge and sense-making, and an <i>authoritative knowledge framing</i> foregrounding numerical, algorithmic problem-solving. All six instructors interviewed described resources that align with both frames, and none explicitly considered student epistemic metacognition.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Instructors' explicit attention to epistemic metacognition can potentially shift students to more productive frames for engineering learning. Findings here also inform two broader issues in STEM instruction: student resistance to active learning, and the direct instruction versus inquiry-based learning debate.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145101405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Priyanka Dhopade, James Tizard, Penelope Watson, Ashleigh Fox, Tom Allen, Hazim Namik, Aryan Karan, Rituparna Roy, Kelly Blincoe
Background
Women, ethnic minorities, and LGBTQIA+ people have historically been excluded from the engineering profession. When they do pursue engineering, they often face challenges within both education and industry. Retention is a growing issue; for example, women in industry have significantly higher turnover rates than men.
Purpose/Hypothesis
Feelings of belonging, satisfaction, and perceptions of one's future career are important for retention in engineering education. However, little is known about the factors that impact these constructs in tertiary education—where foundational engineering experiences occur—for a range of potentially intersectional social identities in contexts other than the United States.
Methods
We designed an online questionnaire (n = 379) and a series of focus groups (n = 17) with engineering students at Waipapa Taumata Rau (The University of Auckland) in Aotearoa (New Zealand). We applied thematic analysis to extract a list of common factors that influenced students' experiences in this unique context.
Results
Students who were unsure of or did not want to disclose parts of their identity reported the lowest sense of belonging and satisfaction. The factors that specifically impacted historically excluded groups included unsupportive working environments, not being respected academically, and exclusionary course content.
Conclusion
Our findings identify factors that contributed to students' experiences that may impact retention in Aotearoa but have implications for other contexts. Finally, we make recommendations to engineering education practitioners on how to support (and retain) students from historically excluded groups, including dedicated learning and social environments, inclusive course content, and awareness education on inclusivity.
{"title":"Factors affecting students' sense of inclusion in the undergraduate engineering program at Waipapa Taumata Rau (The University of Auckland)","authors":"Priyanka Dhopade, James Tizard, Penelope Watson, Ashleigh Fox, Tom Allen, Hazim Namik, Aryan Karan, Rituparna Roy, Kelly Blincoe","doi":"10.1002/jee.70029","DOIUrl":"https://doi.org/10.1002/jee.70029","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Women, ethnic minorities, and LGBTQIA+ people have historically been excluded from the engineering profession. When they do pursue engineering, they often face challenges within both education and industry. Retention is a growing issue; for example, women in industry have significantly higher turnover rates than men.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose/Hypothesis</h3>\u0000 \u0000 <p>Feelings of belonging, satisfaction, and perceptions of one's future career are important for retention in engineering education. However, little is known about the factors that impact these constructs in tertiary education—where foundational engineering experiences occur—for a range of potentially intersectional social identities in contexts other than the United States.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We designed an online questionnaire (<i>n</i> = 379) and a series of focus groups (<i>n</i> = 17) with engineering students at Waipapa Taumata Rau (The University of Auckland) in Aotearoa (New Zealand). We applied thematic analysis to extract a list of common factors that influenced students' experiences in this unique context.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Students who were unsure of or did not want to disclose parts of their identity reported the lowest sense of belonging and satisfaction. The factors that specifically impacted historically excluded groups included unsupportive working environments, not being respected academically, and exclusionary course content.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Our findings identify factors that contributed to students' experiences that may impact retention in Aotearoa but have implications for other contexts. Finally, we make recommendations to engineering education practitioners on how to support (and retain) students from historically excluded groups, including dedicated learning and social environments, inclusive course content, and awareness education on inclusivity.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/jee.70029","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145101723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Harpreet Auby, Namrata Shivagunde, Vijeta Deshpande, Anna Rumshisky, Milo D. Koretsky
Background
Analyzing student short-answer written justifications to conceptually challenging questions has proven helpful to understand student thinking and improve conceptual understanding. However, qualitative analyses are limited by the burden of analyzing large amounts of text.
Purpose
We apply dense and sparse Large Language Models (LLMs) to explore how machine learning can automate coding for responses in engineering mechanics and thermodynamics.
Design/Method
We first identify the cognitive resources students use through human coding of seven questions. We then compare the performance of four dense LLMs and a sparse Mixture of Experts (Mixtral) model to automate coding. Finally, we investigate the extent to which domain-specific training is necessary for accurate coding.
Findings
In a sample question, we analyze 904 responses to identify 48 unique cognitive resources, which we then organize into six themes. In contrast to recommendations in the literature, students who activate molecular resources were less likely to answer correctly. This example illustrates the usefulness of qualitatively analyzing large datasets. Of the LLMs, Mixtral and Llama-3 performed best at within the same-dataset, in-domain coding tasks, especially as the training set size increases. Phi-3.5-mini, while effective in mechanics, shows inconsistent improvements with additional data and struggles in thermodynamics. In contrast, GPT-4 and GPT-4o-mini stand out for their robust generalization across in- and cross-domain tasks.
Conclusions
Open-source models like Mixtral have the potential to perform well when coding short-answer justifications to challenging concept questions. However, more fine-tuning is needed so that they can be robust enough to be utilized with a resources-based framing.
{"title":"Analysis of student understanding in short-answer explanations to concept questions using a human-centered AI approach","authors":"Harpreet Auby, Namrata Shivagunde, Vijeta Deshpande, Anna Rumshisky, Milo D. Koretsky","doi":"10.1002/jee.70032","DOIUrl":"https://doi.org/10.1002/jee.70032","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Analyzing student short-answer written justifications to conceptually challenging questions has proven helpful to understand student thinking and improve conceptual understanding. However, qualitative analyses are limited by the burden of analyzing large amounts of text.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>We apply dense and sparse Large Language Models (LLMs) to explore how machine learning can automate coding for responses in engineering mechanics and thermodynamics.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Design/Method</h3>\u0000 \u0000 <p>We first identify the cognitive resources students use through human coding of seven questions. We then compare the performance of four dense LLMs and a sparse Mixture of Experts (Mixtral) model to automate coding. Finally, we investigate the extent to which domain-specific training is necessary for accurate coding.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Findings</h3>\u0000 \u0000 <p>In a sample question, we analyze 904 responses to identify 48 unique cognitive resources, which we then organize into six themes. In contrast to recommendations in the literature, students who activate molecular resources were less likely to answer correctly. This example illustrates the usefulness of qualitatively analyzing large datasets. Of the LLMs, Mixtral and Llama-3 performed best at within the same-dataset, in-domain coding tasks, especially as the training set size increases. Phi-3.5-mini, while effective in mechanics, shows inconsistent improvements with additional data and struggles in thermodynamics. In contrast, GPT-4 and GPT-4o-mini stand out for their robust generalization across in- and cross-domain tasks.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Open-source models like Mixtral have the potential to perform well when coding short-answer justifications to challenging concept questions. However, more fine-tuning is needed so that they can be robust enough to be utilized with a resources-based framing.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144923504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stephanie Fuchs, Alexandra Werth, Cristóbal Méndez, Jonathan Butcher
Background
High-quality feedback is crucial for academic success, driving student motivation and engagement while research explores effective delivery and student interactions. Advances in artificial intelligence (AI), particularly natural language processing (NLP), offer innovative methods for analyzing complex qualitative data such as feedback interactions.
Purpose
We developed a framework to train sentence transformers using generative AI–created synthetic data to categorize student-feedback interactions in engineering studios. We compared traditional thematic analysis with modern methods to evaluate the realism of synthetic datasets and their effectiveness in training NLP models by exploring how generative AI can aid qualitative coding.
Methods
We deidentified and transcribed eight audio recordings from engineering studios. Synthetic feedback transcripts were generated using three locally hosted large language models: Llama 3.1, Gemma 2.0, and Mistral NeMo, adjusting parameters to produce datasets mimicking the real transcripts. We assessed the quality of synthetic transcripts using our framework and used a sentence transformer model (trained on both real and synthetic data) to compare changes in the model's percent accuracy when qualitatively coding feedback interactions.
Results
Synthetic data improved the NLP model's performance in classifying feedback interactions, boosting the average accuracy from 68.4% to 81% with Llama 3.1. Although incorporating synthetic data improved classification, all models produced transcripts that occasionally included extraneous details and failed to capture instructor-dominant discourse.
Conclusions
Synthetic data offers an opportunity to expand qualitative research, particularly in contexts where real data for NLP training is limited or hard to obtain; however, transparency in its use is paramount to maintain research integrity.
{"title":"Leveraging AI-generated synthetic data to train natural language processing models for qualitative feedback analysis","authors":"Stephanie Fuchs, Alexandra Werth, Cristóbal Méndez, Jonathan Butcher","doi":"10.1002/jee.70033","DOIUrl":"https://doi.org/10.1002/jee.70033","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>High-quality feedback is crucial for academic success, driving student motivation and engagement while research explores effective delivery and student interactions. Advances in artificial intelligence (AI), particularly natural language processing (NLP), offer innovative methods for analyzing complex qualitative data such as feedback interactions.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>We developed a framework to train sentence transformers using generative AI–created synthetic data to categorize student-feedback interactions in engineering studios. We compared traditional thematic analysis with modern methods to evaluate the realism of synthetic datasets and their effectiveness in training NLP models by exploring how generative AI can aid qualitative coding.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We deidentified and transcribed eight audio recordings from engineering studios. Synthetic feedback transcripts were generated using three locally hosted large language models: Llama 3.1, Gemma 2.0, and Mistral NeMo, adjusting parameters to produce datasets mimicking the real transcripts. We assessed the quality of synthetic transcripts using our framework and used a sentence transformer model (trained on both real and synthetic data) to compare changes in the model's percent accuracy when qualitatively coding feedback interactions.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Synthetic data improved the NLP model's performance in classifying feedback interactions, boosting the average accuracy from 68.4% to 81% with Llama 3.1. Although incorporating synthetic data improved classification, all models produced transcripts that occasionally included extraneous details and failed to capture instructor-dominant discourse.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Synthetic data offers an opportunity to expand qualitative research, particularly in contexts where real data for NLP training is limited or hard to obtain; however, transparency in its use is paramount to maintain research integrity.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144923477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Given the increasing number of researchers engaging in engineering education research (EER) globally, there is intensifying interest in researcher development.
Purpose
This study systematically reviewed the literature guided by the research question: “What are the key factors and how do they influence the development of engineering education researchers?” The review aimed to contribute practical recommendations for enhancing engineering education researcher development and suggest future research directions.
Scope/Method
Database search with the application of screening criteria and a quality assessment resulted in 49 papers being included in the systematic review.
Results
Mapping the context of included papers showed that scholarly activity was increasing over time, that research on participants operating outside of the United States and Australia was underrepresented, and that studies were dominated by qualitative approaches applied to small sample sizes. The key factors influencing engineering education researcher development aligned to the following themes (and subthemes indicated in brackets): environment (funding, employment context, and recognition of impact); community (socialization and collaboration); knowledge acquisition (development pathways and engaging in a new research paradigm); and personal attributes (motivation and identity transition).
Conclusions
Recommendations are made for enhancing engineering education researcher development for researchers, EER leaders, and institutional leaders. Future research should seek to examine development experiences in a wider range of national contexts, ask more questions aligned to the size and significance of phenomena that require greater implementation of quantitative methods, and ask more comparative research questions that can foster understanding of what strategies have the greatest impact according to individual circumstances.
{"title":"Systematic literature review of the factors influencing development of engineering education researchers","authors":"Sarah Dart, Amy Young, Emma-Lee Steindl","doi":"10.1002/jee.70030","DOIUrl":"https://doi.org/10.1002/jee.70030","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Given the increasing number of researchers engaging in engineering education research (EER) globally, there is intensifying interest in researcher development.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>This study systematically reviewed the literature guided by the research question: “What are the key factors and how do they influence the development of engineering education researchers?” The review aimed to contribute practical recommendations for enhancing engineering education researcher development and suggest future research directions.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Scope/Method</h3>\u0000 \u0000 <p>Database search with the application of screening criteria and a quality assessment resulted in 49 papers being included in the systematic review.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Mapping the context of included papers showed that scholarly activity was increasing over time, that research on participants operating outside of the United States and Australia was underrepresented, and that studies were dominated by qualitative approaches applied to small sample sizes. The key factors influencing engineering education researcher development aligned to the following themes (and subthemes indicated in brackets): environment (funding, employment context, and recognition of impact); community (socialization and collaboration); knowledge acquisition (development pathways and engaging in a new research paradigm); and personal attributes (motivation and identity transition).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Recommendations are made for enhancing engineering education researcher development for researchers, EER leaders, and institutional leaders. Future research should seek to examine development experiences in a wider range of national contexts, ask more questions aligned to the size and significance of phenomena that require greater implementation of quantitative methods, and ask more comparative research questions that can foster understanding of what strategies have the greatest impact according to individual circumstances.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/jee.70030","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905556","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There have been many efforts designed to broaden the participation of racially minoritized groups in engineering fields, including Black men. One pathway to attaining an undergraduate engineering degree is the dual-degree program, which has been in existence for several decades. However, much remains unknown about students' experiences in these programs. This study contributes new knowledge by studying four Black male college students who are matriculating through a dual-degree program and expanding conceptualizations of engineering persistence.
Purpose
The purpose of this article is to examine the experiences of four Black male dual-degree engineering majors using the transfer-receiving framework. In so doing, this article also explores their pre-college interests that catapulted their engineering aspirations.
Methods
This study employs the case study methodological approach to understand the four Black men's experiences. More specifically, the study uses multiple cases to build descriptions and explanations, offer different viewpoints, and produce compelling interpretations across the cases.
Findings
The data revealed three overarching themes. The participants (i) had different, yet meaningful pre-college experiences that ignited their engineering interests; (ii) took advantage of the support mechanisms at the engineering-granting institution; and (iii) navigated various challenges as dual-degree students.
Conclusion
The study provides recommendations for practice, policy, and future research about ways to study and acquire more knowledge of dual-degree engineering programs, improve the transfer process, and cultivate future Black male engineers.
{"title":"Expanding conceptualizations of engineering persistence: Examining four undergraduate Black men's dual-degree experiences","authors":"Christopher C. Jett","doi":"10.1002/jee.70031","DOIUrl":"https://doi.org/10.1002/jee.70031","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>There have been many efforts designed to broaden the participation of racially minoritized groups in engineering fields, including Black men. One pathway to attaining an undergraduate engineering degree is the dual-degree program, which has been in existence for several decades. However, much remains unknown about students' experiences in these programs. This study contributes new knowledge by studying four Black male college students who are matriculating through a dual-degree program and expanding conceptualizations of engineering persistence.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Purpose</h3>\u0000 \u0000 <p>The purpose of this article is to examine the experiences of four Black male dual-degree engineering majors using the transfer-receiving framework. In so doing, this article also explores their pre-college interests that catapulted their engineering aspirations.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>This study employs the case study methodological approach to understand the four Black men's experiences. More specifically, the study uses multiple cases to build descriptions and explanations, offer different viewpoints, and produce compelling interpretations across the cases.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Findings</h3>\u0000 \u0000 <p>The data revealed three overarching themes. The participants (i) had different, yet meaningful pre-college experiences that ignited their engineering interests; (ii) took advantage of the support mechanisms at the engineering-granting institution; and (iii) navigated various challenges as dual-degree students.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>The study provides recommendations for practice, policy, and future research about ways to study and acquire more knowledge of dual-degree engineering programs, improve the transfer process, and cultivate future Black male engineers.</p>\u0000 </section>\u0000 </div>","PeriodicalId":50206,"journal":{"name":"Journal of Engineering Education","volume":"114 4","pages":""},"PeriodicalIF":3.4,"publicationDate":"2025-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144885110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号