Pub Date : 2024-04-05DOI: 10.1103/physrevphyseducres.20.010120
Danijela Dodlek, Gorazd Planinsic, Eugenia Etkina
Research carried out through the last 20 years gave us undeniable evidence that to learn anything we need to be active participants, not passive observers. One of the important aspects of learning physics is constructing explanations of physical phenomena. To support and guide students toward constructing their explanations, teachers need to be attentive and responsive to students’ explanations. To learn how physics teachers interpret and respond to students’ explanations we investigated pre- and in-service physics teachers’ responses to students’ written explanations of their answers to a complex physics problem. The survey administered to the participants included the problem statement and four authentic student explanations. The participants were asked to identify each student’s strengths and weaknesses and to provide a response to that student. We found that while the participants were successful in identifying productive and problematic aspects of student reasoning, they rarely built on student reasoning when responding to the students, mostly focusing on addressing problematic aspects. The paper discusses why this finding is important for physics teacher preparation programs and professional development programs.
{"title":"How to help students learn: An investigation of how in- and pre-service physics teachers respond to students’ explanations","authors":"Danijela Dodlek, Gorazd Planinsic, Eugenia Etkina","doi":"10.1103/physrevphyseducres.20.010120","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010120","url":null,"abstract":"Research carried out through the last 20 years gave us undeniable evidence that to learn anything we need to be active participants, not passive observers. One of the important aspects of learning physics is constructing explanations of physical phenomena. To support and guide students toward constructing their explanations, teachers need to be attentive and responsive to students’ explanations. To learn how physics teachers interpret and respond to students’ explanations we investigated pre- and in-service physics teachers’ responses to students’ written explanations of their answers to a complex physics problem. The survey administered to the participants included the problem statement and four authentic student explanations. The participants were asked to identify each student’s strengths and weaknesses and to provide a response to that student. We found that while the participants were successful in identifying productive and problematic aspects of student reasoning, they rarely built on student reasoning when responding to the students, mostly focusing on addressing problematic aspects. The paper discusses why this finding is important for physics teacher preparation programs and professional development programs.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"14 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-05DOI: 10.1103/physrevphyseducres.20.010121
Matthew Dew, Emma Hunt, Viranga Perera, Jonathan Perry, Gregorio Ponti, Andrew Loveridge
[This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Recent studies provide evidence that social constructivist pedagogical methods such as active learning, interactive engagement, and inquiry-based learning, while pedagogically more effective, can enable inequities in the classroom. By conducting a quantitative empirical examination of gender-inequitable group dynamics in two inquiry-based physics labs, we extend results of previous work. Using a survey on group work preferences and video recordings of lab sessions, we find similar patterns of gendered role taking noted in prior studies. These results are not reducible to differences in students’ preferences. We find that an intervention which employed partner agreement forms, with the goal of reducing inequities, had a positive impact on students’ engagement with equipment during a first-semester lab course. Our work will inform implementation of more effective interventions in the future and emphasizes challenges faced by instructors who are dedicated to both research-based pedagogical practices and efforts to promote diversity, equity, and inclusion in their classrooms.
{"title":"Group dynamics in inquiry-based labs: Gender inequities and the efficacy of partner agreements","authors":"Matthew Dew, Emma Hunt, Viranga Perera, Jonathan Perry, Gregorio Ponti, Andrew Loveridge","doi":"10.1103/physrevphyseducres.20.010121","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010121","url":null,"abstract":"[This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Recent studies provide evidence that social constructivist pedagogical methods such as active learning, interactive engagement, and inquiry-based learning, while pedagogically more effective, can enable inequities in the classroom. By conducting a quantitative empirical examination of gender-inequitable group dynamics in two inquiry-based physics labs, we extend results of previous work. Using a survey on group work preferences and video recordings of lab sessions, we find similar patterns of gendered role taking noted in prior studies. These results are not reducible to differences in students’ preferences. We find that an intervention which employed partner agreement forms, with the goal of reducing inequities, had a positive impact on students’ engagement with equipment during a first-semester lab course. Our work will inform implementation of more effective interventions in the future and emphasizes challenges faced by instructors who are dedicated to both research-based pedagogical practices and efforts to promote diversity, equity, and inclusion in their classrooms.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"37 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-05DOI: 10.1103/physrevphyseducres.20.010122
Moritz Waitzmann, Ruediger Scholz, Susanne Wessnigk
Clear and rigorous quantum reasoning is needed to explain quantum physical phenomena. As pillars of true quantum physical explanations, we suggest specific quantum reasoning derived from quantum physical key ideas. An experiment is suggested to support such a quantum reasoning, in which a quantized radiation field interacts with an optical beam splitter, leading to experimental results conflicting with classical physical predictions. The results, however, can be explained consistently with a quantum reasoning based on the key ideas of probability, superposition, and interference (PSI). In this quantum optical key experiment the optical beam splitter prepares a superposition of single photon states and a Michelson interferometer is used to detect the superposition via controlled propagation phases. Although different single photon experimental setups (aimed at helping students to gain access to foundational issues in quantum physics) have been discussed in the past, the wave-particle dualism bound to classical physics maintains its predominance as an explanation pattern for the interpretation of these experiments. The study presented here investigates the effect of the quantum optical key experiment on the ability of students to use quantum reasoning based on the key ideas of PSI to overcome the naive wave-particle dualism. The current state of relevant studies that test student access to quantum physics can roughly be divided into two distinct areas: one tests how mathematical abilities help them to understand quantum physics and one tests how nonmathematical representations of a set of specific quantum theoretical traits (“Wesenszüge”) lead to a deeper understanding of quantum physics. There is a lack of questionnaires that focus on the idea of developing quantum reasoning based on superposition, probability, and interference of quantum states combined with a real experiment using true quantum light. In the first part of the article, we describe the physical modeling and present the development of the questionnaire. The set of items has been constructed from newly developed items and combined with well-tested ones. The validation of the set addresses qualitative and quantitative methods. In the second part, we give a pre- and poststudy examination of the impact of the quantum optical key experiment on students’ quantum reasoning. A significant increase in the number of students using quantum arguments is based on PSI reasoning for the explanation of an interference, such as the behavior of single photon states. Though the increase is significant, we found only minor changes in a particular issue to the students’ reasoning when approaching quantum physics as illustrated by a sample of answers given in the second part of the article. The concept of quantum states and the principle of superposition still appear particularly difficult.
{"title":"Testing quantum reasoning: Developing, validating, and application of a questionnaire","authors":"Moritz Waitzmann, Ruediger Scholz, Susanne Wessnigk","doi":"10.1103/physrevphyseducres.20.010122","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010122","url":null,"abstract":"Clear and rigorous quantum reasoning is needed to explain quantum physical phenomena. As pillars of true quantum physical explanations, we suggest specific quantum reasoning derived from quantum physical key ideas. An experiment is suggested to support such a quantum reasoning, in which a quantized radiation field interacts with an optical beam splitter, leading to experimental results conflicting with classical physical predictions. The results, however, can be explained consistently with a quantum reasoning based on the key ideas of probability, superposition, and interference (PSI). In this quantum optical key experiment the optical beam splitter prepares a superposition of single photon states and a Michelson interferometer is used to detect the superposition via controlled propagation phases. Although different single photon experimental setups (aimed at helping students to gain access to foundational issues in quantum physics) have been discussed in the past, the wave-particle dualism bound to classical physics maintains its predominance as an explanation pattern for the interpretation of these experiments. The study presented here investigates the effect of the quantum optical key experiment on the ability of students to use quantum reasoning based on the key ideas of PSI to overcome the naive wave-particle dualism. The current state of relevant studies that test student access to quantum physics can roughly be divided into two distinct areas: one tests how mathematical abilities help them to understand quantum physics and one tests how nonmathematical representations of a set of specific quantum theoretical traits (“Wesenszüge”) lead to a deeper understanding of quantum physics. There is a lack of questionnaires that focus on the idea of developing quantum reasoning based on superposition, probability, and interference of quantum states combined with a real experiment using true quantum light. In the first part of the article, we describe the physical modeling and present the development of the questionnaire. The set of items has been constructed from newly developed items and combined with well-tested ones. The validation of the set addresses qualitative and quantitative methods. In the second part, we give a pre- and poststudy examination of the impact of the quantum optical key experiment on students’ quantum reasoning. A significant increase in the number of students using quantum arguments is based on PSI reasoning for the explanation of an interference, such as the behavior of single photon states. Though the increase is significant, we found only minor changes in a particular issue to the students’ reasoning when approaching quantum physics as illustrated by a sample of answers given in the second part of the article. The concept of quantum states and the principle of superposition still appear particularly difficult.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"53 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-02DOI: 10.1103/physrevphyseducres.20.010119
Melissa Dancy, Charles Henderson, Naneh Apkarian, Estrella Johnson, Marilyne Stains, Jeffrey R. Raker, Alexandra Lau
A survey of 722 physics faculty conducted in 2008 found that many physics instructors had knowledge of research-based instructional strategies (RBISs), were interested in using more, but often discontinued use after trying. Considerable effort has been made during the decade following 2008 to develop and disseminate RBISs in physics as well as change the culture within the physics community to value RBIS use and other forms of student-centered instruction. This paper uses data from a 2019 survey of 1176 physics instructors to understand the current state of RBIS use in college-level introductory physics, thus allowing us to better understand some of the impacts of these efforts on physics instruction. Results show that self-reported knowledge and use of RBISs has increased considerably and discontinuation is now relatively low. However, although the percentage of time lecturing is less than 10 years ago, many instructors still engage in substantial lecturing (i.e., more than one-third of class time). Relatedly, we find that the majority of RBIS use centers on pedagogies designed to supplement a primarily lecture-based classroom rather than pedagogies designed to support a primarily active learning classroom. This suggests that the physics education research community and beyond has done well promoting knowledge about RBISs and inspiring instructors to try RBISs in their courses. But, there is still room to improve. Based on available evidence about effective instructional practices, we recommend that change agents focus on supporting instructors to increase the percent of class time in active learning and to implement higher impact strategies.
{"title":"Physics instructors’ knowledge and use of active learning has increased over the last decade but most still lecture too much","authors":"Melissa Dancy, Charles Henderson, Naneh Apkarian, Estrella Johnson, Marilyne Stains, Jeffrey R. Raker, Alexandra Lau","doi":"10.1103/physrevphyseducres.20.010119","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010119","url":null,"abstract":"A survey of 722 physics faculty conducted in 2008 found that many physics instructors had knowledge of research-based instructional strategies (RBISs), were interested in using more, but often discontinued use after trying. Considerable effort has been made during the decade following 2008 to develop and disseminate RBISs in physics as well as change the culture within the physics community to value RBIS use and other forms of student-centered instruction. This paper uses data from a 2019 survey of 1176 physics instructors to understand the current state of RBIS use in college-level introductory physics, thus allowing us to better understand some of the impacts of these efforts on physics instruction. Results show that self-reported knowledge and use of RBISs has increased considerably and discontinuation is now relatively low. However, although the percentage of time lecturing is less than 10 years ago, many instructors still engage in substantial lecturing (i.e., more than one-third of class time). Relatedly, we find that the majority of RBIS use centers on pedagogies designed to supplement a primarily lecture-based classroom rather than pedagogies designed to support a primarily active learning classroom. This suggests that the physics education research community and beyond has done well promoting knowledge about RBISs and inspiring instructors to try RBISs in their courses. But, there is still room to improve. Based on available evidence about effective instructional practices, we recommend that change agents focus on supporting instructors to increase the percent of class time in active learning and to implement higher impact strategies.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"43 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140592937","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1103/physrevphyseducres.20.010118
Ann Y. Kim, Vina Ton, Daniel Vega
Female students, Latinx students, first-generation students, and transfer students often feel uncomfortable in science, technology, engineering, and mathematics (STEM) environments. However, some departments have been making progress in changing that. Guided by double consciousness and person-environment fit theory, we investigated the lived experiences of historically marginalized undergraduate and masters-level physics students at a large state university to understand how this particular department provides an environment encouraging all students they fit in physics. Graduated students and faculty were interviewed from California State University, Long Beach. Through the interviews, we gained an understanding of significant student experiences and their perceptions of fit in this physics environment. Department community members perceived the department environment to be open, which contributed to broadening fit and supporting diverse students to thrive. The importance of faculty agency in creating a welcoming and supportive physics environment is highlighted. Finally, we found students in this department take with them an approach to physics that they see applicable to other areas of study and their lives. We called this a physics state of mind. We include suggestions for other STEM departments based on the findings and previous research.
{"title":"Changing person-environment fit among underrepresented undergraduate physics students: Successes from a small department","authors":"Ann Y. Kim, Vina Ton, Daniel Vega","doi":"10.1103/physrevphyseducres.20.010118","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010118","url":null,"abstract":"Female students, Latinx students, first-generation students, and transfer students often feel uncomfortable in science, technology, engineering, and mathematics (STEM) environments. However, some departments have been making progress in changing that. Guided by double consciousness and person-environment fit theory, we investigated the lived experiences of historically marginalized undergraduate and masters-level physics students at a large state university to understand how this particular department provides an environment encouraging all students they fit in physics. Graduated students and faculty were interviewed from California State University, Long Beach. Through the interviews, we gained an understanding of significant student experiences and their perceptions of fit in this physics environment. Department community members perceived the department environment to be open, which contributed to broadening fit and supporting diverse students to thrive. The importance of faculty agency in creating a welcoming and supportive physics environment is highlighted. Finally, we found students in this department take with them an approach to physics that they see applicable to other areas of study and their lives. We called this a physics state of mind. We include suggestions for other STEM departments based on the findings and previous research.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"59 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140324955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-22DOI: 10.1103/physrevphyseducres.20.018002
Geoff Potvin, Zahra Hazari, Raina Khatri, Hemeng Cheng, T. Blake Head, Robynne M. Lock, Anne F. Kornahrens, Kathryne Sparks Woodle, Rebecca E. Vieyra, Beth A. Cunningham, Laird Kramer, Theodore Hodapp
{"title":"Reply to “Comment on ‘Examining the effect of counternarratives about physics on women’s physics career intentions’ ”","authors":"Geoff Potvin, Zahra Hazari, Raina Khatri, Hemeng Cheng, T. Blake Head, Robynne M. Lock, Anne F. Kornahrens, Kathryne Sparks Woodle, Rebecca E. Vieyra, Beth A. Cunningham, Laird Kramer, Theodore Hodapp","doi":"10.1103/physrevphyseducres.20.018002","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.018002","url":null,"abstract":"<span>DOI:</span><span>https://doi.org/10.1103/PhysRevPhysEducRes.20.018002</span>","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"23 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140201744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comment on “Examining the effect of counternarratives about physics on women’s physics career intentions”","authors":"M. B. Weissman, J. M. Robins","doi":"10.1103/physrevphyseducres.20.018001","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.018001","url":null,"abstract":"<span>DOI:</span><span>https://doi.org/10.1103/PhysRevPhysEducRes.20.018001</span>","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"25 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140201740","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-22DOI: 10.1103/physrevphyseducres.20.010117
C. F. J. Pols, P. J. J. M. Dekkers
[This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Acknowledgement of the limited learning outcomes in our first-year physics lab course, strikingly similar to the observed and reported issues in literature, incited renewal of the course with a focus on developing students’ ability to engage in experimental physics research. The procedural and conceptual knowledge (PACKS) model—addressing different types of knowledge required for scientific investigation—was used as a “guide” in the transformation of the course. This educational design research study—distinguishing three stages—describes our approach in transforming the course and provides theoretical insights and practical solutions through the combined study of both the process of learning and the means that support that process. The merits and trade-offs of our approach and the effectiveness of the course transformation are evaluated through surveys, interviews, and assessment of students’ inquiry skills. The findings provide insights into the application of the PACKS model and its effectiveness in facilitating students’ development of physics inquiry abilities. The results reveal an alignment between perceived, attained and intended learning goals. The self-conceived experiment at the end of the course showcases students’ successful integration of the targeted knowledge types, previously addressed in isolated “preparatory” activities. We argue that the PACKS-model and the design principles are useful attributes when transforming a traditional lab activity, but also specify the limitations.
{"title":"Redesigning a first year physics lab course on the basis of the procedural and conceptual knowledge in science model","authors":"C. F. J. Pols, P. J. J. M. Dekkers","doi":"10.1103/physrevphyseducres.20.010117","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010117","url":null,"abstract":"[This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Acknowledgement of the limited learning outcomes in our first-year physics lab course, strikingly similar to the observed and reported issues in literature, incited renewal of the course with a focus on developing students’ ability to engage in experimental physics research. The <i>procedural and conceptual knowledge (PACKS) model</i>—addressing different types of knowledge required for scientific investigation—was used as a “guide” in the transformation of the course. This educational design research study—distinguishing three stages—describes our approach in transforming the course and provides theoretical insights and practical solutions through the combined study of both the process of learning and the means that support that process. The merits and trade-offs of our approach and the effectiveness of the course transformation are evaluated through surveys, interviews, and assessment of students’ inquiry skills. The findings provide insights into the application of the PACKS model and its effectiveness in facilitating students’ development of physics inquiry abilities. The results reveal an alignment between perceived, attained and intended learning goals. The self-conceived experiment at the end of the course showcases students’ successful integration of the targeted knowledge types, previously addressed in isolated “preparatory” activities. We argue that the PACKS-model and the design principles are useful attributes when transforming a traditional lab activity, but also specify the limitations.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"32 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140205668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-22DOI: 10.1103/physrevphyseducres.20.010116
Jennifer Campbell, Katie Ansell, Tim Stelzer
Recent advances in publicly available natural language processors (NLP) may enhance the efficiency of analyzing student short-answer responses in physics education research (PER). We train a state-of-the-art NLP, IBM’s Watson, and test its agreement with human coders using two different studies that gathered text responses in which students explain their reasoning on physics-related questions. The first study analyzes 479 student responses to a lab data analysis question and categorizes them by main idea. The second study analyzes 732 student answers to identify the presence or absence of each of the two conceptual themes. When training Watson with approximately one-third to half of the samples, we find that samples labeled with high confidence scores have similar accuracy to human agreement; yet for lower confidence scores, humans outperform the NLP’s labeling accuracy. In addition to studying Watson’s overall accuracy, we use this analysis to better understand factors that impact how Watson categorizes. Using the data from the categorization study, we find that Watson’s algorithm does not appear to be impacted by the disproportionate representation of categories in the training set, and we examine mislabeled statements to identify vocabulary and phrasing that may increase the rate of false positives. Based on this work, we find that, with careful consideration of the research study design and an awareness of the NLP’s limitations, Watson may present a useful tool for large-scale PER studies or classroom analysis tools.
{"title":"Evaluating IBM’s Watson natural language processing artificial intelligence as a short-answer categorization tool for physics education research","authors":"Jennifer Campbell, Katie Ansell, Tim Stelzer","doi":"10.1103/physrevphyseducres.20.010116","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010116","url":null,"abstract":"Recent advances in publicly available natural language processors (NLP) may enhance the efficiency of analyzing student short-answer responses in physics education research (PER). We train a state-of-the-art NLP, IBM’s Watson, and test its agreement with human coders using two different studies that gathered text responses in which students explain their reasoning on physics-related questions. The first study analyzes 479 student responses to a lab data analysis question and categorizes them by main idea. The second study analyzes 732 student answers to identify the presence or absence of each of the two conceptual themes. When training Watson with approximately one-third to half of the samples, we find that samples labeled with high confidence scores have similar accuracy to human agreement; yet for lower confidence scores, humans outperform the NLP’s labeling accuracy. In addition to studying Watson’s overall accuracy, we use this analysis to better understand factors that impact how Watson categorizes. Using the data from the categorization study, we find that Watson’s algorithm does not appear to be impacted by the disproportionate representation of categories in the training set, and we examine mislabeled statements to identify vocabulary and phrasing that may increase the rate of false positives. Based on this work, we find that, with careful consideration of the research study design and an awareness of the NLP’s limitations, Watson may present a useful tool for large-scale PER studies or classroom analysis tools.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"121 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140198533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-12DOI: 10.1103/physrevphyseducres.20.010115
Mary Jane Brundage, David E. Meltzer, Chandralekha Singh
We use the Survey of Thermodynamic Processes and First and Second Laws-Long, a research-based survey instrument with 78 items at the level of introductory physics, to investigate introductory and advanced students’ difficulties with internal energy, work, and heat transfer. We present analysis of data from 12 different introductory and advanced physics classes at four different higher education public institutions in the U.S. in which the survey was administered in person to more than 1000 students. We find that not only introductory but also advanced physics students have many common difficulties with these introductory thermodynamic concepts after traditional lecture-based instruction in relevant concepts. We utilize a wide variety of problem types and contexts and our sample includes large numbers of introductory algebra-based, calculus-based, and advanced students. Some of our findings are consistent with prior research in this area, but others expand upon them and reveal previously unreported aspects of students’ thinking. Findings related to common difficulties of students before and after traditional lecture-based instruction in college physics courses can help instructors of these courses plan instruction and curricula to improve student understanding. These findings can also be valuable for developing effective research-based curricula and pedagogies to address student difficulties and help students develop a functional understanding of these fundamental thermodynamic concepts.
{"title":"Investigating introductory and advanced students’ difficulties with change in internal energy, work, and heat transfer using a validated instrument","authors":"Mary Jane Brundage, David E. Meltzer, Chandralekha Singh","doi":"10.1103/physrevphyseducres.20.010115","DOIUrl":"https://doi.org/10.1103/physrevphyseducres.20.010115","url":null,"abstract":"We use the Survey of Thermodynamic Processes and First and Second Laws-Long, a research-based survey instrument with 78 items at the level of introductory physics, to investigate introductory and advanced students’ difficulties with internal energy, work, and heat transfer. We present analysis of data from 12 different introductory and advanced physics classes at four different higher education public institutions in the U.S. in which the survey was administered in person to more than 1000 students. We find that not only introductory but also advanced physics students have many common difficulties with these introductory thermodynamic concepts after traditional lecture-based instruction in relevant concepts. We utilize a wide variety of problem types and contexts and our sample includes large numbers of introductory algebra-based, calculus-based, and advanced students. Some of our findings are consistent with prior research in this area, but others expand upon them and reveal previously unreported aspects of students’ thinking. Findings related to common difficulties of students before and after traditional lecture-based instruction in college physics courses can help instructors of these courses plan instruction and curricula to improve student understanding. These findings can also be valuable for developing effective research-based curricula and pedagogies to address student difficulties and help students develop a functional understanding of these fundamental thermodynamic concepts.","PeriodicalId":54296,"journal":{"name":"Physical Review Physics Education Research","volume":"2 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140147543","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：