Wearable smart glasses for first-person video analysis to evaluate nursing skills: A pilot study with a mixed method design

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH Journal of Computer Assisted Learning Pub Date : 2024-10-26 DOI:10.1111/jcal.13080

Sun Kyung Kim, Youngho Lee, Hye Ri Hwang, Oe Nam Kim

{"title":"Wearable smart glasses for first-person video analysis to evaluate nursing skills: A pilot study with a mixed method design","authors":"Sun Kyung Kim, Youngho Lee, Hye Ri Hwang, Oe Nam Kim","doi":"10.1111/jcal.13080","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Comprehensive assessment of skills and performance are necessary to improve the quality of care in nursing education. Various factors pose challenges to accurate assessments, including high student–teacher ratio and observer bias.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>To establish an assessment system based on first-person video of smart glasses and validate its usability and feasibility.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We used two cameras (smart glasses for first-person perspective and a web cam) to perform video analysis for skill and performance assessments. Usability tests were conducted using quantitative and qualitative questionnaires for the usefulness and acceptability of the current assessment system. Data from standardised checklists for three clinical nursing skills (intramuscular injection, endotracheal suction, and Levin tube feeding) were used to analyse the inter-rater reliability in addition to Fleiss' Kappa coefficient analysis. A total of 22 participants (18 students and 4 reviewers) were recruited. They completed the study. Apart from physical discomfort component, the proposed smart glasses-based first-person video assessment system achieved good usability scores from both students and reviewers.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>Fleiss' Kappa values for individual items ranged from 0.450 to 0.808, with an overall average value of 0.592. Results of this study validated the potential of smart glasses-based assessment system where a set of beneficial mechanisms may meaningfully support the formative learning process. However, issues regarding upgrading the device and need for guidelines for roles of raters and students still remain.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 1","pages":""},"PeriodicalIF":5.1000,"publicationDate":"2024-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.13080","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Comprehensive assessment of skills and performance are necessary to improve the quality of care in nursing education. Various factors pose challenges to accurate assessments, including high student–teacher ratio and observer bias.

Objectives

To establish an assessment system based on first-person video of smart glasses and validate its usability and feasibility.

Methods

We used two cameras (smart glasses for first-person perspective and a web cam) to perform video analysis for skill and performance assessments. Usability tests were conducted using quantitative and qualitative questionnaires for the usefulness and acceptability of the current assessment system. Data from standardised checklists for three clinical nursing skills (intramuscular injection, endotracheal suction, and Levin tube feeding) were used to analyse the inter-rater reliability in addition to Fleiss' Kappa coefficient analysis. A total of 22 participants (18 students and 4 reviewers) were recruited. They completed the study. Apart from physical discomfort component, the proposed smart glasses-based first-person video assessment system achieved good usability scores from both students and reviewers.

Results and Conclusions

Fleiss' Kappa values for individual items ranged from 0.450 to 0.808, with an overall average value of 0.592. Results of this study validated the potential of smart glasses-based assessment system where a set of beneficial mechanisms may meaningfully support the formative learning process. However, issues regarding upgrading the device and need for guidelines for roles of raters and students still remain.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

用于第一人称视频分析评估护理技能的可穿戴智能眼镜：一项混合方法设计的试点研究

背景在护理教育中，综合评估技能和绩效是提高护理质量的必要条件。各种因素对准确评估构成挑战，包括高师生比例和观察者偏见。目的建立基于第一人称视频的智能眼镜评估系统，验证其可用性和可行性。方法我们使用两个摄像头（第一人称视角的智能眼镜和一个网络摄像头）进行视频分析，以评估技能和表现。可用性测试采用定量和定性问卷，以评估当前评估系统的有用性和可接受性。采用三种临床护理技能（肌内注射、气管内吸引和Levin管饲）的标准化核对表数据，除Fleiss' Kappa系数分析外，还分析了评分间信度。共招募了22名参与者（18名学生和4名审稿人）。他们完成了这项研究。除了身体不适的部分，所提出的基于智能眼镜的第一人称视频评估系统在学生和评论者中都获得了很好的可用性分数。结果与结论各单项Fleiss’Kappa值范围为0.450 ~ 0.808，总体平均值为0.592。本研究的结果验证了基于智能眼镜的评估系统的潜力，其中一套有益的机制可能有意地支持形成性学习过程。然而，有关设备升级的问题，以及对评分者和学生角色的指导方针的需求仍然存在。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope