{"title":"在网络浏览器中评估手写技能:日语汉字自动在线测试的开发与验证。","authors":"Tomohiro Inoue, Yucan Chen, Toshio Ohyanagi","doi":"10.3758/s13428-024-02562-6","DOIUrl":null,"url":null,"abstract":"<p><p>Online language and literacy assessments have become prevalent in research and practice across settings. However, a notable exception is the assessment of handwriting and spelling, which has traditionally been conducted in person with paper and pencil. In light of this, we developed an automated, browser-based handwriting test application (Online Assessment of Handwriting and Spelling: OAHaS) for Japanese Kanji (Study 1) and examined its psychometric properties (Study 2). The automated scoring function using convolutional neural network (CNN) models achieved high recall (98.7%) and specificity (84.4%), as well as high agreement with manual scoring (95.4%). Additionally, behavioral validation with data from primary school children (N = 261, 49.0% female, age range = 6-12 years) indicated the high reliability and validity of our online test application, with a strong correlation between children's scores on the online and paper-based tests (r = .86). Moreover, our analysis indicated the potential utility of writing fluency measures (latency and duration) that are automatically recorded by OAHaS. Taken together, our browser-based application demonstrated the feasibility and viability of remote and automated assessment of handwriting skills, providing a streamlined approach to research and practice on handwriting. The source code of the application and supporting materials are available on Open Science Framework ( https://osf.io/gver2/ ).</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"32"},"PeriodicalIF":4.6000,"publicationDate":"2024-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11685258/pdf/","citationCount":"0","resultStr":"{\"title\":\"Assessing handwriting skills in a web browser: Development and validation of an automated online test in Japanese Kanji.\",\"authors\":\"Tomohiro Inoue, Yucan Chen, Toshio Ohyanagi\",\"doi\":\"10.3758/s13428-024-02562-6\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Online language and literacy assessments have become prevalent in research and practice across settings. However, a notable exception is the assessment of handwriting and spelling, which has traditionally been conducted in person with paper and pencil. In light of this, we developed an automated, browser-based handwriting test application (Online Assessment of Handwriting and Spelling: OAHaS) for Japanese Kanji (Study 1) and examined its psychometric properties (Study 2). The automated scoring function using convolutional neural network (CNN) models achieved high recall (98.7%) and specificity (84.4%), as well as high agreement with manual scoring (95.4%). Additionally, behavioral validation with data from primary school children (N = 261, 49.0% female, age range = 6-12 years) indicated the high reliability and validity of our online test application, with a strong correlation between children's scores on the online and paper-based tests (r = .86). Moreover, our analysis indicated the potential utility of writing fluency measures (latency and duration) that are automatically recorded by OAHaS. Taken together, our browser-based application demonstrated the feasibility and viability of remote and automated assessment of handwriting skills, providing a streamlined approach to research and practice on handwriting. The source code of the application and supporting materials are available on Open Science Framework ( https://osf.io/gver2/ ).</p>\",\"PeriodicalId\":8717,\"journal\":{\"name\":\"Behavior Research Methods\",\"volume\":\"57 1\",\"pages\":\"32\"},\"PeriodicalIF\":4.6000,\"publicationDate\":\"2024-12-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11685258/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Behavior Research Methods\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.3758/s13428-024-02562-6\",\"RegionNum\":2,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PSYCHOLOGY, EXPERIMENTAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Behavior Research Methods","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.3758/s13428-024-02562-6","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, EXPERIMENTAL","Score":null,"Total":0}
引用次数: 0
摘要
在线语言和读写能力评估在各种研究和实践中都很普遍。然而,一个值得注意的例外是对笔迹和拼写的评估,传统上是亲自用纸和铅笔进行的。鉴于此,我们开发了一个基于浏览器的日语汉字自动书写测试应用程序(Online Assessment of handwriting and Spelling: OAHaS)(研究1),并检验了其心理测量学特性(研究2)。使用卷积神经网络(CNN)模型的自动评分功能达到了高召回率(98.7%)和特异性(84.4%),与人工评分的一致性(95.4%)很高。此外,对小学生(N = 261,女性49.0%,年龄范围= 6-12岁)的行为验证表明,我们的在线测试应用程序具有高信度和效度,儿童在线测试成绩与纸质测试成绩之间存在很强的相关性(r = .86)。此外,我们的分析表明,oaa自动记录的写作流畅性测量(延迟和持续时间)的潜在效用。总之,我们基于浏览器的应用程序展示了远程和自动评估手写技能的可行性和可行性,为手写研究和实践提供了一种简化的方法。该应用程序的源代码和支持材料可在Open Science Framework (https://osf.io/gver2/)上获得。
Assessing handwriting skills in a web browser: Development and validation of an automated online test in Japanese Kanji.
Online language and literacy assessments have become prevalent in research and practice across settings. However, a notable exception is the assessment of handwriting and spelling, which has traditionally been conducted in person with paper and pencil. In light of this, we developed an automated, browser-based handwriting test application (Online Assessment of Handwriting and Spelling: OAHaS) for Japanese Kanji (Study 1) and examined its psychometric properties (Study 2). The automated scoring function using convolutional neural network (CNN) models achieved high recall (98.7%) and specificity (84.4%), as well as high agreement with manual scoring (95.4%). Additionally, behavioral validation with data from primary school children (N = 261, 49.0% female, age range = 6-12 years) indicated the high reliability and validity of our online test application, with a strong correlation between children's scores on the online and paper-based tests (r = .86). Moreover, our analysis indicated the potential utility of writing fluency measures (latency and duration) that are automatically recorded by OAHaS. Taken together, our browser-based application demonstrated the feasibility and viability of remote and automated assessment of handwriting skills, providing a streamlined approach to research and practice on handwriting. The source code of the application and supporting materials are available on Open Science Framework ( https://osf.io/gver2/ ).
期刊介绍:
Behavior Research Methods publishes articles concerned with the methods, techniques, and instrumentation of research in experimental psychology. The journal focuses particularly on the use of computer technology in psychological research. An annual special issue is devoted to this field.