EfficientNetV2-based dynamic gesture recognition using transformed scalogram from triaxial acceleration signal

IF 4.8 2区 工程技术 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Journal of Computational Design and Engineering Pub Date : 2023-07-03 DOI:10.1093/jcde/qwad068
Bumsoo Kim, Sanghyun Seo
{"title":"EfficientNetV2-based dynamic gesture recognition using transformed scalogram from triaxial acceleration signal","authors":"Bumsoo Kim, Sanghyun Seo","doi":"10.1093/jcde/qwad068","DOIUrl":null,"url":null,"abstract":"\n In this paper, a dynamic gesture recognition system is proposed using triaxial acceleration signal and image-based deep neural network. With our dexterous glove device, 1D acceleration signal can be measured from each finger and decomposed to time-divided frequency components via wavelet transformation, which known as scalogram as image-like format. To feed-forward the scalogram with single 2D convolutional neural networks(CNN) allows the gesture having temporality to be easily recognized without any complex system such as RNN, LSTM, or spatio-temporal feature as 3D CNN, etc. To classify the image with general input dimension of image RGB channels, we numerically reconstruct fifteen scalograms into one RGB image with various representation methods. In experiments, we employ the off-the-shelf model, EfficientNetV2 small to large model as an image classification model with fine-tuning. To evaluate our system, we bulid our custom bicycle hand signals as dynamic gesture dataset under our transformation system, and then qualitatively compare the reconstruction method with matrix representation methods. In addition, we use other signal transformation tools such as the fast Fourier transform, and short-time Fourier transform and then explain the advantages of scalogram classification in the terms of time-frequency resolution trade-off issue.","PeriodicalId":48611,"journal":{"name":"Journal of Computational Design and Engineering","volume":"53 1","pages":"1694-1706"},"PeriodicalIF":4.8000,"publicationDate":"2023-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computational Design and Engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1093/jcde/qwad068","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

Abstract

In this paper, a dynamic gesture recognition system is proposed using triaxial acceleration signal and image-based deep neural network. With our dexterous glove device, 1D acceleration signal can be measured from each finger and decomposed to time-divided frequency components via wavelet transformation, which known as scalogram as image-like format. To feed-forward the scalogram with single 2D convolutional neural networks(CNN) allows the gesture having temporality to be easily recognized without any complex system such as RNN, LSTM, or spatio-temporal feature as 3D CNN, etc. To classify the image with general input dimension of image RGB channels, we numerically reconstruct fifteen scalograms into one RGB image with various representation methods. In experiments, we employ the off-the-shelf model, EfficientNetV2 small to large model as an image classification model with fine-tuning. To evaluate our system, we bulid our custom bicycle hand signals as dynamic gesture dataset under our transformation system, and then qualitatively compare the reconstruction method with matrix representation methods. In addition, we use other signal transformation tools such as the fast Fourier transform, and short-time Fourier transform and then explain the advantages of scalogram classification in the terms of time-frequency resolution trade-off issue.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于effentnetv2的动态手势识别,利用变换后的三轴加速度信号尺度图
本文提出了一种基于三轴加速度信号和基于图像的深度神经网络的动态手势识别系统。我们的灵巧手套装置可以测量每个手指的一维加速度信号,并通过小波变换将其分解为时域频率分量,称为尺度图,类似图像格式。用单个二维卷积神经网络(CNN)对尺度图进行前馈,使得具有时间性的手势不需要像3D CNN那样使用RNN、LSTM或时空特征等复杂系统,就可以很容易地识别出来。为了对具有图像RGB通道一般输入维数的图像进行分类,我们用不同的表示方法对15个尺度图进行数值重建,得到了一幅RGB图像。在实验中,我们采用了现成的模型——EfficientNetV2从小到大模型作为图像分类模型,并进行了微调。为了评估我们的系统,我们在我们的变换系统下建立了自定义的自行车手势信号作为动态手势数据集,然后定性地比较了重构方法和矩阵表示方法。此外,我们还使用了其他信号变换工具,如快速傅立叶变换和短时傅立叶变换,然后解释了尺度图分类在时频分辨率权衡问题方面的优势。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Journal of Computational Design and Engineering
Journal of Computational Design and Engineering Computer Science-Human-Computer Interaction
CiteScore
7.70
自引率
20.40%
发文量
125
期刊介绍: Journal of Computational Design and Engineering is an international journal that aims to provide academia and industry with a venue for rapid publication of research papers reporting innovative computational methods and applications to achieve a major breakthrough, practical improvements, and bold new research directions within a wide range of design and engineering: • Theory and its progress in computational advancement for design and engineering • Development of computational framework to support large scale design and engineering • Interaction issues among human, designed artifacts, and systems • Knowledge-intensive technologies for intelligent and sustainable systems • Emerging technology and convergence of technology fields presented with convincing design examples • Educational issues for academia, practitioners, and future generation • Proposal on new research directions as well as survey and retrospectives on mature field.
期刊最新文献
A Study on Ship Hull Form Transformation Using Convolutional Autoencoder A new approach for solving global optimization and engineering problems based on modified Sea Horse Optimizer Multi-strategy enhanced kernel search optimization and its application in economic emission dispatch problems BRepGAT: Graph neural network to segment machining feature faces in a B-rep model Embedding Deep Neural Network in Enhanced Schapery Theory for Progressive Failure Analysis of Fiber Reinforced Laminates
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1