首页 > 最新文献

Frontiers in Neurorobotics最新文献

英文 中文
A multimodal educational robots driven via dynamic attention. 通过动态注意力驱动的多模式教育机器人
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-31 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1453061
An Jianliang

Introduction: With the development of artificial intelligence and robotics technology, the application of educational robots in teaching is becoming increasingly popular. However, effectively evaluating and optimizing multimodal educational robots remains a challenge.

Methods: This study introduces Res-ALBEF, a multimodal educational robot framework driven by dynamic attention. Res-ALBEF enhances the ALBEF (Align Before Fuse) method by incorporating residual connections to align visual and textual data more effectively before fusion. In addition, the model integrates a VGG19-based convolutional network for image feature extraction and utilizes a dynamic attention mechanism to dynamically focus on relevant parts of multimodal inputs. Our model was trained using a diverse dataset consisting of 50,000 multimodal educational instances, covering a variety of subjects and instructional content.

Results and discussion: The evaluation on an independent validation set of 10,000 samples demonstrated significant performance improvements: the model achieved an overall accuracy of 97.38% in educational content recognition. These results highlight the model's ability to improve alignment and fusion of multimodal information, making it a robust solution for multimodal educational robots.

导言:随着人工智能和机器人技术的发展,教育机器人在教学中的应用日益普及。然而,有效评估和优化多模态教育机器人仍是一项挑战:本研究介绍了由动态注意力驱动的多模态教育机器人框架 Res-ALBEF。Res-ALBEF增强了ALBEF(先对齐后融合)方法,通过整合残差连接,在融合前更有效地对齐视觉和文本数据。此外,该模型还集成了一个基于 VGG19 的卷积网络,用于图像特征提取,并利用动态注意力机制动态关注多模态输入的相关部分。我们的模型使用了一个由 50,000 个多模态教育实例组成的多样化数据集进行训练,涵盖了各种学科和教学内容:在由 10,000 个样本组成的独立验证集上进行的评估表明,模型的性能有了显著提高:在教育内容识别方面,模型的总体准确率达到了 97.38%。这些结果凸显了该模型在改善多模态信息的对齐和融合方面的能力,使其成为多模态教育机器人的强大解决方案。
{"title":"A multimodal educational robots driven via dynamic attention.","authors":"An Jianliang","doi":"10.3389/fnbot.2024.1453061","DOIUrl":"10.3389/fnbot.2024.1453061","url":null,"abstract":"<p><strong>Introduction: </strong>With the development of artificial intelligence and robotics technology, the application of educational robots in teaching is becoming increasingly popular. However, effectively evaluating and optimizing multimodal educational robots remains a challenge.</p><p><strong>Methods: </strong>This study introduces Res-ALBEF, a multimodal educational robot framework driven by dynamic attention. Res-ALBEF enhances the ALBEF (Align Before Fuse) method by incorporating residual connections to align visual and textual data more effectively before fusion. In addition, the model integrates a VGG19-based convolutional network for image feature extraction and utilizes a dynamic attention mechanism to dynamically focus on relevant parts of multimodal inputs. Our model was trained using a diverse dataset consisting of 50,000 multimodal educational instances, covering a variety of subjects and instructional content.</p><p><strong>Results and discussion: </strong>The evaluation on an independent validation set of 10,000 samples demonstrated significant performance improvements: the model achieved an overall accuracy of 97.38% in educational content recognition. These results highlight the model's ability to improve alignment and fusion of multimodal information, making it a robust solution for multimodal educational robots.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1453061"},"PeriodicalIF":2.6,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11560911/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142618559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
LS-VIT: Vision Transformer for action recognition based on long and short-term temporal difference. LS-VIT:基于长短时间差的动作识别视觉转换器。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-31 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1457843
Dong Chen, Peisong Wu, Mingdong Chen, Mengtao Wu, Tao Zhang, Chuanqi Li

Over the past few years, a growing number of researchers have dedicated their efforts to focusing on temporal modeling. The advent of transformer-based methods has notably advanced the field of 2D image-based vision tasks. However, with respect to 3D video tasks such as action recognition, applying temporal transformations directly to video data significantly increases both computational and memory demands. This surge in resource consumption is due to the multiplication of data patches and the added complexity of self-aware computations. Accordingly, building efficient and precise 3D self-attentive models for video content represents as a major challenge for transformers. In our research, we introduce an Long and Short-term Temporal Difference Vision Transformer (LS-VIT). This method incorporates short-term motion details into images by weighting the difference across several consecutive frames, thereby equipping the original image with the ability to model short-term motions. Concurrently, we integrate a module designed to understand long-term motion details. This module enhances the model's capacity for long-term motion modeling by directly integrating temporal differences from various segments via motion excitation. Our thorough analysis confirms that the LS-VIT achieves high recognition accuracy across multiple benchmarks (e.g., UCF101, HMDB51, Kinetics-400). These research results indicate that LS-VIT has the potential for further optimization, which can improve real-time performance and action prediction capabilities.

在过去几年中,越来越多的研究人员致力于时间建模。基于变换器的方法的出现显著推动了基于二维图像的视觉任务领域的发展。然而,对于三维视频任务(如动作识别)而言,直接对视频数据应用时空变换会大大增加计算和内存需求。资源消耗激增的原因是数据片段的倍增和自我感知计算的复杂性增加。因此,为视频内容建立高效、精确的三维自感知模型是变换器面临的一大挑战。在我们的研究中,我们引入了长短时差视觉变换器(LS-VIT)。这种方法通过对几个连续帧的差值进行加权处理,将短期运动细节纳入图像,从而使原始图像具备了建立短期运动模型的能力。与此同时,我们还集成了一个旨在理解长期运动细节的模块。该模块通过运动激励直接整合来自不同片段的时间差,从而增强了模型的长期运动建模能力。我们的全面分析证实,LS-VIT 在多个基准测试(如 UCF101、HMDB51、Kinetics-400)中都达到了很高的识别准确率。这些研究结果表明,LS-VIT 具有进一步优化的潜力,可以提高实时性能和动作预测能力。
{"title":"LS-VIT: Vision Transformer for action recognition based on long and short-term temporal difference.","authors":"Dong Chen, Peisong Wu, Mingdong Chen, Mengtao Wu, Tao Zhang, Chuanqi Li","doi":"10.3389/fnbot.2024.1457843","DOIUrl":"10.3389/fnbot.2024.1457843","url":null,"abstract":"<p><p>Over the past few years, a growing number of researchers have dedicated their efforts to focusing on temporal modeling. The advent of transformer-based methods has notably advanced the field of 2D image-based vision tasks. However, with respect to 3D video tasks such as action recognition, applying temporal transformations directly to video data significantly increases both computational and memory demands. This surge in resource consumption is due to the multiplication of data patches and the added complexity of self-aware computations. Accordingly, building efficient and precise 3D self-attentive models for video content represents as a major challenge for transformers. In our research, we introduce an Long and Short-term Temporal Difference Vision Transformer (LS-VIT). This method incorporates short-term motion details into images by weighting the difference across several consecutive frames, thereby equipping the original image with the ability to model short-term motions. Concurrently, we integrate a module designed to understand long-term motion details. This module enhances the model's capacity for long-term motion modeling by directly integrating temporal differences from various segments via motion excitation. Our thorough analysis confirms that the LS-VIT achieves high recognition accuracy across multiple benchmarks (e.g., UCF101, HMDB51, Kinetics-400). These research results indicate that LS-VIT has the potential for further optimization, which can improve real-time performance and action prediction capabilities.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1457843"},"PeriodicalIF":2.6,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11560894/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142618575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Neuro-motor controlled wearable augmentations: current research and emerging trends. 神经运动控制可穿戴增强设备:当前研究与新兴趋势。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-31 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1443010
Haneen Alsuradi, Joseph Hong, Helin Mazi, Mohamad Eid

Wearable augmentations (WAs) designed for movement and manipulation, such as exoskeletons and supernumerary robotic limbs, are used to enhance the physical abilities of healthy individuals and substitute or restore lost functionality for impaired individuals. Non-invasive neuro-motor (NM) technologies, including electroencephalography (EEG) and sufrace electromyography (sEMG), promise direct and intuitive communication between the brain and the WA. After presenting a historical perspective, this review proposes a conceptual model for NM-controlled WAs, analyzes key design aspects, such as hardware design, mounting methods, control paradigms, and sensory feedback, that have direct implications on the user experience, and in the long term, on the embodiment of WAs. The literature is surveyed and categorized into three main areas: hand WAs, upper body WAs, and lower body WAs. The review concludes by highlighting the primary findings, challenges, and trends in NM-controlled WAs. This review motivates researchers and practitioners to further explore and evaluate the development of WAs, ensuring a better quality of life.

为运动和操纵而设计的可穿戴增强装置(WA),如外骨骼和编外机器人肢体,可用于增强健康人的体能,替代或恢复受损人失去的功能。非侵入性神经运动(NM)技术,包括脑电图(EEG)和超声肌电图(sEMG),有望在大脑和WA之间实现直接而直观的交流。在介绍了历史视角之后,本综述提出了一个由 NM 控制的 WA 概念模型,分析了关键的设计方面,如硬件设计、安装方法、控制范例和感觉反馈,这些方面对用户体验有直接影响,从长远来看,对 WA 的体现有直接影响。文献概览分为三个主要领域:手部 WA、上半身 WA 和下半身 WA。综述最后强调了由 NM 控制的 WA 的主要发现、挑战和趋势。本综述激励研究人员和从业人员进一步探索和评估 WAs 的发展,以确保更好的生活质量。
{"title":"Neuro-motor controlled wearable augmentations: current research and emerging trends.","authors":"Haneen Alsuradi, Joseph Hong, Helin Mazi, Mohamad Eid","doi":"10.3389/fnbot.2024.1443010","DOIUrl":"10.3389/fnbot.2024.1443010","url":null,"abstract":"<p><p>Wearable augmentations (WAs) designed for movement and manipulation, such as exoskeletons and supernumerary robotic limbs, are used to enhance the physical abilities of healthy individuals and substitute or restore lost functionality for impaired individuals. Non-invasive neuro-motor (NM) technologies, including electroencephalography (EEG) and sufrace electromyography (sEMG), promise direct and intuitive communication between the brain and the WA. After presenting a historical perspective, this review proposes a conceptual model for NM-controlled WAs, analyzes key design aspects, such as hardware design, mounting methods, control paradigms, and sensory feedback, that have direct implications on the user experience, and in the long term, on the embodiment of WAs. The literature is surveyed and categorized into three main areas: hand WAs, upper body WAs, and lower body WAs. The review concludes by highlighting the primary findings, challenges, and trends in NM-controlled WAs. This review motivates researchers and practitioners to further explore and evaluate the development of WAs, ensuring a better quality of life.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1443010"},"PeriodicalIF":2.6,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11560910/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142618577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Editorial: Assistive and service robots for health and home applications (RH3 - Robot Helpers in Health and Home). 社论:用于健康和家庭应用的辅助和服务机器人(RH3--健康和家庭机器人助手)。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-29 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1503038
Paloma de la Puente, Markus Vincze, Diego Guffanti, Daniel Galan
{"title":"Editorial: Assistive and service robots for health and home applications (RH3 - Robot Helpers in Health and Home).","authors":"Paloma de la Puente, Markus Vincze, Diego Guffanti, Daniel Galan","doi":"10.3389/fnbot.2024.1503038","DOIUrl":"https://doi.org/10.3389/fnbot.2024.1503038","url":null,"abstract":"","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1503038"},"PeriodicalIF":2.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11554614/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142618571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A modified A* algorithm combining remote sensing technique to collect representative samples from unmanned surface vehicles. 一种结合遥感技术的改良 A* 算法,用于从无人驾驶地表飞行器上采集具有代表性的样本。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-22 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1488337
Lei Wang, Danping Liu, Jun Wang

Ensuring representativeness of collected samples is the most critical requirement of water sampling. Unmanned surface vehicles (USVs) have been widely adopted in water sampling, but current USV sampling path planning tend to overemphasize path optimization, neglecting the representative samples collection. This study proposed a modified A* algorithm that combined remote sensing technique while considering both path length and the representativeness of collected samples. Water quality parameters were initially retrieved using satellite remote sensing imagery and a deep belief network model, with the parameter value incorporated as coefficient Q in the heuristic function of A* algorithm. The adjustment coefficient k was then introduced into the coefficient Q to optimize the trade-off between sampling representativeness and path length. To evaluate the effectiveness of this algorithm, Chlorophyll-a concentration (Chl-a) was employed as the test parameter, with Chaohu Lake as the study area. Results showed that the algorithm was effective in collecting more representative samples in real-world conditions. As the coefficient k increased, the representativeness of collected samples enhanced, indicated by the Chl-a closely approximating the overall mean Chl-a and exhibiting a gradient distribution. This enhancement was also associated with increased path length. This study is significant in USV water sampling and water environment protection.

确保采集样本的代表性是水样采集的最关键要求。无人水面飞行器(USV)已被广泛应用于水样采集,但目前的 USV 采样路径规划往往过于强调路径优化,而忽视了样品采集的代表性。本研究提出了一种改进的 A* 算法,该算法结合了遥感技术,同时考虑了路径长度和采集样本的代表性。首先利用卫星遥感图像和深度信念网络模型检索水质参数,并将参数值作为系数 Q 加入 A* 算法的启发式函数中。然后在系数 Q 中引入调整系数 k,以优化取样代表性和路径长度之间的权衡。为评估该算法的有效性,以巢湖为研究区域,采用叶绿素 a 浓度(Chl-a)作为测试参数。结果表明,该算法在实际条件下能有效地采集到更具代表性的样本。随着系数 k 的增大,所采集样本的代表性增强,表现为 Chl-a 非常接近总体平均 Chl-a,并呈现梯度分布。这种增强也与路径长度的增加有关。这项研究对 USV 水样采集和水环境保护具有重要意义。
{"title":"A modified A* algorithm combining remote sensing technique to collect representative samples from unmanned surface vehicles.","authors":"Lei Wang, Danping Liu, Jun Wang","doi":"10.3389/fnbot.2024.1488337","DOIUrl":"10.3389/fnbot.2024.1488337","url":null,"abstract":"<p><p>Ensuring representativeness of collected samples is the most critical requirement of water sampling. Unmanned surface vehicles (USVs) have been widely adopted in water sampling, but current USV sampling path planning tend to overemphasize path optimization, neglecting the representative samples collection. This study proposed a modified A* algorithm that combined remote sensing technique while considering both path length and the representativeness of collected samples. Water quality parameters were initially retrieved using satellite remote sensing imagery and a deep belief network model, with the parameter value incorporated as coefficient <i>Q</i> in the heuristic function of A* algorithm. The adjustment coefficient <i>k</i> was then introduced into the coefficient <i>Q</i> to optimize the trade-off between sampling representativeness and path length. To evaluate the effectiveness of this algorithm, Chlorophyll-a concentration (Chl-a) was employed as the test parameter, with Chaohu Lake as the study area. Results showed that the algorithm was effective in collecting more representative samples in real-world conditions. As the coefficient <i>k</i> increased, the representativeness of collected samples enhanced, indicated by the Chl-a closely approximating the overall mean Chl-a and exhibiting a gradient distribution. This enhancement was also associated with increased path length. This study is significant in USV water sampling and water environment protection.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1488337"},"PeriodicalIF":2.6,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11535655/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142582574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
TL-CStrans Net: a vision robot for table tennis player action recognition driven via CS-Transformer. TL-CStrans Net:通过 CS 变压器驱动的乒乓球运动员动作识别视觉机器人。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-21 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1443177
Libo Ma, Yan Tong

Currently, the application of robotics technology in sports training and competitions is rapidly increasing. Traditional methods mainly rely on image or video data, neglecting the effective utilization of textual information. To address this issue, we propose: TL-CStrans Net: A vision robot for table tennis player action recognition driven via CS-Transformer. This is a multimodal approach that combines CS-Transformer, CLIP, and transfer learning techniques to effectively integrate visual and textual information. Firstly, we employ the CS-Transformer model as the neural computing backbone. By utilizing the CS-Transformer, we can effectively process visual information extracted from table tennis game scenes, enabling accurate stroke recognition. Then, we introduce the CLIP model, which combines computer vision and natural language processing. CLIP allows us to jointly learn representations of images and text, thereby aligning the visual and textual modalities. Finally, to reduce training and computational requirements, we leverage pre-trained CS-Transformer and CLIP models through transfer learning, which have already acquired knowledge from relevant domains, and apply them to table tennis stroke recognition tasks. Experimental results demonstrate the outstanding performance of TL-CStrans Net in table tennis stroke recognition. Our research is of significant importance in promoting the application of multimodal robotics technology in the field of sports and bridging the gap between neural computing, computer vision, and neuroscience.

目前,机器人技术在体育训练和比赛中的应用正在迅速增加。传统方法主要依赖图像或视频数据,忽视了文本信息的有效利用。针对这一问题,我们提出了:TL-CStrans Net:通过 CS 变换器驱动的乒乓球运动员动作识别视觉机器人。这是一种多模态方法,结合了 CS-Transformer、CLIP 和迁移学习技术,有效地整合了视觉和文本信息。首先,我们采用 CS-Transformer 模型作为神经计算骨干。通过利用 CS-Transformer,我们可以有效处理从乒乓球比赛场景中提取的视觉信息,从而实现准确的击球识别。然后,我们介绍了结合计算机视觉和自然语言处理的 CLIP 模型。CLIP 允许我们联合学习图像和文本的表征,从而使视觉和文本模式保持一致。最后,为了降低训练和计算要求,我们通过迁移学习利用预先训练好的 CS-Transformer 和 CLIP 模型,这些模型已经从相关领域获取了知识,并将它们应用于乒乓球击球识别任务。实验结果表明,TL-CStrans Net 在乒乓球击球识别中表现出色。我们的研究对于促进多模态机器人技术在体育领域的应用,以及弥合神经计算、计算机视觉和神经科学之间的鸿沟具有重要意义。
{"title":"TL-CStrans Net: a vision robot for table tennis player action recognition driven via CS-Transformer.","authors":"Libo Ma, Yan Tong","doi":"10.3389/fnbot.2024.1443177","DOIUrl":"10.3389/fnbot.2024.1443177","url":null,"abstract":"<p><p>Currently, the application of robotics technology in sports training and competitions is rapidly increasing. Traditional methods mainly rely on image or video data, neglecting the effective utilization of textual information. To address this issue, we propose: TL-CStrans Net: A vision robot for table tennis player action recognition driven via CS-Transformer. This is a multimodal approach that combines CS-Transformer, CLIP, and transfer learning techniques to effectively integrate visual and textual information. Firstly, we employ the CS-Transformer model as the neural computing backbone. By utilizing the CS-Transformer, we can effectively process visual information extracted from table tennis game scenes, enabling accurate stroke recognition. Then, we introduce the CLIP model, which combines computer vision and natural language processing. CLIP allows us to jointly learn representations of images and text, thereby aligning the visual and textual modalities. Finally, to reduce training and computational requirements, we leverage pre-trained CS-Transformer and CLIP models through transfer learning, which have already acquired knowledge from relevant domains, and apply them to table tennis stroke recognition tasks. Experimental results demonstrate the outstanding performance of TL-CStrans Net in table tennis stroke recognition. Our research is of significant importance in promoting the application of multimodal robotics technology in the field of sports and bridging the gap between neural computing, computer vision, and neuroscience.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1443177"},"PeriodicalIF":2.6,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11532032/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142575211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Erratum: Swimtrans Net: a multimodal robotic system for swimming action recognition driven via Swin-Transformer. 更正:Swimtrans Net:通过斯温变换器驱动的游泳动作识别多模式机器人系统。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-21 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1508032

[This corrects the article DOI: 10.3389/fnbot.2024.1452019.].

[此处更正了文章 DOI:10.3389/fnbot.2024.1452019.]。
{"title":"Erratum: Swimtrans Net: a multimodal robotic system for swimming action recognition driven via Swin-Transformer.","authors":"","doi":"10.3389/fnbot.2024.1508032","DOIUrl":"https://doi.org/10.3389/fnbot.2024.1508032","url":null,"abstract":"<p><p>[This corrects the article DOI: 10.3389/fnbot.2024.1452019.].</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1508032"},"PeriodicalIF":2.6,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11551536/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142618573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Cascade contour-enhanced panoptic segmentation for robotic vision perception. 用于机器人视觉感知的级联轮廓增强全景分割。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-21 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1489021
Yue Xu, Runze Liu, Dongchen Zhu, Lili Chen, Xiaolin Zhang, Jiamao Li

Panoptic segmentation plays a crucial role in enabling robots to comprehend their surroundings, providing fine-grained scene understanding information for robots' intelligent tasks. Although existing methods have made some progress, they are prone to fail in areas with weak textures, small objects, etc. Inspired by biological vision research, we propose a cascaded contour-enhanced panoptic segmentation network called CCPSNet, attempting to enhance the discriminability of instances through structural knowledge. To acquire the scene structure, a cascade contour detection stream is designed, which extracts comprehensive scene contours using channel regulation structural perception module and coarse-to-fine cascade strategy. Furthermore, the contour-guided multi-scale feature enhancement stream is developed to boost the discrimination ability for small objects and weak textures. The stream integrates contour information and multi-scale context features through structural-aware feature modulation module and inverse aggregation technique. Experimental results show that our method improves accuracy on the Cityscapes (61.2 PQ) and COCO (43.5 PQ) datasets while also demonstrating robustness in challenging simulated real-world complex scenarios faced by robots, such as dirty cameras and rainy conditions. The proposed network promises to help the robot perceive the real scene. In future work, an unsupervised training strategy for the network could be explored to reduce the training cost.

全景分割在帮助机器人理解周围环境方面发挥着至关重要的作用,它为机器人的智能任务提供了精细的场景理解信息。虽然现有的方法已经取得了一些进展,但在纹理较弱、物体较小等区域容易失效。受生物视觉研究的启发,我们提出了一种级联轮廓增强全景分割网络(CCPSNet),试图通过结构知识增强实例的可辨别性。为了获取场景结构,我们设计了一个级联轮廓检测流,利用通道调节结构感知模块和从粗到细的级联策略提取全面的场景轮廓。此外,还开发了轮廓引导的多尺度特征增强流,以提高对小物体和弱纹理的辨别能力。该信息流通过结构感知特征调制模块和反向聚合技术整合了轮廓信息和多尺度背景特征。实验结果表明,我们的方法在城市景观(61.2 PQ)和 COCO(43.5 PQ)数据集上提高了准确性,同时在机器人面临的具有挑战性的模拟真实世界复杂场景(如肮脏的摄像头和雨天环境)中也表现出了鲁棒性。拟议的网络有望帮助机器人感知真实场景。在未来的工作中,可以探索网络的无监督训练策略,以降低训练成本。
{"title":"Cascade contour-enhanced panoptic segmentation for robotic vision perception.","authors":"Yue Xu, Runze Liu, Dongchen Zhu, Lili Chen, Xiaolin Zhang, Jiamao Li","doi":"10.3389/fnbot.2024.1489021","DOIUrl":"10.3389/fnbot.2024.1489021","url":null,"abstract":"<p><p>Panoptic segmentation plays a crucial role in enabling robots to comprehend their surroundings, providing fine-grained scene understanding information for robots' intelligent tasks. Although existing methods have made some progress, they are prone to fail in areas with weak textures, small objects, etc. Inspired by biological vision research, we propose a cascaded contour-enhanced panoptic segmentation network called CCPSNet, attempting to enhance the discriminability of instances through structural knowledge. To acquire the scene structure, a cascade contour detection stream is designed, which extracts comprehensive scene contours using channel regulation structural perception module and coarse-to-fine cascade strategy. Furthermore, the contour-guided multi-scale feature enhancement stream is developed to boost the discrimination ability for small objects and weak textures. The stream integrates contour information and multi-scale context features through structural-aware feature modulation module and inverse aggregation technique. Experimental results show that our method improves accuracy on the Cityscapes (61.2 PQ) and COCO (43.5 PQ) datasets while also demonstrating robustness in challenging simulated real-world complex scenarios faced by robots, such as dirty cameras and rainy conditions. The proposed network promises to help the robot perceive the real scene. In future work, an unsupervised training strategy for the network could be explored to reduce the training cost.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1489021"},"PeriodicalIF":2.6,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11532083/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142577450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Design and analysis of exoskeleton devices for rehabilitation of distal radius fracture. 设计和分析用于桡骨远端骨折康复的外骨骼装置。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-18 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1477232
Zhiquan Chen, Jiabao Guo, Yishan Liu, Mengqian Tian, Xingsong Wang

In this work, the mechanical principles of external fixation and resistance training for the wrist affected by a distal radius fracture (DRF) are revealed. Based on the biomechanical analysis, two wearable exoskeleton devices are proposed to facilitate the DRF rehabilitation progress. Chronologically, the adjustable fixation device (AFD) provides fixed protection and limited mobilization of the fractured wrist in the early stage, while the functional recovery of relevant muscles is achieved by the resistance training device (RTD) in the later stage. According to the designed mechatronic systems of AFD and RTD, the experimental prototypes for these two apparatuses are established. By experiments, the actual motion ranges of AFD are investigated, and the feasibility in monitoring joint angles are validated. Meanwhile, the resistant influences of RTD are analyzed based on the surface electromyography (sEMG) signal features, the results demonstrate that the training-induced muscle strength enhancement is generally increased with the increment in external resistance. The exoskeleton devices presented in this work would be beneficial for the active rehabilitation of patients with DRF.

本研究揭示了桡骨远端骨折(DRF)腕部外固定和阻力训练的力学原理。根据生物力学分析,提出了两种可穿戴外骨骼装置,以促进桡骨远端骨折的康复进展。从时间上看,可调节固定装置(AFD)可在早期为骨折腕部提供固定保护和有限的活动能力,而阻力训练装置(RTD)则可在后期实现相关肌肉的功能恢复。根据所设计的 AFD 和 RTD 机电系统,建立了这两个装置的实验原型。通过实验,研究了 AFD 的实际运动范围,并验证了监测关节角度的可行性。同时,根据表面肌电图(sEMG)信号特征分析了 RTD 的阻力影响,结果表明训练引起的肌肉力量增强一般随外部阻力的增加而增加。本研究提出的外骨骼装置将有利于 DRF 患者的积极康复。
{"title":"Design and analysis of exoskeleton devices for rehabilitation of distal radius fracture.","authors":"Zhiquan Chen, Jiabao Guo, Yishan Liu, Mengqian Tian, Xingsong Wang","doi":"10.3389/fnbot.2024.1477232","DOIUrl":"10.3389/fnbot.2024.1477232","url":null,"abstract":"<p><p>In this work, the mechanical principles of external fixation and resistance training for the wrist affected by a distal radius fracture (DRF) are revealed. Based on the biomechanical analysis, two wearable exoskeleton devices are proposed to facilitate the DRF rehabilitation progress. Chronologically, the adjustable fixation device (AFD) provides fixed protection and limited mobilization of the fractured wrist in the early stage, while the functional recovery of relevant muscles is achieved by the resistance training device (RTD) in the later stage. According to the designed mechatronic systems of AFD and RTD, the experimental prototypes for these two apparatuses are established. By experiments, the actual motion ranges of AFD are investigated, and the feasibility in monitoring joint angles are validated. Meanwhile, the resistant influences of RTD are analyzed based on the surface electromyography (sEMG) signal features, the results demonstrate that the training-induced muscle strength enhancement is generally increased with the increment in external resistance. The exoskeleton devices presented in this work would be beneficial for the active rehabilitation of patients with DRF.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1477232"},"PeriodicalIF":2.6,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11527727/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142570965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
NAN-DETR: noising multi-anchor makes DETR better for object detection. NAN-DETR:噪声多锚使 DETR 更好地用于物体检测。
IF 2.6 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-10-14 eCollection Date: 2024-01-01 DOI: 10.3389/fnbot.2024.1484088
Zixin Huang, Xuesong Tao, Xinyuan Liu

Object detection plays a crucial role in robotic vision, focusing on accurately identifying and localizing objects within images. However, many existing methods encounter limitations, particularly when it comes to effectively implementing a one-to-many matching strategy. To address these challenges, we propose NAN-DETR (Noising Multi-Anchor Detection Transformer), an innovative framework based on DETR (Detection Transformer). NAN-DETR introduces three key improvements to transformer-based object detection: a decoder-based multi-anchor strategy, a centralization noising mechanism, and the integration of Complete Intersection over Union (CIoU) loss. The multi-anchor strategy leverages multiple anchors per object, significantly enhancing detection accuracy by improving the one-to-many matching process. The centralization noising mechanism mitigates conflicts among anchors by injecting controlled noise into the detection boxes, thereby increasing the robustness of the model. Additionally, CIoU loss, which incorporates both aspect ratio and spatial distance in its calculations, results in more precise bounding box predictions compared to the conventional IoU loss. Although NAN-DETR may not drastically improve real-time processing capabilities, its exceptional performance positions it as a highly reliable solution for diverse object detection scenarios.

物体检测在机器人视觉中起着至关重要的作用,其重点是准确识别和定位图像中的物体。然而,许多现有方法都存在局限性,尤其是在有效实施一对多匹配策略时。为了应对这些挑战,我们提出了基于 DETR(检测变换器)的创新框架 NAN-DETR(噪声多锚检测变换器)。NAN-DETR 对基于变换器的物体检测引入了三项关键改进:基于解码器的多锚(multi-anchor)策略、集中噪声机制以及完整交叉联合(CIoU)损失的集成。多锚策略利用每个对象的多个锚点,通过改进一对多的匹配过程显著提高了检测精度。集中噪声机制通过向检测盒注入受控噪声来缓解锚点之间的冲突,从而提高模型的鲁棒性。此外,CIoU 丢失在计算中同时考虑了长宽比和空间距离,因此与传统的 IoU 丢失相比,CIoU 丢失能更精确地预测边界框。尽管 NAN-DETR 可能无法大幅提高实时处理能力,但其卓越的性能使其成为适用于各种物体检测场景的高度可靠的解决方案。
{"title":"NAN-DETR: noising multi-anchor makes DETR better for object detection.","authors":"Zixin Huang, Xuesong Tao, Xinyuan Liu","doi":"10.3389/fnbot.2024.1484088","DOIUrl":"10.3389/fnbot.2024.1484088","url":null,"abstract":"<p><p>Object detection plays a crucial role in robotic vision, focusing on accurately identifying and localizing objects within images. However, many existing methods encounter limitations, particularly when it comes to effectively implementing a one-to-many matching strategy. To address these challenges, we propose NAN-DETR (Noising Multi-Anchor Detection Transformer), an innovative framework based on DETR (Detection Transformer). NAN-DETR introduces three key improvements to transformer-based object detection: a decoder-based multi-anchor strategy, a centralization noising mechanism, and the integration of Complete Intersection over Union (CIoU) loss. The multi-anchor strategy leverages multiple anchors per object, significantly enhancing detection accuracy by improving the one-to-many matching process. The centralization noising mechanism mitigates conflicts among anchors by injecting controlled noise into the detection boxes, thereby increasing the robustness of the model. Additionally, CIoU loss, which incorporates both aspect ratio and spatial distance in its calculations, results in more precise bounding box predictions compared to the conventional IoU loss. Although NAN-DETR may not drastically improve real-time processing capabilities, its exceptional performance positions it as a highly reliable solution for diverse object detection scenarios.</p>","PeriodicalId":12628,"journal":{"name":"Frontiers in Neurorobotics","volume":"18 ","pages":"1484088"},"PeriodicalIF":2.6,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11513373/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142521681","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Frontiers in Neurorobotics
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1