Engineering Applications of Artificial Intelligence最新文献_第9页

Fine-tuning pre-trained networks with emphasis on image segmentation: A multi-network approach for enhanced breast cancer detection 微调预训练网络，重点是图像分割：增强乳腺癌检测的多网络方法

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-18 DOI: 10.1016/j.engappai.2024.109666

Parviz Ghafariasl , Masoomeh Zeinalnezhad , Shing Chang

Accurate classification of mammography images into normal and cancerous categories is critical for the early detection of breast cancer. This study utilizes transfer learning and deep learning models to extract and diversify features from a combined dataset consisting of the Mini Digital Database for Screening Mammography (DDSM, containing 7808 images) and the Mammographic Image Analysis Society (MIAS) dataset (containing 322 images). The preprocessing steps involve image cropping, removal of artifacts, and enhancement of contrast using Contrast-Limited Adaptive Histogram Equalization (CLAHE). Data augmentation techniques, including the application of median and Gaussian blur, were used to improve the robustness of the models. Three pre-trained networks—Residual Networks with 50 layers (ResNet-50), Visual Geometry Group Network with 19 layers (VGG-19), and Residual Networks with 152 layers Version 2 (ResNet-152V2)—were fine-tuned specifically for mammography data. Image segmentation and the removal of the pectoral muscle significantly improved classification accuracy. The VGG-19 model achieved an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.80 for segmented images and 0.86 for non-segmented images. A stacked generalization model, which combined features from all three networks, further optimized performance. Artificial Neural Network (ANN) and Extreme Gradient Boosting (XGBoost) models achieved AUCs of 0.897 and 0.890, respectively, for segmented images. Data augmentation improved performance by 2.7%–4%.

准确地将乳腺 X 射线摄影图像分为正常和癌变两类对于早期检测乳腺癌至关重要。本研究利用迁移学习和深度学习模型，从由筛查乳腺摄影小型数字数据库（DDSM，包含 7808 幅图像）和乳腺图像分析协会（MIAS）数据集（包含 322 幅图像）组成的组合数据集中提取特征并使其多样化。预处理步骤包括图像裁剪、去除伪影以及使用对比度限制自适应直方图均衡化（CLAHE）增强对比度。为了提高模型的鲁棒性，还使用了数据增强技术，包括应用中值模糊和高斯模糊。三个预先训练好的网络--50 层残差网络（ResNet-50）、19 层视觉几何组网络（VGG-19）和 152 层残差网络第 2 版（ResNet-152V2）--专门针对乳腺 X 射线摄影数据进行了微调。图像分割和去除胸肌大大提高了分类准确性。VGG-19 模型的接收者工作特征曲线下面积 (AUC) 在分割图像中为 0.80，在非分割图像中为 0.86。叠加泛化模型结合了所有三个网络的特征，进一步优化了性能。人工神经网络（ANN）和极梯度提升（XGBoost）模型对分割图像的 AUC 分别达到了 0.897 和 0.890。数据增强将性能提高了 2.7%-4%。

{"title":"Fine-tuning pre-trained networks with emphasis on image segmentation: A multi-network approach for enhanced breast cancer detection","authors":"Parviz Ghafariasl , Masoomeh Zeinalnezhad , Shing Chang","doi":"10.1016/j.engappai.2024.109666","DOIUrl":"10.1016/j.engappai.2024.109666","url":null,"abstract":"<div><div>Accurate classification of mammography images into normal and cancerous categories is critical for the early detection of breast cancer. This study utilizes transfer learning and deep learning models to extract and diversify features from a combined dataset consisting of the Mini Digital Database for Screening Mammography (DDSM, containing 7808 images) and the Mammographic Image Analysis Society (MIAS) dataset (containing 322 images). The preprocessing steps involve image cropping, removal of artifacts, and enhancement of contrast using Contrast-Limited Adaptive Histogram Equalization (CLAHE). Data augmentation techniques, including the application of median and Gaussian blur, were used to improve the robustness of the models. Three pre-trained networks—Residual Networks with 50 layers (ResNet-50), Visual Geometry Group Network with 19 layers (VGG-19), and Residual Networks with 152 layers Version 2 (ResNet-152V2)—were fine-tuned specifically for mammography data. Image segmentation and the removal of the pectoral muscle significantly improved classification accuracy. The VGG-19 model achieved an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.80 for segmented images and 0.86 for non-segmented images. A stacked generalization model, which combined features from all three networks, further optimized performance. Artificial Neural Network (ANN) and Extreme Gradient Boosting (XGBoost) models achieved AUCs of 0.897 and 0.890, respectively, for segmented images. Data augmentation improved performance by 2.7%–4%.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109666"},"PeriodicalIF":7.5,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142720760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Chimney detection and size estimation from high-resolution optical satellite imagery using deep learning models 利用深度学习模型从高分辨率光学卫星图像中检测和估算烟囱大小

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-17 DOI: 10.1016/j.engappai.2024.109686

Che-Won Park , Hyung-Sup Jung , Won-Jin Lee , Kwang-Jae Lee , Kwan-Young Oh , Joong-Sun Won

This study shows an efficient method to estimate the location and size of chimneys from high-resolution satellite optical images using deep learning models. Korean multi-purpose satellite (KOMPSAT) −3 and -3A satellite images with spatial resolutions of 0.7 m and 0.55 m were used for model performance estimation, and the You Only Look Once version 8 (YOLOv8) and Residual Network (ResNet) regression models were integrated for the detection and size estimation of the chimneys. In the chimney detection and size estimation, we compared the model performances between 1) imbalanced and balanced data, 2) South Korea and Thailand data, and 3) KOMPSAT-3 and -3A data. We also analyzed the model performance according to the ResNet convolutional layers in chimney size estimation. In chimney detection, the model performances between the imbalanced and balanced data, South Korea and Thailand data, and KOMPSAT-3 and -3A data were about 0.723 and 0.739, 0.674 and 0.805, and 0.702 and 0.786 in the average precision (AP) 50–95 measure, respectively. The model performance between the South Korea and Thailand data showed a significant difference, likely because the chimneys in South Korea are very diverse, making it harder to generalize the YOLOv8 model. Furthermore, the model root mean square errors (RMSE) between the imbalanced and balanced data, South Korea and Thailand data, and KOMPSAT-3 and -3A data were about 2.917 and 2.788, 2.690 and 2.951, and 2.913 and 2.580 in chimney height, respectively, and about 1.285 and 1.190, 1.228 and 1.120, and 1.291 and 1.013 in chimney diameter, respectively. Keywords: Chimneys; deep learning; You Only Look Once version 8; Residual Network; Korean Multi-purpose Satellite-3/3A; object detection; regression model.

本研究展示了一种利用深度学习模型从高分辨率卫星光学图像中估算烟囱位置和大小的有效方法。我们使用空间分辨率分别为 0.7 m 和 0.55 m 的韩国多用途卫星（KOMPSAT）-3 和-3A 卫星图像进行了模型性能评估，并整合了 You Only Look Once version 8（YOLOv8）和 Residual Network（ResNet）回归模型，用于烟囱的检测和尺寸估算。在烟囱检测和大小估计中，我们比较了 1）不平衡数据和平衡数据；2）韩国和泰国数据；3）KOMPSAT-3 和 -3A 数据的模型性能。我们还分析了 ResNet 卷积层在烟囱大小估计中的模型性能。在烟囱检测中，不平衡数据和平衡数据、韩国和泰国数据以及 KOMPSAT-3 和 -3A 数据之间的模型性能在平均精度（AP）50-95 测量中分别约为 0.723 和 0.739，0.674 和 0.805，以及 0.702 和 0.786。韩国和泰国数据之间的模型性能存在显著差异，这可能是因为韩国的烟囱种类繁多，使得 YOLOv8 模型难以推广。此外，不平衡和平衡数据、韩国和泰国数据以及 KOMPSAT-3 和 -3A 数据之间的模型均方根误差（RMSE）在烟囱高度上分别约为 2.917 和 2.788、2.690 和 2.951、2.913 和 2.580，在烟囱直径上分别约为 1.285 和 1.190、1.228 和 1.120、1.291 和 1.013。关键词烟囱；深度学习；You Only Look Once version 8；残差网络；韩国多用途卫星-3/3A；物体检测；回归模型。

{"title":"Chimney detection and size estimation from high-resolution optical satellite imagery using deep learning models","authors":"Che-Won Park , Hyung-Sup Jung , Won-Jin Lee , Kwang-Jae Lee , Kwan-Young Oh , Joong-Sun Won","doi":"10.1016/j.engappai.2024.109686","DOIUrl":"10.1016/j.engappai.2024.109686","url":null,"abstract":"<div><div>This study shows an efficient method to estimate the location and size of chimneys from high-resolution satellite optical images using deep learning models. Korean multi-purpose satellite (KOMPSAT) −3 and -3A satellite images with spatial resolutions of 0.7 m and 0.55 m were used for model performance estimation, and the You Only Look Once version 8 (YOLOv8) and Residual Network (ResNet) regression models were integrated for the detection and size estimation of the chimneys. In the chimney detection and size estimation, we compared the model performances between 1) imbalanced and balanced data, 2) South Korea and Thailand data, and 3) KOMPSAT-3 and -3A data. We also analyzed the model performance according to the ResNet convolutional layers in chimney size estimation. In chimney detection, the model performances between the imbalanced and balanced data, South Korea and Thailand data, and KOMPSAT-3 and -3A data were about 0.723 and 0.739, 0.674 and 0.805, and 0.702 and 0.786 in the average precision (AP) 50–95 measure, respectively. The model performance between the South Korea and Thailand data showed a significant difference, likely because the chimneys in South Korea are very diverse, making it harder to generalize the YOLOv8 model. Furthermore, the model root mean square errors (RMSE) between the imbalanced and balanced data, South Korea and Thailand data, and KOMPSAT-3 and -3A data were about 2.917 and 2.788, 2.690 and 2.951, and 2.913 and 2.580 in chimney height, respectively, and about 1.285 and 1.190, 1.228 and 1.120, and 1.291 and 1.013 in chimney diameter, respectively. Keywords: Chimneys; deep learning; You Only Look Once version 8; Residual Network; Korean Multi-purpose Satellite-3/3A; object detection; regression model.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109686"},"PeriodicalIF":7.5,"publicationDate":"2024-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142658986","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fault diagnosis in electric machines and propellers for electrical propulsion aircraft: A review 用于电力推进飞机的电机和螺旋桨的故障诊断：综述

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109577

Leonardo Duarte Milfont , Gabriela Torllone de Carvalho Ferreira , Mateus Giesbrecht

The present work aims to conduct an extensive literature review on the fault diagnosis and classification in electric machines, especially those with permanent magnets, for aeronautical propulsion applications. The main contribution of this research is to assess how intelligent systems focused on fault detection and diagnosis in electric propulsion systems have evolved over the past five years, what are the main types of algorithms used, and how the rapid advancement of machine learning techniques has impacted this research area. Initially, an introduction to the main diagnostic methods is provided, including techniques based on mathematical models, signal analysis, as well as the use of machine learning and deep learning. Subsequently, a detailed study of the main references found in recent years for each type of fault, whether electrical, magnetic, or mechanical, is undertaken. Regarding aeronautical applications, a study of faults in rotating blades and on coupling systems between an electric motor and a set of propellers is conducted. Throughout the text, some of the main datasets found during the research are presented. These datasets include characteristics of healthy operation and fault of windings, bearings, as well as other mechanical components that can be connected to the machine’s shaft, such as gearboxes. Finally, some statistics from this research are presented showing results regarding the annual distribution of publication of all reviewed references, the proportion of faults addressed in all articles, as well as a detailed analysis of the proportion in which each type of algorithm appears in the cited references.

本研究旨在对用于航空推进应用的电机，特别是带永磁体的电机的故障诊断和分类进行广泛的文献综述。本研究的主要贡献在于评估过去五年来侧重于电力推进系统故障检测和诊断的智能系统是如何发展的，使用的主要算法类型是什么，以及机器学习技术的快速发展对这一研究领域产生了怎样的影响。首先，介绍了主要的诊断方法，包括基于数学模型、信号分析以及机器学习和深度学习的技术。随后，详细研究了近年来针对各类故障（无论是电气故障、磁性故障还是机械故障）发现的主要参考文献。在航空应用方面，对旋转叶片中的故障以及电机和一组螺旋桨之间的耦合系统进行了研究。全文介绍了研究过程中发现的一些主要数据集。这些数据集包括绕组、轴承以及与机器轴相连的其他机械部件（如齿轮箱）的健康运行和故障特征。最后，本研究还提供了一些统计数据，显示了所有被引用参考文献的年度出版分布情况、所有文章中涉及故障的比例，以及每种算法在被引用参考文献中所占比例的详细分析。

{"title":"Fault diagnosis in electric machines and propellers for electrical propulsion aircraft: A review","authors":"Leonardo Duarte Milfont , Gabriela Torllone de Carvalho Ferreira , Mateus Giesbrecht","doi":"10.1016/j.engappai.2024.109577","DOIUrl":"10.1016/j.engappai.2024.109577","url":null,"abstract":"<div><div>The present work aims to conduct an extensive literature review on the fault diagnosis and classification in electric machines, especially those with permanent magnets, for aeronautical propulsion applications. The main contribution of this research is to assess how intelligent systems focused on fault detection and diagnosis in electric propulsion systems have evolved over the past five years, what are the main types of algorithms used, and how the rapid advancement of machine learning techniques has impacted this research area. Initially, an introduction to the main diagnostic methods is provided, including techniques based on mathematical models, signal analysis, as well as the use of machine learning and deep learning. Subsequently, a detailed study of the main references found in recent years for each type of fault, whether electrical, magnetic, or mechanical, is undertaken. Regarding aeronautical applications, a study of faults in rotating blades and on coupling systems between an electric motor and a set of propellers is conducted. Throughout the text, some of the main datasets found during the research are presented. These datasets include characteristics of healthy operation and fault of windings, bearings, as well as other mechanical components that can be connected to the machine’s shaft, such as gearboxes. Finally, some statistics from this research are presented showing results regarding the annual distribution of publication of all reviewed references, the proportion of faults addressed in all articles, as well as a detailed analysis of the proportion in which each type of algorithm appears in the cited references.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109577"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Cooperative task assignment of heterogeneous unmanned aerial vehicles for simultaneous multi-directional attack on a moving target 异构无人飞行器合作分配任务，同时多方位攻击移动目标

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109595

Sami Shahid , Ziyang Zhen , Umair Javaid

Multiple unmanned aerial vehicle (UAV) attacks, on moving targets with directional priorities, need careful task allocation especially when the UAVs have variable attacking power. In addition, the constraints of simultaneous attack from different directions multi-folds the complexity of the problem. In this work, an extended contract net protocol (ECNP) based autonomous and cooperative task assignment method is proposed to deal with the time-sensitive position assignment for multi-directional attack and uniform resource allocation. Initially, an optimization problem is formulated using distances between each UAV and expected attack positions (APs), arrival time, and attack power. In addition, for uniform resource allocation, a variable is introduced to monitor available resources at a given time. An agent-based model is built with UAV information, such as position and speed, along with the direction of the high-value moving target (HVMT). Each UAV identifies the possible arrival points based on its speed constraints, and current position, and the position and velocity of HVMT. Moreover, distance and expected arrival time to all APs are computed. Finally, the agents make attack point allocations using the proposed ECNP after making a consensus about simultaneous attack time. The proposed method ensures uniform resource allocation. The simulation results show the superiority of the proposed method (ECNP) in comparison with classical contract net protocol (CNP) and Genetic Algorithms (GA) in terms of uniform resource allocation and mission accomplishment.

多架无人飞行器（UAV）攻击具有方向优先权的移动目标时，需要谨慎分配任务，尤其是当无人飞行器的攻击力不稳定时。此外，从不同方向同时发起攻击的约束条件也使问题的复杂性成倍增加。本研究提出了一种基于扩展契约网协议（ECNP）的自主合作任务分配方法，用于处理多方向攻击的时间敏感位置分配和统一资源分配问题。首先，利用每架无人机与预期攻击位置（AP）之间的距离、到达时间和攻击功率，提出一个优化问题。此外，在统一资源分配方面，还引入了一个变量来监控给定时间内的可用资源。利用无人机的位置和速度等信息以及高价值移动目标（HVMT）的方向，建立了一个基于代理的模型。每个无人机根据其速度限制、当前位置以及高价值移动目标（HVMT）的位置和速度，确定可能的到达点。此外，还要计算到所有 AP 的距离和预计到达时间。最后，代理在就同时攻击时间达成共识后，使用提议的 ECNP 进行攻击点分配。建议的方法确保了资源的统一分配。仿真结果表明，与经典的契约网协议（CNP）和遗传算法（GA）相比，建议的方法（ECNP）在统一资源分配和完成任务方面更具优势。

{"title":"Cooperative task assignment of heterogeneous unmanned aerial vehicles for simultaneous multi-directional attack on a moving target","authors":"Sami Shahid , Ziyang Zhen , Umair Javaid","doi":"10.1016/j.engappai.2024.109595","DOIUrl":"10.1016/j.engappai.2024.109595","url":null,"abstract":"<div><div>Multiple unmanned aerial vehicle (UAV) attacks, on moving targets with directional priorities, need careful task allocation especially when the UAVs have variable attacking power. In addition, the constraints of simultaneous attack from different directions multi-folds the complexity of the problem. In this work, an extended contract net protocol (ECNP) based autonomous and cooperative task assignment method is proposed to deal with the time-sensitive position assignment for multi-directional attack and uniform resource allocation. Initially, an optimization problem is formulated using distances between each UAV and expected attack positions (APs), arrival time, and attack power. In addition, for uniform resource allocation, a variable is introduced to monitor available resources at a given time. An agent-based model is built with UAV information, such as position and speed, along with the direction of the high-value moving target (HVMT). Each UAV identifies the possible arrival points based on its speed constraints, and current position, and the position and velocity of HVMT. Moreover, distance and expected arrival time to all APs are computed. Finally, the agents make attack point allocations using the proposed ECNP after making a consensus about simultaneous attack time. The proposed method ensures uniform resource allocation. The simulation results show the superiority of the proposed method (ECNP) in comparison with classical contract net protocol (CNP) and Genetic Algorithms (GA) in terms of uniform resource allocation and mission accomplishment.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109595"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659103","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Predicting rapid impact compaction of soil using a parallel transformer and long short-term memory architecture for sequential soil profile encoding 利用并行变压器和长短期存储器架构进行土壤剖面顺序编码，预测土壤的快速冲击压实效果

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109664

Sompote Youwai, Sirasak Detcheewa

This study presents an advanced deep learning approach for predicting the effectiveness of Rapid Impact Compaction (RIC). The model integrates the focused attention mechanisms of transformer architectures with the sequential data processing capabilities of Long Short-Term Memory (LSTM) networks. Input parameters include the initial soil profile and feature vectors representing the soil's initial state, applied compaction effort, and compaction hammer energy. Utilizing an encoder-decoder framework, the model encodes soil profile information at various depths into tokens, which are subsequently decoded to predict the resulting ground improvement. An ablation study was conducted to assess the significance of each model component. The model's predictive accuracy was validated using field test data, demonstrating a strong correlation with observed outcomes (mean absolute error of 0.42 for test data). Shapley value analysis of the trained model revealed that compaction effort exerted the highest influence on predictions, followed by fine content and fill thickness. The model architecture also demonstrated successful application to alternative RIC case studies, indicating potential generalizability. Furthermore, the model's capability to simulate hypothetical scenarios with varying compaction efforts provides valuable insights for strategic planning and optimization of RIC project designs.

本研究提出了一种先进的深度学习方法，用于预测快速冲击压实（RIC）的有效性。该模型集成了变压器架构的集中注意力机制和长短期记忆（LSTM）网络的顺序数据处理能力。输入参数包括初始土壤剖面、代表土壤初始状态的特征向量、施加的压实力和压实锤能量。该模型利用编码器-解码器框架，将不同深度的土壤剖面信息编码成标记，随后对标记进行解码，以预测由此产生的地面改良效果。为评估模型各组成部分的重要性，进行了一项消融研究。该模型的预测准确性通过实地测试数据进行了验证，结果显示与观测结果有很强的相关性（测试数据的平均绝对误差为 0.42）。对训练有素的模型进行的 Shapley 值分析表明，压实力度对预测的影响最大，其次是细粒含量和填土厚度。该模型结构还成功应用于其他路面信息中心案例研究，表明其具有潜在的通用性。此外，该模型还能模拟不同压实力度的假定情况，为 RIC 项目设计的战略规划和优化提供了有价值的见解。

{"title":"Predicting rapid impact compaction of soil using a parallel transformer and long short-term memory architecture for sequential soil profile encoding","authors":"Sompote Youwai, Sirasak Detcheewa","doi":"10.1016/j.engappai.2024.109664","DOIUrl":"10.1016/j.engappai.2024.109664","url":null,"abstract":"<div><div>This study presents an advanced deep learning approach for predicting the effectiveness of Rapid Impact Compaction (RIC). The model integrates the focused attention mechanisms of transformer architectures with the sequential data processing capabilities of Long Short-Term Memory (LSTM) networks. Input parameters include the initial soil profile and feature vectors representing the soil's initial state, applied compaction effort, and compaction hammer energy. Utilizing an encoder-decoder framework, the model encodes soil profile information at various depths into tokens, which are subsequently decoded to predict the resulting ground improvement. An ablation study was conducted to assess the significance of each model component. The model's predictive accuracy was validated using field test data, demonstrating a strong correlation with observed outcomes (mean absolute error of 0.42 for test data). Shapley value analysis of the trained model revealed that compaction effort exerted the highest influence on predictions, followed by fine content and fill thickness. The model architecture also demonstrated successful application to alternative RIC case studies, indicating potential generalizability. Furthermore, the model's capability to simulate hypothetical scenarios with varying compaction efforts provides valuable insights for strategic planning and optimization of RIC project designs.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109664"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning discriminative representations by a Canonical Correlation Analysis-based Siamese Network for offline signature verification 通过基于佳能相关分析的连体网络学习鉴别表征，用于离线签名验证

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109640

Lidong Zheng , Xingbiao Zhao , Shengjie Xu, Yuanyuan Ren, Yuchen Zheng

In offline signature verification tasks, capturing different writing behaviors between genuine and forged signatures is a crucial and challenging step. In this paper, a novel writer independent Canonical Correlation Analysis-based Siamese Network (CCASigNet) is proposed to learn discriminative representations between different signature pairs. Specifically, we first construct signature pairs with three types: genuine-genuine, genuine-forged, and forged-forged. Then, different signature pairs are fed into CCASigNet for training with the Canonical Correlation Analysis (CCA) and classification-based losses. After network training, we extract the feature of signatures by CCASigNet and use writer-dependent classifiers to construct a comprehensive verification system. Extensive experiments on four benchmark signature datasets demonstrate that the proposed CCASigNet learns discriminative representations between different signature pairs and achieves state-of-the-art or competitive performance compared with advanced verification systems. In addition, the proposed CCASigNet has good generalization ability and is easy to transfer to different datasets with different language scripts within the realm of offline signature verification tasks.

在离线签名验证任务中，捕捉真实签名和伪造签名之间的不同书写行为是至关重要且极具挑战性的一步。本文提出了一种新颖的独立于书写者的基于佳能相关分析的连体网络（CCASigNet），用于学习不同签名对之间的鉴别表征。具体来说，我们首先构建了三种类型的签名对：真-伪、真-伪、伪-伪。然后，将不同的签名对输入 CCASigNet，利用典型相关分析（CCA）和基于分类的损失进行训练。网络训练完成后，我们通过 CCASigNet 提取签名特征，并使用依赖于作者的分类器来构建一个全面的验证系统。在四个基准签名数据集上进行的广泛实验表明，所提出的 CCASigNet 可以学习不同签名对之间的判别表征，与先进的验证系统相比，其性能达到一流水平或具有竞争力。此外，所提出的 CCASigNet 还具有良好的泛化能力，可轻松应用于离线签名验证任务中不同语言脚本的不同数据集。

{"title":"Learning discriminative representations by a Canonical Correlation Analysis-based Siamese Network for offline signature verification","authors":"Lidong Zheng , Xingbiao Zhao , Shengjie Xu, Yuanyuan Ren, Yuchen Zheng","doi":"10.1016/j.engappai.2024.109640","DOIUrl":"10.1016/j.engappai.2024.109640","url":null,"abstract":"<div><div>In offline signature verification tasks, capturing different writing behaviors between genuine and forged signatures is a crucial and challenging step. In this paper, a novel writer independent Canonical Correlation Analysis-based Siamese Network (CCASigNet) is proposed to learn discriminative representations between different signature pairs. Specifically, we first construct signature pairs with three types: genuine-genuine, genuine-forged, and forged-forged. Then, different signature pairs are fed into CCASigNet for training with the Canonical Correlation Analysis (CCA) and classification-based losses. After network training, we extract the feature of signatures by CCASigNet and use writer-dependent classifiers to construct a comprehensive verification system. Extensive experiments on four benchmark signature datasets demonstrate that the proposed CCASigNet learns discriminative representations between different signature pairs and achieves state-of-the-art or competitive performance compared with advanced verification systems. In addition, the proposed CCASigNet has good generalization ability and is easy to transfer to different datasets with different language scripts within the realm of offline signature verification tasks.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109640"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A novel hybrid data-driven domain generalization approach with dual-perspective feature fusion for intelligent fault diagnosis 用于智能故障诊断的双视角特征融合的新型混合数据驱动领域泛化方法

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109614

Lanjun Wan , Jian Zhou , Jiaen Ning , Yuanyuan Li , Changyun Li

Domain generalization-based fault diagnosis (DGFD) approaches do not require access to the target domain during model training, but they usually rely on numerous labeled source domain data. However, only few labeled source domain data can be obtained in actual diagnosis scenarios. Therefore, a novel hybrid data-driven domain generalization (DG) approach with dual-perspective feature fusion for intelligent fault diagnosis (FD) is proposed. Firstly, to solve the problem of scarce training samples in the source domains, the rolling bearing (RB) and the gear simulated vibration models are established to generate numerous labeled simulated vibration data, and the improved auxiliary classifier generative adversarial network (ACGAN) is used to effectively balance the simulated and real data. Secondly, a simulated and real data-driven DG network that fuses intra-domain invariant features and mutually-invariant features between domains (SRDGN-IM) is proposed, where the intra-domain invariant features are learned through distillation idea and the mutually-invariant features are learned through adversarial training, which can make the diagnosis model better learn the key generalization features from source domains to obtain more accurate diagnosis results. Finally, a series of DG experiments are conducted on the gearbox and bearing datasets, and the average FD accuracies of the proposed approach reach 87.45% and 89.10% respectively under different DG tasks.

基于领域泛化的故障诊断（DGFD）方法在模型训练过程中不需要访问目标领域，但通常依赖于大量标记的源领域数据。然而，在实际诊断场景中，只能获得很少的标注源域数据。因此，本文提出了一种新颖的混合数据驱动领域泛化（DG）方法和双视角特征融合方法，用于智能故障诊断（FD）。首先，为解决源域训练样本稀缺的问题，建立了滚动轴承（RB）和齿轮模拟振动模型，以生成大量带标签的模拟振动数据，并使用改进的辅助分类器生成对抗网络（ACGAN）来有效平衡模拟数据和真实数据。其次，提出了一种仿真和真实数据驱动的融合域内不变特征和域间互变特征的 DG 网络（SRDGN-IM），其中域内不变特征通过蒸馏思想学习，互变特征通过对抗训练学习，可以使诊断模型更好地学习源域的关键泛化特征，从而获得更准确的诊断结果。最后，在齿轮箱和轴承数据集上进行了一系列 DG 实验，在不同的 DG 任务下，所提出方法的平均 FD 准确率分别达到了 87.45% 和 89.10%。

{"title":"A novel hybrid data-driven domain generalization approach with dual-perspective feature fusion for intelligent fault diagnosis","authors":"Lanjun Wan , Jian Zhou , Jiaen Ning , Yuanyuan Li , Changyun Li","doi":"10.1016/j.engappai.2024.109614","DOIUrl":"10.1016/j.engappai.2024.109614","url":null,"abstract":"<div><div>Domain generalization-based fault diagnosis (DGFD) approaches do not require access to the target domain during model training, but they usually rely on numerous labeled source domain data. However, only few labeled source domain data can be obtained in actual diagnosis scenarios. Therefore, a novel hybrid data-driven domain generalization (DG) approach with dual-perspective feature fusion for intelligent fault diagnosis (FD) is proposed. Firstly, to solve the problem of scarce training samples in the source domains, the rolling bearing (RB) and the gear simulated vibration models are established to generate numerous labeled simulated vibration data, and the improved auxiliary classifier generative adversarial network (ACGAN) is used to effectively balance the simulated and real data. Secondly, a simulated and real data-driven DG network that fuses intra-domain invariant features and mutually-invariant features between domains (SRDGN-IM) is proposed, where the intra-domain invariant features are learned through distillation idea and the mutually-invariant features are learned through adversarial training, which can make the diagnosis model better learn the key generalization features from source domains to obtain more accurate diagnosis results. Finally, a series of DG experiments are conducted on the gearbox and bearing datasets, and the average FD accuracies of the proposed approach reach 87.45% and 89.10% respectively under different DG tasks.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109614"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Decoding text from electroencephalography signals: A novel Hierarchical Gated Recurrent Unit with Masked Residual Attention Mechanism 从脑电图信号解码文本：具有屏蔽残留注意力机制的新型分层门控递归单元

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109615

Qiupu Chen , Yimou Wang , Fenmei Wang , Duolin Sun , Qiankun Li

Progress in both neuroscience and natural language processing has opened doors for investigating brain to text techniques to reconstruct what individuals see, perceive, or focus on from human brain activity patterns. Non-invasive decoding, utilizing electroencephalography (EEG) signals, is preferred due to its comfort, cost-effectiveness, and portability. In brain-to-text applications, a pressing need has arisen to develop effective models that can accurately capture the intricate details of EEG signals, such as global and local contextual information and long-term dependencies. In response to this need, we propose the Hierarchical Gated Recurrent Unit with Masked Residual Attention Mechanism (HGRU-MRAM) model, which ingeniously combines the hierarchical structure and the masked residual attention mechanism to deliver a robust brain-to-text decoding system. Our experimental results on the ZuCo dataset demonstrate that this model significantly outperforms existing baselines, achieving state-of-the-art performance with Bilingual Evaluation Understudy Score (BLEU), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), US National Institute of Standards and Technology Metric (NIST), Metric for Evaluation of Translation with Explicit Ordering (METEOR), Translation Edit Rate (TER), and BiLingual Evaluation Understudy with Representations from Transformers (BLEURT) scores of 48.29, 34.84, 4.07, 34.57, 21.98, and 40.45, respectively. The code is available at https://github.com/qpuchen/EEG-To-Sentence.

神经科学和自然语言处理技术的进步为研究从大脑到文本的技术打开了大门，这些技术可以从人脑活动模式中重建个人看到、感知或关注的内容。利用脑电图（EEG）信号的非侵入式解码因其舒适性、成本效益和便携性而受到青睐。在 "脑到文本 "应用中，迫切需要开发有效的模型，以准确捕捉脑电信号的复杂细节，如全局和局部上下文信息以及长期依赖关系。针对这一需求，我们提出了分层门控递归单元与掩蔽残差注意机制（HGRU-MRAM）模型，该模型巧妙地将分层结构和掩蔽残差注意机制结合在一起，从而提供了一个稳健的脑到文本解码系统。我们在 ZuCo 数据集上的实验结果表明，该模型的性能明显优于现有基线，在双语评估研究得分（BLEU）、以召回为导向的术语评估研究（ROUGE）、美国国家标准与技术研究院度量标准（NIST）、显式排序翻译评估度量标准（METEOR）、翻译编辑率（TER）和具有转换器表征的双语评估研究（BLEURT）得分方面达到了最先进的水平。29、34.84、4.07、34.57、21.98 和 40.45 分。代码见 https://github.com/qpuchen/EEG-To-Sentence。

{"title":"Decoding text from electroencephalography signals: A novel Hierarchical Gated Recurrent Unit with Masked Residual Attention Mechanism","authors":"Qiupu Chen , Yimou Wang , Fenmei Wang , Duolin Sun , Qiankun Li","doi":"10.1016/j.engappai.2024.109615","DOIUrl":"10.1016/j.engappai.2024.109615","url":null,"abstract":"<div><div>Progress in both neuroscience and natural language processing has opened doors for investigating brain to text techniques to reconstruct what individuals see, perceive, or focus on from human brain activity patterns. Non-invasive decoding, utilizing electroencephalography (EEG) signals, is preferred due to its comfort, cost-effectiveness, and portability. In brain-to-text applications, a pressing need has arisen to develop effective models that can accurately capture the intricate details of EEG signals, such as global and local contextual information and long-term dependencies. In response to this need, we propose the Hierarchical Gated Recurrent Unit with Masked Residual Attention Mechanism (HGRU-MRAM) model, which ingeniously combines the hierarchical structure and the masked residual attention mechanism to deliver a robust brain-to-text decoding system. Our experimental results on the ZuCo dataset demonstrate that this model significantly outperforms existing baselines, achieving state-of-the-art performance with Bilingual Evaluation Understudy Score (BLEU), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), US National Institute of Standards and Technology Metric (NIST), Metric for Evaluation of Translation with Explicit Ordering (METEOR), Translation Edit Rate (TER), and BiLingual Evaluation Understudy with Representations from Transformers (BLEURT) scores of 48.29, 34.84, 4.07, 34.57, 21.98, and 40.45, respectively. The code is available at <span><span>https://github.com/qpuchen/EEG-To-Sentence</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109615"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Chin electromyography-based motor unit decomposition for alternative screening of obstructive sleep apnea events: A comprehensive analysis 基于下巴肌电图的运动单元分解，用于阻塞性睡眠呼吸暂停事件的替代筛查：综合分析

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-16 DOI: 10.1016/j.engappai.2024.109534

Adil Rehman , Mostafa Moussa , Hani Saleh , Ali Khraibi , Ahsan H. Khandoker

Obstructive Sleep Apnea-Hypopnea Syndrome (OSAHS) is a prevalent sleep disorder characterized by recurrent episodes of obstructed breathing due to the relaxation of muscles in the upper airway during sleep, often linked with neuromuscular and cardiovascular disorders. This study introduces a novel method using traditional machine learning classifiers and surface electromyography (SEMG) features extracted from motor units (MUs) decomposed from chin electromyography (EMG) signals to screen for OSA events in OSAHS subjects. SEMG features were extracted from individual MUs decomposed from chin EMG segments using a novel dataset. An apnea detection algorithm was designed to label these events for OSAHS subjects across sleep stages. Analysis of motor neuron firing patterns in OSAHS subjects revealed lower activation during OSA events and higher activation during non-OSA segments. Additionally, we evaluated the proposed system on a publicly available dataset, achieving a maximum accuracy of 72% for OSAHS subjects in the midlife phase age group (40–59 years) and 72.5% for subjects in the severe phase of OSAHS using Support Vector Machines (SVM). The random forest (RF) classifier demonstrated robust performance, achieving 97% accuracy, 93.2% sensitivity, 100% specificity, 100% precision, a 96.48% F1-score, and an area under the curve (AUC) of 0.996. This system facilitates early differentiation between OSA and non-OSA events, enabling timely intervention in the mild apnea phase to prevent progression to severe OSAHS. Moreover, it offers a convenient alternative to conventional polysomnography (PSG), enhancing diagnostic accessibility and clinical management.

阻塞性睡眠呼吸暂停-低通气综合征（OSAHS）是一种普遍存在的睡眠障碍，其特点是由于睡眠时上气道肌肉松弛而导致反复发作的呼吸受阻，通常与神经肌肉和心血管疾病有关。本研究采用传统的机器学习分类器和从下巴肌电图（EMG）信号分解的运动单元（MU）中提取的表面肌电图（SEMG）特征，介绍了一种新方法，用于筛查 OSAHS 受试者的 OSA 事件。利用新颖的数据集，从下巴肌电图片段分解出的单个运动单元中提取 SEMG 特征。设计了一种呼吸暂停检测算法，用于标记 OSAHS 受试者在不同睡眠阶段的呼吸暂停事件。对 OSAHS 受试者运动神经元发射模式的分析表明，在 OSA 事件中激活较低，而在非 OSA 片段中激活较高。此外，我们还利用支持向量机（SVM）在一个公开的数据集上对所提出的系统进行了评估，结果表明，对于中年阶段年龄组（40-59 岁）的 OSAHS 受试者，该系统的最高准确率为 72%，而对于 OSAHS 严重阶段的受试者，该系统的准确率为 72.5%。随机森林（RF）分类器表现强劲，准确率达到 97%，灵敏度达到 93.2%，特异性达到 100%，精确度达到 100%，F1 分数达到 96.48%，曲线下面积（AUC）达到 0.996。该系统有助于早期区分 OSA 和非 OSA 事件，从而在轻度呼吸暂停阶段进行及时干预，防止恶化为严重的 OSAHS。此外，它还为传统的多导睡眠图（PSG）提供了一种便捷的替代方法，提高了诊断的可及性和临床管理水平。

{"title":"Chin electromyography-based motor unit decomposition for alternative screening of obstructive sleep apnea events: A comprehensive analysis","authors":"Adil Rehman , Mostafa Moussa , Hani Saleh , Ali Khraibi , Ahsan H. Khandoker","doi":"10.1016/j.engappai.2024.109534","DOIUrl":"10.1016/j.engappai.2024.109534","url":null,"abstract":"<div><div>Obstructive Sleep Apnea-Hypopnea Syndrome (OSAHS) is a prevalent sleep disorder characterized by recurrent episodes of obstructed breathing due to the relaxation of muscles in the upper airway during sleep, often linked with neuromuscular and cardiovascular disorders. This study introduces a novel method using traditional machine learning classifiers and surface electromyography (SEMG) features extracted from motor units (MUs) decomposed from chin electromyography (EMG) signals to screen for OSA events in OSAHS subjects. SEMG features were extracted from individual MUs decomposed from chin EMG segments using a novel dataset. An apnea detection algorithm was designed to label these events for OSAHS subjects across sleep stages. Analysis of motor neuron firing patterns in OSAHS subjects revealed lower activation during OSA events and higher activation during non-OSA segments. Additionally, we evaluated the proposed system on a publicly available dataset, achieving a maximum accuracy of 72% for OSAHS subjects in the midlife phase age group (40–59 years) and 72.5% for subjects in the severe phase of OSAHS using Support Vector Machines (SVM). The random forest (RF) classifier demonstrated robust performance, achieving 97% accuracy, 93.2% sensitivity, 100% specificity, 100% precision, a 96.48% F1-score, and an area under the curve (AUC) of 0.996. This system facilitates early differentiation between OSA and non-OSA events, enabling timely intervention in the mild apnea phase to prevent progression to severe OSAHS. Moreover, it offers a convenient alternative to conventional polysomnography (PSG), enhancing diagnostic accessibility and clinical management.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109534"},"PeriodicalIF":7.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hybrid multi-attention transformer for robust video object detection 用于稳健视频对象检测的混合多注意变换器

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-11-15 DOI: 10.1016/j.engappai.2024.109606

Sathishkumar Moorthy , Sachin Sakthi K.S. , Sathiyamoorthi Arthanari , Jae Hoon Jeong , Young Hoon Joo

Video object detection (VOD) is the task of detecting objects in videos, a challenge due to the changing appearance of objects over time, leading to potential detection errors. Recent research has addressed this by aggregating features from neighboring frames and incorporating information from distant frames to mitigate appearance deterioration. However, relying solely on object candidate regions in distant frames, independent of object position, has limitations, as it depends heavily on the performance of these regions and struggles with deteriorated appearances. To overcome these challenges, we propose a novel Hybrid Multi-Attention Transformer (HyMAT) module as our main contribution. HyMAT enhances relevant correlations while suppressing flawed information by searching for an agreement between whole correlation vectors. This module is designed for flexibility and can be integrated into both self- and cross-attention blocks to significantly improve detection accuracy. Additionally, we introduce a simplified Transformer-based object detection framework, named Hybrid Multi-Attention Object Detection (HyMATOD), which leverages competent feature reprocessing and target-background embeddings to more effectively utilize temporal references. Our approach demonstrates state-of-the-art performance, as evaluated on the ImageNet video object detection benchmark (ImageNet VID) and the University at Albany DEtection and TRACking (UA-DETRAC) benchmarks. Specifically, our HyMATOD model achieves an impressive 86.7% mean Average Precision (mAP) on the ImageNet VID dataset, establishing its superiority and practicality for video object detection tasks. These results underscore the significance of our contributions to advancing the field of VOD.

视频物体检测（VOD）是一项检测视频中物体的任务，由于物体的外观会随着时间的推移而发生变化，从而导致潜在的检测错误。最近的研究已经解决了这一问题，方法是聚合邻近帧的特征并结合远处帧的信息，以减轻外观劣化。但是，仅仅依靠远处帧中的候选物体区域（与物体位置无关）有其局限性，因为它严重依赖于这些区域的性能，并且在外观劣化的情况下也很难发挥作用。为了克服这些挑战，我们提出了一种新颖的混合多注意力转换器（HyMAT）模块，作为我们的主要贡献。HyMAT 通过寻找整个相关向量之间的一致性，在增强相关性的同时抑制有缺陷的信息。该模块设计灵活，可集成到自关注和交叉关注模块中，从而显著提高检测精度。此外，我们还引入了一个基于变换器的简化对象检测框架，名为混合多注意对象检测（HyMATOD），该框架利用胜任的特征再处理和目标-背景嵌入来更有效地利用时间参考。通过对 ImageNet 视频对象检测基准（ImageNet VID）和奥尔巴尼大学 DEtection and TRACking（UA-DETRAC）基准的评估，我们的方法展示了最先进的性能。具体来说，我们的 HyMATOD 模型在 ImageNet VID 数据集上达到了令人印象深刻的 86.7% 平均精度 (mAP)，从而确立了其在视频对象检测任务中的优越性和实用性。这些结果凸显了我们在推进 VOD 领域所做贡献的重要性。

{"title":"Hybrid multi-attention transformer for robust video object detection","authors":"Sathishkumar Moorthy , Sachin Sakthi K.S. , Sathiyamoorthi Arthanari , Jae Hoon Jeong , Young Hoon Joo","doi":"10.1016/j.engappai.2024.109606","DOIUrl":"10.1016/j.engappai.2024.109606","url":null,"abstract":"<div><div>Video object detection (VOD) is the task of detecting objects in videos, a challenge due to the changing appearance of objects over time, leading to potential detection errors. Recent research has addressed this by aggregating features from neighboring frames and incorporating information from distant frames to mitigate appearance deterioration. However, relying solely on object candidate regions in distant frames, independent of object position, has limitations, as it depends heavily on the performance of these regions and struggles with deteriorated appearances. To overcome these challenges, we propose a novel Hybrid Multi-Attention Transformer (HyMAT) module as our main contribution. HyMAT enhances relevant correlations while suppressing flawed information by searching for an agreement between whole correlation vectors. This module is designed for flexibility and can be integrated into both self- and cross-attention blocks to significantly improve detection accuracy. Additionally, we introduce a simplified Transformer-based object detection framework, named Hybrid Multi-Attention Object Detection (HyMATOD), which leverages competent feature reprocessing and target-background embeddings to more effectively utilize temporal references. Our approach demonstrates state-of-the-art performance, as evaluated on the ImageNet video object detection benchmark (ImageNet VID) and the University at Albany DEtection and TRACking (UA-DETRAC) benchmarks. Specifically, our HyMATOD model achieves an impressive 86.7% mean Average Precision (mAP) on the ImageNet VID dataset, establishing its superiority and practicality for video object detection tasks. These results underscore the significance of our contributions to advancing the field of VOD.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"139 ","pages":"Article 109606"},"PeriodicalIF":7.5,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0