Engineering Applications of Artificial Intelligence最新文献_第9页

Complete coverage planning using Deep Reinforcement Learning for polyiamonds-based reconfigurable robot 利用深度强化学习为基于多钻石的可重构机器人制定完整的覆盖规划

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109424

Anh Vu Le , Dinh Tung Vo , Nguyen Tien Dat , Minh Bui Vu , Mohan Rajesh Elara

Achieving complete coverage in complex areas is a critical objective for tilling tasks such as cleaning, painting, maintenance, and inspection. However, existing robots in the market, with their fixed morphologies, face limitations when it comes to accessing confined spaces. Reconfigurable tiling robots provide a feasible solution to this challenge. By shapeshifting among the available morphologies to adapt to the different conditions of complex environments, these robots can enhance the efficiency of complete coverage. However, the ability to change shape is constrained by energy usage considerations. Hence, it is important to have an optimal strategy to generate a trajectory that covers confined areas with minimal reconfiguration actions while taking into account the finite set of possible shapes. This paper proposes a complete coverage planning (CCP) framework for a reconfigurable tiling robot called hTetrakis, which consists of three polyiamonds blocks. The CCP framework leverages Deep Reinforcement Learning (DRL) to derive an optimal action policy within a polyiamonds shape-based workspace. By maximizing cumulative rewards to optimize the overall kinetic energy-based costweight, the proposed DRL model plans the hTetrakis shapes and its trajectories simultaneously. To this end, the DRL model utilizes Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) network and adopts the Actor–Critic deep reinforcement learning agent with Experience Replay (ACER) approach for off-policy decision-making. By producing trajectories with reduced costs and time, the proposed CCP framework surpasses conventional heuristic optimization methods like Particle Swarm Optimization (PSO), Differential Evolution (DE), Genetic Algorithm (GA) and Ant Colony Optimization (ACO) rely on tiling strategies.

实现复杂区域的完全覆盖是清洁、喷漆、维护和检查等耕作任务的关键目标。然而，市场上现有的机器人形态固定，在进入狭窄空间时受到限制。可重新配置的铲运机器人为这一挑战提供了可行的解决方案。通过在现有形态中变换形态以适应复杂环境的不同条件，这些机器人可以提高全面覆盖的效率。然而，改变形状的能力受到能源使用方面的限制。因此，重要的是要有一个最佳策略，在考虑到有限的可能形态集的同时，以最小的重新配置行动生成覆盖有限区域的轨迹。本文为一种名为 hTetrakis 的可重构平铺机器人提出了一个完整覆盖规划（CCP）框架，该机器人由三个多钻块组成。CCP 框架利用深度强化学习（DRL）在基于多钻石形状的工作空间内推导出最佳行动策略。通过最大化累积奖励来优化基于动能的总体成本权重，拟议的 DRL 模型可同时规划 hTetrakis 形状及其轨迹。为此，DRL 模型利用带有长短期记忆（LSTM）网络的卷积神经网络（CNN），并采用带有经验重放（ACER）的行为批判深度强化学习代理方法进行非政策决策。所提出的 CCP 框架能以更低的成本和更短的时间生成轨迹，超越了传统的启发式优化方法，如粒子群优化 (PSO)、差分进化 (DE)、遗传算法 (GA) 和蚁群优化 (ACO)。

{"title":"Complete coverage planning using Deep Reinforcement Learning for polyiamonds-based reconfigurable robot","authors":"Anh Vu Le , Dinh Tung Vo , Nguyen Tien Dat , Minh Bui Vu , Mohan Rajesh Elara","doi":"10.1016/j.engappai.2024.109424","DOIUrl":"10.1016/j.engappai.2024.109424","url":null,"abstract":"<div><div>Achieving complete coverage in complex areas is a critical objective for tilling tasks such as cleaning, painting, maintenance, and inspection. However, existing robots in the market, with their fixed morphologies, face limitations when it comes to accessing confined spaces. Reconfigurable tiling robots provide a feasible solution to this challenge. By shapeshifting among the available morphologies to adapt to the different conditions of complex environments, these robots can enhance the efficiency of complete coverage. However, the ability to change shape is constrained by energy usage considerations. Hence, it is important to have an optimal strategy to generate a trajectory that covers confined areas with minimal reconfiguration actions while taking into account the finite set of possible shapes. This paper proposes a complete coverage planning (CCP) framework for a reconfigurable tiling robot called hTetrakis, which consists of three polyiamonds blocks. The CCP framework leverages Deep Reinforcement Learning (DRL) to derive an optimal action policy within a polyiamonds shape-based workspace. By maximizing cumulative rewards to optimize the overall kinetic energy-based costweight, the proposed DRL model plans the hTetrakis shapes and its trajectories simultaneously. To this end, the DRL model utilizes Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) network and adopts the Actor–Critic deep reinforcement learning agent with Experience Replay (ACER) approach for off-policy decision-making. By producing trajectories with reduced costs and time, the proposed CCP framework surpasses conventional heuristic optimization methods like Particle Swarm Optimization (PSO), Differential Evolution (DE), Genetic Algorithm (GA) and Ant Colony Optimization (ACO) rely on tiling strategies.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142535246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Resilient dynamic microgrid formation by deep reinforcement learning integrating physics-informed neural networks 通过整合物理信息神经网络的深度强化学习形成弹性动态微电网

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109470

Mingze Xu , Shunbo Lei , Chong Wang , Liang Liang , Junhua Zhao , Chaoyi Peng

Dynamic microgrid formation can enhance topological flexibility within the distribution system, particularly during extreme events, thereby facilitating a more efficient restoration process. However, existing research has overlooked the impact of cold load pickup on system restoration efforts. A sudden load spike can lead to the overloading of generators and transformers, which can result in the failure of the system restoration process. This study leverages the topological flexibility through dynamic microgrid formation of the system to mitigate the impact of cold load pickup, thereby enhancing the efficiency of sequential load restoration. To alleviate the computational complexity arising from intricate operational constraints and the uncertainties inherent in cold load pickup conditions, this paper proposes a novel model-free framework. Unlike existing deep reinforcement learning models, we incorporate physical constraint information into the model by means of physics-informed neural networks, where the solution of an optimization problem is regarded as knowledge, enabling the agent to learn operational constraints more efficiently and stably. The proposed approach is compatible with and can be integrated into any deep reinforcement learning algorithm that utilizes the advantage actor–critic framework with neural networks. This research employs the deep deterministic policy gradient algorithm as a representative example for investigation. The effectiveness and generalization performance of the proposed method are validated on a modified IEEE 123-node test feeder, while its scalability is assessed using the IEEE 8500-node test feeder system.

动态微电网的形成可以提高配电系统的拓扑灵活性，尤其是在极端事件发生时，从而促进更高效的恢复过程。然而，现有研究忽略了冷负荷骤增对系统恢复工作的影响。突然的负荷激增会导致发电机和变压器过载，从而导致系统恢复过程失败。本研究通过系统的动态微电网形成，利用拓扑灵活性来减轻冷负荷拾取的影响，从而提高顺序负荷恢复的效率。为了减轻错综复杂的运行约束和冷负荷恢复条件固有的不确定性所带来的计算复杂性，本文提出了一种新颖的无模型框架。与现有的深度强化学习模型不同，我们通过物理信息神经网络将物理约束信息纳入模型，将优化问题的解视为知识，使代理能够更高效、更稳定地学习操作约束。所提出的方法与任何利用神经网络优势行动者批判框架的深度强化学习算法兼容，并可集成到任何深度强化学习算法中。本研究采用深度确定性策略梯度算法作为研究的代表实例。在改进的 IEEE 123 节点测试馈线上验证了所提方法的有效性和泛化性能，同时使用 IEEE 8500 节点测试馈线系统评估了其可扩展性。

{"title":"Resilient dynamic microgrid formation by deep reinforcement learning integrating physics-informed neural networks","authors":"Mingze Xu , Shunbo Lei , Chong Wang , Liang Liang , Junhua Zhao , Chaoyi Peng","doi":"10.1016/j.engappai.2024.109470","DOIUrl":"10.1016/j.engappai.2024.109470","url":null,"abstract":"<div><div>Dynamic microgrid formation can enhance topological flexibility within the distribution system, particularly during extreme events, thereby facilitating a more efficient restoration process. However, existing research has overlooked the impact of cold load pickup on system restoration efforts. A sudden load spike can lead to the overloading of generators and transformers, which can result in the failure of the system restoration process. This study leverages the topological flexibility through dynamic microgrid formation of the system to mitigate the impact of cold load pickup, thereby enhancing the efficiency of sequential load restoration. To alleviate the computational complexity arising from intricate operational constraints and the uncertainties inherent in cold load pickup conditions, this paper proposes a novel model-free framework. Unlike existing deep reinforcement learning models, we incorporate physical constraint information into the model by means of physics-informed neural networks, where the solution of an optimization problem is regarded as knowledge, enabling the agent to learn operational constraints more efficiently and stably. The proposed approach is compatible with and can be integrated into any deep reinforcement learning algorithm that utilizes the advantage actor–critic framework with neural networks. This research employs the deep deterministic policy gradient algorithm as a representative example for investigation. The effectiveness and generalization performance of the proposed method are validated on a modified IEEE 123-node test feeder, while its scalability is assessed using the IEEE 8500-node test feeder system.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reconstruction error based implicit regularization method and its engineering application to lung cancer diagnosis 基于重建误差的隐式正则化方法及其在肺癌诊断中的工程应用

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109439

Qinghe Zheng , Xinyu Tian , Mingqiang Yang , Shuang Han , Abdussalam Elhanashi , Sergio Saponara , Kidiyo Kpalma

The automatic diagnosis of lung cancer via artificial intelligence faces two hotspot issues: (1) insufficient data and (2) excessive redundant information, which make it difficult for convolutional neural networks (CNNs) to learn discriminative information of lung cancer. In this paper, we present the reconstruction error based implicit regularization method (REbIRM) that regularizes CNNs at the loss layer. During each training iteration, the reconstruction errors introduced by the two-stage discriminative auto-encoder are used to sharpen the generalization ability of deep CNNs by improving the decision boundary. In the application process, the trained deep CNN is used for completing computed tomography (CT) diagnostics. The main clinical benefit of our approach is that it is domain independent, requiring no specialized knowledge, and can therefore be applied to different types of datasets. To the best of our knowledge, this is the first attempt to implicitly regularize CNNs based on the reconstruction errors. Finally, experimental results on three CT image classification datasets show that REbIRM can achieve impressive results and that, in conjunction with Dropout, it obtains the state-of-the-art performance. REbIRM is also robust to the selection of hyper-parameters and only has the sublinear influence on the convergence of deep CNNs. Besides, empirical and theoretical evidence are provided to indicate that REbIRM prefers to converges in a constrained parameter space with flatter minima, which explains why it can generalize to new data. Finally, the nature of REbIRM is further explored through visualization techniques to analyze how it works in training deep CNNs.

人工智能对肺癌的自动诊断面临两个热点问题：（1）数据不足；（2）冗余信息过多，这使得卷积神经网络（CNN）难以学习肺癌的鉴别信息。本文提出了基于重构误差的隐式正则化方法（REbIRM），该方法可在损失层对卷积神经网络进行正则化。在每次训练迭代中，两级判别自动编码器引入的重构误差被用于通过改善决策边界来增强深度 CNN 的泛化能力。在应用过程中，训练好的深度 CNN 被用于完成计算机断层扫描（CT）诊断。我们的方法的主要临床优势在于它不受领域限制，不需要专业知识，因此可以应用于不同类型的数据集。据我们所知，这是首次尝试根据重建误差对 CNN 进行隐式正则化。最后，在三个 CT 图像分类数据集上的实验结果表明，REbIRM 可以获得令人印象深刻的结果，与 Dropout 结合使用，它可以获得最先进的性能。REbIRM 对超参数的选择也很稳健，对深度 CNN 的收敛性只有亚线性影响。此外，经验和理论证据还表明，REbIRM 更倾向于在具有更平坦最小值的受限参数空间中收敛，这也解释了为什么它可以泛化到新数据。最后，通过可视化技术进一步探索了 REbIRM 的本质，分析了它在训练深度 CNN 时的工作原理。

{"title":"Reconstruction error based implicit regularization method and its engineering application to lung cancer diagnosis","authors":"Qinghe Zheng , Xinyu Tian , Mingqiang Yang , Shuang Han , Abdussalam Elhanashi , Sergio Saponara , Kidiyo Kpalma","doi":"10.1016/j.engappai.2024.109439","DOIUrl":"10.1016/j.engappai.2024.109439","url":null,"abstract":"<div><div>The automatic diagnosis of lung cancer via artificial intelligence faces two hotspot issues: (1) insufficient data and (2) excessive redundant information, which make it difficult for convolutional neural networks (CNNs) to learn discriminative information of lung cancer. In this paper, we present the reconstruction error based implicit regularization method (REbIRM) that regularizes CNNs at the loss layer. During each training iteration, the reconstruction errors introduced by the two-stage discriminative auto-encoder are used to sharpen the generalization ability of deep CNNs by improving the decision boundary. In the application process, the trained deep CNN is used for completing computed tomography (CT) diagnostics. The main clinical benefit of our approach is that it is domain independent, requiring no specialized knowledge, and can therefore be applied to different types of datasets. To the best of our knowledge, this is the first attempt to implicitly regularize CNNs based on the reconstruction errors. Finally, experimental results on three CT image classification datasets show that REbIRM can achieve impressive results and that, in conjunction with Dropout, it obtains the state-of-the-art performance. REbIRM is also robust to the selection of hyper-parameters and only has the sublinear influence on the convergence of deep CNNs. Besides, empirical and theoretical evidence are provided to indicate that REbIRM prefers to converges in a constrained parameter space with flatter minima, which explains why it can generalize to new data. Finally, the nature of REbIRM is further explored through visualization techniques to analyze how it works in training deep CNNs.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142533399","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dynamic model-based intelligent fault diagnosis method for fault detection in a rod fastening rotor 基于动态模型的智能故障诊断方法，用于检测杆紧固转子的故障

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109499

Wuhui Xu , Hui Wang , Jiabin Jin , Ronggang Yang , Jiawei Xiang

A complete fault sample database is of great significance for the intelligent fault diagnosis method of rod fastening rotor. However, the lack of fault samples makes the fault diagnosis results unbelievable. To solve this issue, the dynamic model-based intelligent fault diagnosis method is established for a rod fastening rotor, and the fault sample database is enriched by numerical simulations. First, the lumped parameter model of the rod fastening rotor system is constructed and further updated using Euclidean Distance between measurement and numerical simulation of the intact system. Second, mathematical models of various fault types are incorporate into the updated model to obtain numerical simulation fault samples. Thirdly, the utilization of numerical simulation fault samples is severed as training data to the artificial intelligence (AI) models and the unknown measurement test samples will be finally classified. In this paper, Support Vector Machine, Random Forest, Bayesian Network and Decision Tree are selected as the typical AI models. Subsequently, the feasibility of classification is validated by the test bench of the rod fastening rotor system, and the problem of insufficient fault samples can be solved.

完整的故障样本数据库对于杆紧固转子的智能故障诊断方法具有重要意义。然而，由于缺乏故障样本，故障诊断结果难以令人信服。为解决这一问题，本文建立了基于动态模型的连杆拧紧转子智能故障诊断方法，并通过数值模拟丰富了故障样本数据库。首先，构建连杆拧紧转子系统的整块参数模型，并利用完整系统测量与数值模拟之间的欧氏距离进一步更新模型。其次，将各种故障类型的数学模型纳入更新后的模型，以获得数值模拟故障样本。第三，利用数值模拟故障样本作为人工智能（AI）模型的训练数据，对未知测量测试样本进行最终分类。本文选择支持向量机、随机森林、贝叶斯网络和决策树作为典型的人工智能模型。随后，通过连杆紧固转子系统的试验台验证了分类的可行性，解决了故障样本不足的问题。

{"title":"Dynamic model-based intelligent fault diagnosis method for fault detection in a rod fastening rotor","authors":"Wuhui Xu , Hui Wang , Jiabin Jin , Ronggang Yang , Jiawei Xiang","doi":"10.1016/j.engappai.2024.109499","DOIUrl":"10.1016/j.engappai.2024.109499","url":null,"abstract":"<div><div>A complete fault sample database is of great significance for the intelligent fault diagnosis method of rod fastening rotor. However, the lack of fault samples makes the fault diagnosis results unbelievable. To solve this issue, the dynamic model-based intelligent fault diagnosis method is established for a rod fastening rotor, and the fault sample database is enriched by numerical simulations. First, the lumped parameter model of the rod fastening rotor system is constructed and further updated using Euclidean Distance between measurement and numerical simulation of the intact system. Second, mathematical models of various fault types are incorporate into the updated model to obtain numerical simulation fault samples. Thirdly, the utilization of numerical simulation fault samples is severed as training data to the artificial intelligence (AI) models and the unknown measurement test samples will be finally classified. In this paper, Support Vector Machine, Random Forest, Bayesian Network and Decision Tree are selected as the typical AI models. Subsequently, the feasibility of classification is validated by the test bench of the rod fastening rotor system, and the problem of insufficient fault samples can be solved.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142533400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Two-stage encoder multi-decoder network with global–local up-sampling for defect segmentation of strip steel surface defects 采用全局-局部上采样的两级编码器多解码器网络，用于带钢表面缺陷分割

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109469

Mingxian Xu , Jingliang Wei , Xinglong Feng

Precisely segmenting surface defects in steel strips is essential for enhancing product quality. Despite the potential improvements in defect segmentation accuracy and robustness offered by deep learning methods, including autoencoder, the following challenges still persist. Firstly, in industrial environments characterized by low-contrast defects against the background and high noise-to-signal ratios, current defect detection methods still face challenges in accurately segmenting defects. Secondly, in industrial production, defects often follow a long-tail distribution, current defect detection methods exhibit lower accuracy in identifying defects in the tail-end categories. To tackle these challenges, a novel two-stage encoder multi-decoder network was introduced, comprising an initial defect detection stage and a category-specific refined stage. In the initial defect detection stage, the network’s decoder employs global–local up-sampling modules to utilize deconvolution of multiple receptive fields for up-sampling feature maps. Subsequently, in the category-specific refined stage, the network initially separates the defect feature maps by employing a category separation module. It integrates prior information through a defect refinement module and a fusion module, fusing prior decoder features with corresponding one. Simulation experiments were conducted using the real-world strip steel defect dataset, and validation experiments were performed on the industrial imbalanced dataset collected from an actual project. The experimental results demonstrate the proposed method reliability in industrial production, with the segmentation mean intersection over union achieving 87.35% and 84.98% on these respective datasets.

精确分割钢带表面缺陷对提高产品质量至关重要。尽管包括自动编码器在内的深度学习方法可能会提高缺陷分割的准确性和鲁棒性，但以下挑战依然存在。首先，在缺陷与背景对比度低、噪声信号比高的工业环境中，当前的缺陷检测方法在准确分割缺陷方面仍面临挑战。其次，在工业生产中，缺陷通常呈长尾分布，现有的缺陷检测方法在识别尾端类别缺陷时表现出较低的准确性。为应对这些挑战，我们引入了一种新型的两阶段编码器多解码器网络，包括初始缺陷检测阶段和特定类别细化阶段。在初始缺陷检测阶段，网络的解码器采用全局-局部上采样模块，利用多个感受野的解卷积对特征图进行上采样。随后，在分类细化阶段，网络利用分类分离模块对缺陷特征图进行初步分离。它通过缺陷细化模块和融合模块整合先验信息，将先验解码器特征与相应的特征融合在一起。仿真实验使用了真实世界的带钢缺陷数据集，验证实验使用了从实际项目中收集的工业不平衡数据集。实验结果证明了所提出的方法在工业生产中的可靠性，在这些数据集上，分割平均相交率比联合率分别达到了 87.35% 和 84.98%。

{"title":"Two-stage encoder multi-decoder network with global–local up-sampling for defect segmentation of strip steel surface defects","authors":"Mingxian Xu , Jingliang Wei , Xinglong Feng","doi":"10.1016/j.engappai.2024.109469","DOIUrl":"10.1016/j.engappai.2024.109469","url":null,"abstract":"<div><div>Precisely segmenting surface defects in steel strips is essential for enhancing product quality. Despite the potential improvements in defect segmentation accuracy and robustness offered by deep learning methods, including autoencoder, the following challenges still persist. Firstly, in industrial environments characterized by low-contrast defects against the background and high noise-to-signal ratios, current defect detection methods still face challenges in accurately segmenting defects. Secondly, in industrial production, defects often follow a long-tail distribution, current defect detection methods exhibit lower accuracy in identifying defects in the tail-end categories. To tackle these challenges, a novel two-stage encoder multi-decoder network was introduced, comprising an initial defect detection stage and a category-specific refined stage. In the initial defect detection stage, the network’s decoder employs global–local up-sampling modules to utilize deconvolution of multiple receptive fields for up-sampling feature maps. Subsequently, in the category-specific refined stage, the network initially separates the defect feature maps by employing a category separation module. It integrates prior information through a defect refinement module and a fusion module, fusing prior decoder features with corresponding one. Simulation experiments were conducted using the real-world strip steel defect dataset, and validation experiments were performed on the industrial imbalanced dataset collected from an actual project. The experimental results demonstrate the proposed method reliability in industrial production, with the segmentation mean intersection over union achieving 87.35% and 84.98% on these respective datasets.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Artificial intelligence-powered precision: Unveiling the landscape of liver disease diagnosis—A comprehensive review 人工智能助力精准诊断：揭开肝病诊断的面纱--全面回顾

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109452

Sireesha Vadlamudi , Vimal Kumar , Debjani Ghosh , Ajith Abraham

The global significance of diagnosing liver diseases is heightened, particularly in under-resourced regions with limited healthcare facilities. Traditional diagnostic methods, characterized by time and labor-intensive processes, have led to a growing demand for telemedicine-based solutions. The incorporation of Artificial Intelligence is deemed essential to enhance the efficiency and accuracy of diagnostic models. This review explores the seamless integration of diverse data modalities, including clinical records, demographics, laboratory values, biopsy specimens, and imaging data, emphasizing the importance of combining both types for comprehensive liver disease diagnosis. The study rigorously examines various approaches, focusing on pre-processing and feature engineering in non-image data diagnostic model development. Additionally, it analyzes studies employing Convolutional Neural Networks for cutting-edge solutions in image data interpretation. The paper provides insights into existing liver disease datasets, encompassing both image and non-image data, offering a comprehensive understanding of crucial research data sources. Emphasis is placed on performance evaluation metrics and their correlation in assessing diagnostic model efficiency. The review also explores open-source software tools dedicated to computer-aided liver analysis, enhancing exploration in liver disease diagnostics. Serving as a concise handbook, it caters to novice and experienced researchers alike, offering essential insights, summarizing the latest research, and providing a glimpse into emerging trends, challenges, and future trajectories in liver disease diagnosis.

在全球范围内，肝病诊断的重要性日益凸显，尤其是在医疗设施有限、资源匮乏的地区。传统诊断方法耗时耗力，因此对远程医疗解决方案的需求日益增长。为了提高诊断模型的效率和准确性，人工智能的融入被认为是必不可少的。本综述探讨了各种数据模式的无缝整合，包括临床记录、人口统计学、实验室值、活检标本和成像数据，强调了结合这两种数据对肝病综合诊断的重要性。该研究严格审查了各种方法，重点是非图像数据诊断模型开发中的预处理和特征工程。此外，它还分析了采用卷积神经网络的研究，为图像数据解读提供了前沿解决方案。论文深入分析了现有的肝病数据集，包括图像和非图像数据，提供了对重要研究数据源的全面了解。重点是评估诊断模型效率的性能评估指标及其相关性。该综述还探讨了专用于计算机辅助肝脏分析的开源软件工具，加强了对肝病诊断的探索。作为一本简明手册，它为新手和经验丰富的研究人员提供了重要的见解，总结了最新的研究，并提供了肝病诊断的新兴趋势、挑战和未来轨迹的一瞥。

{"title":"Artificial intelligence-powered precision: Unveiling the landscape of liver disease diagnosis—A comprehensive review","authors":"Sireesha Vadlamudi , Vimal Kumar , Debjani Ghosh , Ajith Abraham","doi":"10.1016/j.engappai.2024.109452","DOIUrl":"10.1016/j.engappai.2024.109452","url":null,"abstract":"<div><div>The global significance of diagnosing liver diseases is heightened, particularly in under-resourced regions with limited healthcare facilities. Traditional diagnostic methods, characterized by time and labor-intensive processes, have led to a growing demand for telemedicine-based solutions. The incorporation of Artificial Intelligence is deemed essential to enhance the efficiency and accuracy of diagnostic models. This review explores the seamless integration of diverse data modalities, including clinical records, demographics, laboratory values, biopsy specimens, and imaging data, emphasizing the importance of combining both types for comprehensive liver disease diagnosis. The study rigorously examines various approaches, focusing on pre-processing and feature engineering in non-image data diagnostic model development. Additionally, it analyzes studies employing Convolutional Neural Networks for cutting-edge solutions in image data interpretation. The paper provides insights into existing liver disease datasets, encompassing both image and non-image data, offering a comprehensive understanding of crucial research data sources. Emphasis is placed on performance evaluation metrics and their correlation in assessing diagnostic model efficiency. The review also explores open-source software tools dedicated to computer-aided liver analysis, enhancing exploration in liver disease diagnostics. Serving as a concise handbook, it caters to novice and experienced researchers alike, offering essential insights, summarizing the latest research, and providing a glimpse into emerging trends, challenges, and future trajectories in liver disease diagnosis.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-source information fused loose particle localization and material identification method for sealed electronic equipment 用于密封电子设备的多源信息融合松散粒子定位和材料识别方法

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109529

Zhigang Sun , Qi Liang , Guofu Zhai , Guotao Wang , Min Zhang , Jingting Sun

Sealed electronic equipment are1 an important component of aerospace defense systems, and loose particles pose a significant threat to their reliable operation. Loose particle detection is crucial. For sealed electronic equipment with large scale and complex structure, loose particle detection should not only include the judgment of existence, but also obtain location and material information to facilitate the cleaning and control work. In this paper, the authors proposed a multi-source information fused loose particle localization and material identification method. Firstly, the equipment model was designed, the loose particle samples were made, and loose particle signals were collected. Secondly, the two-stage adaptive energy threshold pulse extraction algorithm was newly proposed to extract effective pulses, and the threshold-judgement-search pulse matching algorithm was improved to match the effective pulse groups. Next, spectrograms were transformed from effective pulses to create the localization and material image set. The time-domain, frequency-domain and gray-level co-occurrence matrix features were used to construct the joint feature library. Then, the channel-weighting feature selection method was used to create the localization and material data set. Finally, PReLU-VGG19-Plus was trained on the localization and material image set to obtain the optimal localization and material neural network, while parameter-optimized XGBoost was trained on the localization and material data set to obtain the optimal localization and material classifier. On this basis, combined with the triple majority voting process, the combined localization and material framework were constructed. Extensive test results show that, the location-identification achieved by combined localization framework and the material-identification accuracy achieved by combined material framework are all 100%. The feasibility, stability, and superiority of the method proposed in this paper have been fully verified. It is an important supplement to the existing loose particle detection research, providing important reference for signal detection and classification research in similar fields, and effectively improving the reliability of sealed electronic equipment.

密封电子设备1 是航空航天防御系统的重要组成部分，松散微粒对其可靠运行构成重大威胁。松散粒子检测至关重要。对于规模大、结构复杂的密封电子设备，松散颗粒检测不仅要判断是否存在，还要获取位置和物质信息，以方便清洁和控制工作。本文作者提出了一种多源信息融合的松散颗粒定位和材料识别方法。首先，设计了设备模型，制作了松散颗粒样本，并采集了松散颗粒信号。其次，新提出了两阶段自适应能量阈值脉冲提取算法来提取有效脉冲，并改进了阈值判断-搜索脉冲匹配算法来匹配有效脉冲群。然后，从有效脉冲中转换出频谱图，创建定位和材料图像集。利用时域、频域和灰度共现矩阵特征构建联合特征库。然后，使用通道加权特征选择法创建定位和材料数据集。最后，对定位和材料图像集进行 PReLU-VGG19-Plus 训练，得到最优的定位和材料神经网络；对定位和材料数据集进行参数优化 XGBoost 训练，得到最优的定位和材料分类器。在此基础上，结合三重多数投票过程，构建了本地化和材料组合框架。大量测试结果表明，组合定位框架的定位识别准确率和组合材料框架的材料识别准确率均为 100%。本文提出的方法的可行性、稳定性和优越性得到了充分验证。它是对现有松散粒子检测研究的重要补充，为同类领域的信号检测和分类研究提供了重要参考，有效提高了密封电子设备的可靠性。

{"title":"Multi-source information fused loose particle localization and material identification method for sealed electronic equipment","authors":"Zhigang Sun , Qi Liang , Guofu Zhai , Guotao Wang , Min Zhang , Jingting Sun","doi":"10.1016/j.engappai.2024.109529","DOIUrl":"10.1016/j.engappai.2024.109529","url":null,"abstract":"<div><div>Sealed electronic equipment are1 an important component of aerospace defense systems, and loose particles pose a significant threat to their reliable operation. Loose particle detection is crucial. For sealed electronic equipment with large scale and complex structure, loose particle detection should not only include the judgment of existence, but also obtain location and material information to facilitate the cleaning and control work. In this paper, the authors proposed a multi-source information fused loose particle localization and material identification method. Firstly, the equipment model was designed, the loose particle samples were made, and loose particle signals were collected. Secondly, the two-stage adaptive energy threshold pulse extraction algorithm was newly proposed to extract effective pulses, and the threshold-judgement-search pulse matching algorithm was improved to match the effective pulse groups. Next, spectrograms were transformed from effective pulses to create the localization and material image set. The time-domain, frequency-domain and gray-level co-occurrence matrix features were used to construct the joint feature library. Then, the channel-weighting feature selection method was used to create the localization and material data set. Finally, PReLU-VGG19-Plus was trained on the localization and material image set to obtain the optimal localization and material neural network, while parameter-optimized XGBoost was trained on the localization and material data set to obtain the optimal localization and material classifier. On this basis, combined with the triple majority voting process, the combined localization and material framework were constructed. Extensive test results show that, the location-identification achieved by combined localization framework and the material-identification accuracy achieved by combined material framework are all 100%. The feasibility, stability, and superiority of the method proposed in this paper have been fully verified. It is an important supplement to the existing loose particle detection research, providing important reference for signal detection and classification research in similar fields, and effectively improving the reliability of sealed electronic equipment.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142533282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unsupervised domain adaptation for drive-by condition monitoring of multiple railway tracks 无监督领域适应性，用于对多条铁轨进行逐一状态监测

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109516

Ramin Ghiasi , Nicolas Lestoille , Cassandre Diaine , Abdollah Malekjafarian

Monitoring railway tracks through drive-by vibration data collected by in-service trains offers a cost-effective and adaptable solution for inspecting multiple railway lines. However, numerous existing drive-by monitoring methods rely on supervised learning models, necessitating extensive labelled data for each line. In this paper, a novel framework is proposed based on Unsupervised Domain Adaptation (UDA) concept which facilitates the transfer of a geometric defects diagnosis model learned from one line to a new line without the need for any labelled data from the new line. The proposed framework learns the dynamic-based features that are sensitive to damage and also invariant to different railway tracks. It comprises three components: data pre-processing, UDA implementation, and damage diagnosis. The framework uses the data from the source domain, including corresponding labels, as well as the unlabelled data from the target domain as input. The outputs of the framework consist of the predicted labels for the target domain. The performance of the proposed framework is evaluated using a comprehensive dataset of field measurements of a high-speed train passing 4 different lines within the French high-speed rail network. The proposed UDA framework is implemented using four common UDA algorithms including Information-Theoretical Learning (ITL), Geodesic Flow Kernel (GFK), Transfer Component Analysis (TCA), and Subspace Alignment (SA). The results show that the proposed framework has a 14% increase in the anomaly detection accuracy compared to traditional unsupervised learning methods in which UDA is not used. Furthermore, this study investigates the impact of incorporating a percentage of target data labels during training (semi-supervised domain adaptation), along with various sensor layouts and different tuning parameters, on the accuracy of the proposed approach. The results show that the proposed framework can significantly facilitate the monitoring of railway track conditions using the data collected by in-service trains which could be great interest of railway owners.

通过在役列车采集的旁路振动数据来监测铁路轨道，为检查多条铁路线提供了一种成本效益高、适应性强的解决方案。然而，现有的许多驱车监测方法都依赖于监督学习模型，需要为每条线路提供大量标签数据。本文提出了一种基于无监督领域适应（UDA）概念的新型框架，该框架有助于将从一条线路学习到的几何缺陷诊断模型转移到新线路，而无需从新线路获取任何标记数据。所提出的框架学习基于动态的特征，这些特征对损坏敏感，并且对不同的铁轨具有不变性。它由三个部分组成：数据预处理、UDA 实施和损坏诊断。该框架使用源域的数据（包括相应的标签）以及目标域的无标签数据作为输入。框架的输出包括目标域的预测标签。我们使用法国高速铁路网络中高速列车通过 4 条不同线路的现场测量数据集，对所提出框架的性能进行了评估。拟议的 UDA 框架使用四种常见的 UDA 算法来实现，包括信息理论学习 (ITL)、大地流核 (GFK)、转移成分分析 (TCA) 和子空间对齐 (SA)。结果表明，与未使用 UDA 的传统无监督学习方法相比，拟议框架的异常检测准确率提高了 14%。此外，本研究还探讨了在训练过程中加入一定比例的目标数据标签（半监督域自适应）以及各种传感器布局和不同调整参数对所提方法准确性的影响。研究结果表明，所提出的框架可以利用在役列车收集的数据极大地促进对铁路轨道状况的监测，这可能是铁路业主非常感兴趣的。

{"title":"Unsupervised domain adaptation for drive-by condition monitoring of multiple railway tracks","authors":"Ramin Ghiasi , Nicolas Lestoille , Cassandre Diaine , Abdollah Malekjafarian","doi":"10.1016/j.engappai.2024.109516","DOIUrl":"10.1016/j.engappai.2024.109516","url":null,"abstract":"<div><div>Monitoring railway tracks through drive-by vibration data collected by in-service trains offers a cost-effective and adaptable solution for inspecting multiple railway lines. However, numerous existing drive-by monitoring methods rely on supervised learning models, necessitating extensive labelled data for each line. In this paper, a novel framework is proposed based on Unsupervised Domain Adaptation (UDA) concept which facilitates the transfer of a geometric defects diagnosis model learned from one line to a new line without the need for any labelled data from the new line. The proposed framework learns the dynamic-based features that are sensitive to damage and also invariant to different railway tracks. It comprises three components: data pre-processing, UDA implementation, and damage diagnosis. The framework uses the data from the source domain, including corresponding labels, as well as the unlabelled data from the target domain as input. The outputs of the framework consist of the predicted labels for the target domain. The performance of the proposed framework is evaluated using a comprehensive dataset of field measurements of a high-speed train passing 4 different lines within the French high-speed rail network. The proposed UDA framework is implemented using four common UDA algorithms including Information-Theoretical Learning (ITL), Geodesic Flow Kernel (GFK), Transfer Component Analysis (TCA), and Subspace Alignment (SA). The results show that the proposed framework has a 14% increase in the anomaly detection accuracy compared to traditional unsupervised learning methods in which UDA is not used. Furthermore, this study investigates the impact of incorporating a percentage of target data labels during training (semi-supervised domain adaptation), along with various sensor layouts and different tuning parameters, on the accuracy of the proposed approach. The results show that the proposed framework can significantly facilitate the monitoring of railway track conditions using the data collected by in-service trains which could be great interest of railway owners.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142533283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

BABE: Backdoor attack with bokeh effects via latent separation suppression BABE：通过潜在分离抑制实现虚化效果的后门攻击

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-22 DOI: 10.1016/j.engappai.2024.109462

Junjian Li , Honglong Chen , Yudong Gao , Shaozhong Guo , Kai Lin , Yuping Liu , Peng Sun

The escalating menace of backdoor attacks constitutes a formidable obstacle to the ongoing advancement of deep neural networks (DNNs), particularly in the security-sensitive applications such as face recognition and self-driving. Backdoored models render deliberately incorrect predictions on the inputs with the crafted triggers while behaving normally with the benign ones. Despite demonstrating the varying degrees of threat, existing backdoor attack strategies often prioritize stealthiness and defense evasions but neglect the practical feasibility in the real-world deployment scenarios. In this paper, we develop a backdoor attack leveraging bokeh effects (

B A B E

), which introduces the bokeh effects as the trigger. Once the backdoored model is deployed in the vision application, the model’s malicious behaviors can be activated only by utilizing the captured bokeh images without any other modifications. Specially, we employ the saliency and depth estimation maps to derive the bokeh images, thereby serving as the poisoned samples. Furthermore, to avoid the latent separation of the generated poisoned images, we propose distinct attack strategies on the basis of the adversary’s prior abilities. For the adversary only with the data manipulation, we retain the original semantic labels for a subset of poisoned data during the training process. For the adversary with the manipulation of both the data and models, we construct a reference model trained on the clean samples to impose constraints on the latent representations of the poisoned images. Extensive experiments demonstrate the attack effects of the proposed

B A B E

, even on the bokeh photos captured from Digital Still Cameras (DSC) and smartphones.

不断升级的后门攻击威胁对深度神经网络（DNN）的持续发展构成了巨大障碍，尤其是在人脸识别和自动驾驶等对安全敏感的应用领域。受后门攻击的模型会故意用精心制作的触发器对输入做出错误的预测，而对良性触发器则表现正常。尽管存在不同程度的威胁，但现有的后门攻击策略往往优先考虑隐蔽性和防御规避，却忽视了在现实世界部署场景中的实际可行性。本文开发了一种利用虚化效果的后门攻击（BABE），引入虚化效果作为触发器。一旦在视觉应用中部署了后门模型，只需利用捕捉到的虚化图像就能激活模型的恶意行为，而无需做任何其他修改。特别是，我们利用显著性和深度估计图来获取虚化图像，从而作为中毒样本。此外，为了避免对生成的中毒图像进行潜在分离，我们根据对手的先验能力提出了不同的攻击策略。对于只具有数据操作能力的对手，我们在训练过程中保留了中毒数据子集的原始语义标签。对于同时操纵数据和模型的对手，我们构建一个在干净样本上训练的参考模型，对中毒图像的潜在表示施加约束。广泛的实验证明了所提出的 BABE 的攻击效果，即使是在从数码相机（DSC）和智能手机捕获的虚化照片上也是如此。

{"title":"BABE: Backdoor attack with bokeh effects via latent separation suppression","authors":"Junjian Li , Honglong Chen , Yudong Gao , Shaozhong Guo , Kai Lin , Yuping Liu , Peng Sun","doi":"10.1016/j.engappai.2024.109462","DOIUrl":"10.1016/j.engappai.2024.109462","url":null,"abstract":"<div><div>The escalating menace of backdoor attacks constitutes a formidable obstacle to the ongoing advancement of deep neural networks (DNNs), particularly in the security-sensitive applications such as face recognition and self-driving. Backdoored models render deliberately incorrect predictions on the inputs with the crafted triggers while behaving normally with the benign ones. Despite demonstrating the varying degrees of threat, existing backdoor attack strategies often prioritize stealthiness and defense evasions but neglect the practical feasibility in the real-world deployment scenarios. In this paper, we develop a backdoor attack leveraging bokeh effects (<span><math><mrow><mi>B</mi><mi>A</mi><mi>B</mi><mi>E</mi></mrow></math></span>), which introduces the bokeh effects as the trigger. Once the backdoored model is deployed in the vision application, the model’s malicious behaviors can be activated only by utilizing the captured bokeh images without any other modifications. Specially, we employ the saliency and depth estimation maps to derive the bokeh images, thereby serving as the poisoned samples. Furthermore, to avoid the latent separation of the generated poisoned images, we propose distinct attack strategies on the basis of the adversary’s prior abilities. For the adversary only with the data manipulation, we retain the original semantic labels for a subset of poisoned data during the training process. For the adversary with the manipulation of both the data and models, we construct a reference model trained on the clean samples to impose constraints on the latent representations of the poisoned images. Extensive experiments demonstrate the attack effects of the proposed <span><math><mrow><mi>B</mi><mi>A</mi><mi>B</mi><mi>E</mi></mrow></math></span>, even on the bokeh photos captured from Digital Still Cameras (DSC) and smartphones.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142535033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spiking neural networks for autonomous driving: A review 用于自动驾驶的尖峰神经网络：综述

IF 7.5 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Engineering Applications of Artificial Intelligence

Pub Date : 2024-10-21 DOI: 10.1016/j.engappai.2024.109415

Fernando S. Martínez , Jordi Casas-Roma , Laia Subirats , Raúl Parada

The rapid progress of autonomous driving (AD) has triggered a surge in demand for safer and more efficient autonomous vehicles, owing to the intricacy of modern urban environments. Traditional approaches to autonomous driving have heavily relied on conventional machine learning methodologies, particularly convolutional neural networks (CNNs) and recurrent neural networks (RNNs), for tasks such as perception, decision-making, and control. Presently, major companies such as Tesla, Waymo, Uber, and Volkswagen Group (VW) leverage neural networks for advanced perception and autonomous decision-making. However, concerns have been raised about the escalating computational requirements of training these neural models, primarily in terms of energy consumption and environmental impact. In the situation of optimisation and sustainability, Spiking Neural Networks (SNNs), inspired by the temporal processing of the human brain, have come forth as a third-generation of neural networks, famed for their energy efficiency, potential for handling real-time driving scenarios and processing temporal information efficiently. However, SNNs have not yet achieved the performance levels of their predecessors in critical AD tasks, partly due to the intricate dynamics of neurons, their non-differentiable spike operations, and the lack of specialised benchmark workloads and datasets, among others. This paper examines the principles, models, learning rules, and recent advancements of SNNs in the AD domain. Neuromorphic hardware, hand in hand with SNNs, shows potential but has challenges in accessibility, cost, integration, and scalability. This examination aims to bridge gaps by providing a comprehensive understanding of SNNs in the AD field. It emphasises the role of SNNs in shaping the future of AD while considering optimisation and sustainability.

由于现代城市环境错综复杂，自动驾驶（AD）的快速发展引发了对更安全、更高效的自动驾驶汽车的需求激增。传统的自动驾驶方法在很大程度上依赖于传统的机器学习方法，特别是卷积神经网络（CNN）和递归神经网络（RNN），用于感知、决策和控制等任务。目前，特斯拉、Waymo、Uber 和大众汽车集团（VW）等大型公司都在利用神经网络进行高级感知和自主决策。然而，人们对训练这些神经网络模型的计算要求不断提高表示担忧，主要表现在能源消耗和环境影响方面。在优化和可持续发展的形势下，受人脑时间处理启发的尖峰神经网络（SNN）作为第三代神经网络应运而生，以其能源效率、处理实时驾驶场景的潜力和高效处理时间信息而著称。然而，SNN 在关键的 AD 任务中尚未达到其前辈的性能水平，部分原因在于神经元错综复杂的动态特性、其无差别的尖峰操作，以及缺乏专门的基准工作负载和数据集等。本文探讨了反向神经网络的原理、模型、学习规则和最近在反向神经网络领域取得的进展。神经形态硬件与 SNNs 的结合显示出潜力，但在可访问性、成本、集成性和可扩展性方面存在挑战。本研究旨在通过提供对反向障碍领域中 SNNs 的全面了解来缩小差距。在考虑优化和可持续性的同时，它还强调了智能网络在塑造未来自动驾驶技术中的作用。

{"title":"Spiking neural networks for autonomous driving: A review","authors":"Fernando S. Martínez , Jordi Casas-Roma , Laia Subirats , Raúl Parada","doi":"10.1016/j.engappai.2024.109415","DOIUrl":"10.1016/j.engappai.2024.109415","url":null,"abstract":"<div><div>The rapid progress of autonomous driving (AD) has triggered a surge in demand for safer and more efficient autonomous vehicles, owing to the intricacy of modern urban environments. Traditional approaches to autonomous driving have heavily relied on conventional machine learning methodologies, particularly convolutional neural networks (CNNs) and recurrent neural networks (RNNs), for tasks such as perception, decision-making, and control. Presently, major companies such as Tesla, Waymo, Uber, and Volkswagen Group (VW) leverage neural networks for advanced perception and autonomous decision-making. However, concerns have been raised about the escalating computational requirements of training these neural models, primarily in terms of energy consumption and environmental impact. In the situation of optimisation and sustainability, Spiking Neural Networks (SNNs), inspired by the temporal processing of the human brain, have come forth as a third-generation of neural networks, famed for their energy efficiency, potential for handling real-time driving scenarios and processing temporal information efficiently. However, SNNs have not yet achieved the performance levels of their predecessors in critical AD tasks, partly due to the intricate dynamics of neurons, their non-differentiable spike operations, and the lack of specialised benchmark workloads and datasets, among others. This paper examines the principles, models, learning rules, and recent advancements of SNNs in the AD domain. Neuromorphic hardware, hand in hand with SNNs, shows potential but has challenges in accessibility, cost, integration, and scalability. This examination aims to bridge gaps by providing a comprehensive understanding of SNNs in the AD field. It emphasises the role of SNNs in shaping the future of AD while considering optimisation and sustainability.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142535248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0