The orienteering problem (OP) is widely applied in real life. However, as the scale of real-world problem scenarios grows quickly, traditional exact, heuristics, and learning-based methods have difficulty balancing optimization accuracy and efficiency. This study proposes a problem decomposition-based double-layer optimization framework named DEA-DYPN to solve OPs. Using a diversity evolutionary algorithm (DEA) as the external optimizer and a dynamic pointer network (DYPN) as the inner optimizer, we significantly reduce the difficulty of solving large-scale OPs. Several targeted optimization operators are innovatively designed for stronger search ability, including a greedy population initialization heuristic, an elite strategy, a population restart mechanism, and a fitness-sharing selection strategy. Moreover, a dynamic embedding mechanism is introduced to DYPN to improve its characteristic learning ability. Extensive comparative experiments on OP instances with sizes from 20 to 500 are conducted for algorithmic performance validation. More experiments and analyses, including the significance test, stability analysis, complexity analysis, sensitivity analysis, and ablation experiments, are also conducted for comprehensive algorithmic evaluation. Experimental results show that our proposed DEA-DYPN ranks first according to the Friedman test and outperforms the competitor algorithms by 69%.
定向行走问题（OP）在现实生活中应用广泛。然而，随着现实世界问题场景规模的快速增长，传统的精确、启发式和基于学习的方法难以兼顾优化精度和效率。本研究提出了一种基于问题分解的双层优化框架，名为 DEA-DYPN，用于解决 OPs。以多样性进化算法（DEA）为外部优化器，以动态指针网络（DYPN）为内部优化器，大大降低了大规模 OP 的求解难度。为了增强搜索能力，我们创新性地设计了几种有针对性的优化算子，包括贪婪种群初始化启发式、精英策略、种群重启机制和适配性共享选择策略。此外，DYPN 还引入了动态嵌入机制，以提高其特有的学习能力。为了验证算法的性能，我们在 20 到 500 个 OP 实例上进行了广泛的对比实验。此外，还进行了更多的实验和分析，包括显著性检验、稳定性分析、复杂性分析、灵敏度分析和消融实验，以对算法进行综合评估。实验结果表明，根据弗里德曼测试，我们提出的 DEA-DYPN 排在第一位，比竞争算法高出 69%。
{"title":"Solving Orienteering Problems by Hybridizing Evolutionary Algorithm and Deep Reinforcement Learning","authors":"Rui Wang;Wei Liu;Kaiwen Li;Tao Zhang;Ling Wang;Xin Xu","doi":"10.1109/TAI.2024.3409520","DOIUrl":"https://doi.org/10.1109/TAI.2024.3409520","url":null,"abstract":"The orienteering problem (OP) is widely applied in real life. However, as the scale of real-world problem scenarios grows quickly, traditional exact, heuristics, and learning-based methods have difficulty balancing optimization accuracy and efficiency. This study proposes a problem decomposition-based double-layer optimization framework named DEA-DYPN to solve OPs. Using a diversity evolutionary algorithm (DEA) as the external optimizer and a dynamic pointer network (DYPN) as the inner optimizer, we significantly reduce the difficulty of solving large-scale OPs. Several targeted optimization operators are innovatively designed for stronger search ability, including a greedy population initialization heuristic, an elite strategy, a population restart mechanism, and a fitness-sharing selection strategy. Moreover, a dynamic embedding mechanism is introduced to DYPN to improve its characteristic learning ability. Extensive comparative experiments on OP instances with sizes from 20 to 500 are conducted for algorithmic performance validation. More experiments and analyses, including the significance test, stability analysis, complexity analysis, sensitivity analysis, and ablation experiments, are also conducted for comprehensive algorithmic evaluation. Experimental results show that our proposed DEA-DYPN ranks first according to the Friedman test and outperforms the competitor algorithms by 69%.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5493-5508"},"PeriodicalIF":0.0,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-04DOI: 10.1109/TAI.2024.3408717
Kyle Worrall;Zongyu Yin;Tom Collins
There have been many attempts to model the ability of human musicians to take a score and perform or render it expressively, by adding tempo, timing, loudness, and articulation changes to nonexpressive music data. While expressive rendering models exist in academic research, most of these are not open source or accessible, meaning they are difficult to evaluate empirically and have not been widely adopted in professional music software. Systematic comparative evaluation of such algorithms stopped after the last performance rendering contest (RENCON) in 2013, making it difficult to compare newer models to existing work in a fair and valid way. In this article, we introduce the first transformer-based model for expressive rendering, cue-free express + pedal (CFE + P), which predicts expressive attributes such as notewise dynamics and micro-timing adjustments, and beatwise tempo and sustain pedal use based only on the start and end times and pitches of notes (e.g., inexpressive musical instrument digital interface (MIDI) input). We perform two comparative evaluations on our model against a nonmachine learning baseline taken from professional music software and two open-source algorithms—a feedforward neural network (FFNN) and hierarchical recurrent neural network (HRNN). The results of two listening studies indicate that our model renders passages that outperform what can be done in professional music software such as Logic Pro and Ableton Live. 1
All data and preexisting hypotheses can be accessed via the Open Science Foundation: https://osf.io/6uwjk/.
{"title":"Comparative Evaluation in the Wild: Systems for the Expressive Rendering of Music","authors":"Kyle Worrall;Zongyu Yin;Tom Collins","doi":"10.1109/TAI.2024.3408717","DOIUrl":"https://doi.org/10.1109/TAI.2024.3408717","url":null,"abstract":"There have been many attempts to model the ability of human musicians to take a score and perform or render it expressively, by adding tempo, timing, loudness, and articulation changes to nonexpressive music data. While expressive rendering models exist in academic research, most of these are not open source or accessible, meaning they are difficult to evaluate empirically and have not been widely adopted in professional music software. Systematic comparative evaluation of such algorithms stopped after the last performance rendering contest (RENCON) in 2013, making it difficult to compare newer models to existing work in a fair and valid way. In this article, we introduce the first transformer-based model for expressive rendering, cue-free express + pedal (CFE + P), which predicts expressive attributes such as notewise dynamics and micro-timing adjustments, and beatwise tempo and sustain pedal use based only on the start and end times and pitches of notes (e.g., inexpressive musical instrument digital interface (MIDI) input). We perform two comparative evaluations on our model against a nonmachine learning baseline taken from professional music software and two open-source algorithms—a feedforward neural network (FFNN) and hierarchical recurrent neural network (HRNN). The results of two listening studies indicate that our model renders passages that outperform what can be done in professional music software such as Logic Pro and Ableton Live.\u0000<xref><sup>1</sup></xref>\u0000<fn><label><sup>1</sup></label><p>All data and preexisting hypotheses can be accessed via the Open Science Foundation: <uri>https://osf.io/6uwjk/</uri>.</p></fn>","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"5290-5303"},"PeriodicalIF":0.0,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142443129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pancreatic cancer is a highly fatal cancer type. Patients are typically in an advanced stage at their first diagnosis, mainly due to the absence of distinctive early stage symptoms and lack of effective early diagnostic methods. In this work, we propose an automated method for pancreatic cancer diagnosis using noncontrast computed tomography (CT), taking advantage of its widespread availability in clinic. Currently, a primary challenge limiting the clinical value of intelligent systems is low generalization, i.e., the difficulty of achieving stable performance across datasets from different medical sources. To address this challenge, a novel causality-informed graph intervention model is developed based on a multi-instance-learning framework integrated with graph neural network (GNN) for the extraction of local discriminative features. Within this model, we develop a graph causal intervention scheme with three levels of intervention for graph nodes, structures, and representations. This scheme systematically suppresses noncausal factors and thus lead to generalizable predictions. Specifically, first, a target node perturbation strategy is designed to capture target-region features. Second, a causal-structure separation module is developed to automatically identify the causal graph structures for obtaining stable representations of whole target regions. Third, a graph-level feature consistency mechanism is proposed to extract invariant features. Comprehensive experiments on large-scale datasets validated the promising early diagnosis performance of our proposed model. The model generalizability was confirmed on three independent datasets, where the classification accuracy reached 86.3%, 80.4%, and 82.2%, respectively. Overall, we provide a valuable potential tool for pancreatic cancer screening and early diagnosis.
{"title":"A Causality-Informed Graph Intervention Model for Pancreatic Cancer Early Diagnosis","authors":"Xinyue Li;Rui Guo;Hongzhang Zhu;Tao Chen;Xiaohua Qian","doi":"10.1109/TAI.2024.3395586","DOIUrl":"https://doi.org/10.1109/TAI.2024.3395586","url":null,"abstract":"Pancreatic cancer is a highly fatal cancer type. Patients are typically in an advanced stage at their first diagnosis, mainly due to the absence of distinctive early stage symptoms and lack of effective early diagnostic methods. In this work, we propose an automated method for pancreatic cancer diagnosis using noncontrast computed tomography (CT), taking advantage of its widespread availability in clinic. Currently, a primary challenge limiting the clinical value of intelligent systems is low generalization, i.e., the difficulty of achieving stable performance across datasets from different medical sources. To address this challenge, a novel causality-informed graph intervention model is developed based on a multi-instance-learning framework integrated with graph neural network (GNN) for the extraction of local discriminative features. Within this model, we develop a graph causal intervention scheme with three levels of intervention for graph nodes, structures, and representations. This scheme systematically suppresses noncausal factors and thus lead to generalizable predictions. Specifically, first, a target node perturbation strategy is designed to capture target-region features. Second, a causal-structure separation module is developed to automatically identify the causal graph structures for obtaining stable representations of whole target regions. Third, a graph-level feature consistency mechanism is proposed to extract invariant features. Comprehensive experiments on large-scale datasets validated the promising early diagnosis performance of our proposed model. The model generalizability was confirmed on three independent datasets, where the classification accuracy reached 86.3%, 80.4%, and 82.2%, respectively. Overall, we provide a valuable potential tool for pancreatic cancer screening and early diagnosis.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4675-4685"},"PeriodicalIF":0.0,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169722","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-30DOI: 10.1109/TAI.2024.3395574
Yufei Liu;Jia Wu;Jie Cao
Social behavior prediction on social media is attracting significant attention from researchers. Social e-commerce focuses on engagement marketing, which emphasizes social behavior because it effectively increases brand recognition. Currently, existing works on social behavior prediction suffer from two main problems: 1) they assume that social influence probabilities can be learned independently of each other, and their calculations do not include any influence probability estimations based on friends’ behavior; and 2) negative sampling of subgraphs is usually ignored in social behavior prediction work. To the best of our knowledge, introducing graph contrastive learning (GCL) to social behavior prediction is novel and interesting. In this article, we propose a framework, social behavior prediction via graph contrastive learning with attention named SBP-GCA, to promote social behavior prediction. First, two methods are designed to extract subgraphs from the original graph, and their structural features are learned by GCL. Then, it models how a user's behavior is influenced by neighbors and learns influence features via graph attention networks (GATs). Furthermore, it combines structural features, influence features, and intrinsic features to predict social behavior. Extensive and systematic experiments on three datasets validate the superiority of the proposed SBP-GCA.
社交媒体上的社交行为预测正引起研究人员的极大关注。社交电子商务侧重于参与式营销，强调社交行为，因为它能有效提高品牌认知度。目前，有关社交行为预测的现有研究存在两个主要问题：1）假设社交影响概率可以独立学习，其计算不包括任何基于好友行为的影响概率估计；2）社交行为预测工作通常忽略子图的负采样。据我们所知，将图对比学习（GCL）引入社交行为预测是一项新颖而有趣的工作。在本文中，我们提出了一个通过图对比学习（graph contrastive learning with attention）进行社会行为预测的框架，命名为 SBP-GCA，以促进社会行为预测。首先，我们设计了两种方法从原始图中提取子图，并通过 GCL 学习子图的结构特征。然后，它对用户行为如何受邻居影响进行建模，并通过图注意力网络（GAT）学习影响特征。此外，它还结合了结构特征、影响特征和内在特征来预测社交行为。在三个数据集上进行的广泛而系统的实验验证了所提出的 SBP-GCA 的优越性。
{"title":"SBP-GCA: Social Behavior Prediction via Graph Contrastive Learning With Attention","authors":"Yufei Liu;Jia Wu;Jie Cao","doi":"10.1109/TAI.2024.3395574","DOIUrl":"https://doi.org/10.1109/TAI.2024.3395574","url":null,"abstract":"Social behavior prediction on social media is attracting significant attention from researchers. Social e-commerce focuses on engagement marketing, which emphasizes social behavior because it effectively increases brand recognition. Currently, existing works on social behavior prediction suffer from two main problems: 1) they assume that social influence probabilities can be learned independently of each other, and their calculations do not include any influence probability estimations based on friends’ behavior; and 2) negative sampling of subgraphs is usually ignored in social behavior prediction work. To the best of our knowledge, introducing graph contrastive learning (GCL) to social behavior prediction is novel and interesting. In this article, we propose a framework, social behavior prediction via graph contrastive learning with attention named SBP-GCA, to promote social behavior prediction. First, two methods are designed to extract subgraphs from the original graph, and their structural features are learned by GCL. Then, it models how a user's behavior is influenced by neighbors and learns influence features via graph attention networks (GATs). Furthermore, it combines structural features, influence features, and intrinsic features to predict social behavior. Extensive and systematic experiments on three datasets validate the superiority of the proposed SBP-GCA.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4708-4722"},"PeriodicalIF":0.0,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-30DOI: 10.1109/TAI.2024.3394792
Israq Ahmed Anik;A. H. M. Kamal;Muhammad Ashad Kabir;Shahadat Uddin;Mohammad Ali Moni
Major depressive disorder (MDD), commonly called depression, is a prevalent psychiatric condition diagnosed via questionnaire-based mental status assessments. However, this method often yields inconsistent and inaccurate results. Furthermore, there is currently a lack of a comprehensive diagnostic framework for MDD that assesses various brainwaves (alpha, theta, gamma, etc.) of electroencephalogram (EEG) signals as potential biomarkers, aiming to identify the most effective one for achieving accurate and robust diagnostic outcomes. To address this issue, we propose an innovative approach employing a deep convolutional neural network (DCNN) for MDD diagnosis utilizing the brainwaves present in EEG signals. Our proposed model, an extended 11-layer 1-D convolutional neural network (Ex-1DCNN), is designed to automatically learn from input EEG signals, foregoing the need for manual feature selection. By harnessing intrinsic brainwave patterns, our model demonstrates adaptability in classifying EEG signals into depressive and healthy categories. We have conducted an extensive analysis to identify optimal brainwave features and epoch duration for accurate MDD diagnosis. Leveraging EEG data from 34 MDD patients and 30 healthy subjects, we have identified that the Gamma brainwave at a 15-s epoch duration is the most effective configuration, achieving an accuracy of 99.60%, sensitivity of 100%, specificity of 99.21%, and an F1 score of 99.60%. This study highlights the potential of deep-learning techniques in streamlining the diagnostic process for MDD and offering a reliable aid to clinicians in MDD diagnosis.
{"title":"A Robust Deep-Learning Model to Detect Major Depressive Disorder Utilizing EEG Signals","authors":"Israq Ahmed Anik;A. H. M. Kamal;Muhammad Ashad Kabir;Shahadat Uddin;Mohammad Ali Moni","doi":"10.1109/TAI.2024.3394792","DOIUrl":"https://doi.org/10.1109/TAI.2024.3394792","url":null,"abstract":"Major depressive disorder (MDD), commonly called depression, is a prevalent psychiatric condition diagnosed via questionnaire-based mental status assessments. However, this method often yields inconsistent and inaccurate results. Furthermore, there is currently a lack of a comprehensive diagnostic framework for MDD that assesses various brainwaves (alpha, theta, gamma, etc.) of electroencephalogram (EEG) signals as potential biomarkers, aiming to identify the most effective one for achieving accurate and robust diagnostic outcomes. To address this issue, we propose an innovative approach employing a deep convolutional neural network (DCNN) for MDD diagnosis utilizing the brainwaves present in EEG signals. Our proposed model, an extended 11-layer 1-D convolutional neural network (Ex-1DCNN), is designed to automatically learn from input EEG signals, foregoing the need for manual feature selection. By harnessing intrinsic brainwave patterns, our model demonstrates adaptability in classifying EEG signals into depressive and healthy categories. We have conducted an extensive analysis to identify optimal brainwave features and epoch duration for accurate MDD diagnosis. Leveraging EEG data from 34 MDD patients and 30 healthy subjects, we have identified that the Gamma brainwave at a 15-s epoch duration is the most effective configuration, achieving an accuracy of 99.60%, sensitivity of 100%, specificity of 99.21%, and an F1 score of 99.60%. This study highlights the potential of deep-learning techniques in streamlining the diagnostic process for MDD and offering a reliable aid to clinicians in MDD diagnosis.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"4938-4947"},"PeriodicalIF":0.0,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142443088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-30DOI: 10.1109/TAI.2024.3395416
Hamed Rafiei;Mohammad-R. Akbarzadeh-T.
In the realm of natural language processing, hedge-embedded structures have contributed considerably by appreciating linguistic variables and distinguishing overlapped classes. This aspect of natural languages considerably affects the building of linguistically interpretable architectures for fuzzy neural networks (FNNs). Here, we propose extending the idea of hedge-embedded linguistic fuzzy neural networks (LiFNNs) to the systems identification and control paradigm. This perspective leads us to the universal approximation property for this mathematical construct using the Stone–Weierstrass theorem and the proof of stability for the resulting nonlinear system identification process using the Lyapunov function. Furthermore, the power activation functions in the membership degrees of the proposed network enable linguistic hedge interpretation and more precise learning. Finally, the proposed LiFNN, optimized using a backpropagation learning algorithm, is evaluated on several problems in function approximation (periodic functions and quadratic Hermite function), system identification (a nonlinear system), and direct adaptive control fields. Results show that memberships are more distinguishable in the proposed LiFNN, leading to $sim$ 50% less error on the average and higher granulation and interpretability.
{"title":"Hedge-Embedded Linguistic Fuzzy Neural Networks for Systems Identification and Control","authors":"Hamed Rafiei;Mohammad-R. Akbarzadeh-T.","doi":"10.1109/TAI.2024.3395416","DOIUrl":"https://doi.org/10.1109/TAI.2024.3395416","url":null,"abstract":"In the realm of natural language processing, hedge-embedded structures have contributed considerably by appreciating linguistic variables and distinguishing overlapped classes. This aspect of natural languages considerably affects the building of linguistically interpretable architectures for fuzzy neural networks (FNNs). Here, we propose extending the idea of hedge-embedded linguistic fuzzy neural networks (LiFNNs) to the systems identification and control paradigm. This perspective leads us to the universal approximation property for this mathematical construct using the Stone–Weierstrass theorem and the proof of stability for the resulting nonlinear system identification process using the Lyapunov function. Furthermore, the power activation functions in the membership degrees of the proposed network enable linguistic hedge interpretation and more precise learning. Finally, the proposed LiFNN, optimized using a backpropagation learning algorithm, is evaluated on several problems in function approximation (periodic functions and quadratic Hermite function), system identification (a nonlinear system), and direct adaptive control fields. Results show that memberships are more distinguishable in the proposed LiFNN, leading to \u0000<inline-formula><tex-math>$sim$</tex-math></inline-formula>\u000050% less error on the average and higher granulation and interpretability.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"4928-4937"},"PeriodicalIF":0.0,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-30DOI: 10.1109/TAI.2024.3395231
Jie Song;Zexin Zhang;Zirui Tang;Shi Feng;Yu Gu
Code summarization aims to generate natural language (NL) summaries automatically given the source code snippet, which aids developers in understanding source code faster and improves software maintenance. Recent approaches using NL techniques in code summarization fall short of adequately capturing the syntactic characteristics of programming languages (PLs), particularly the position-related syntax, from which the semantics of the source code can be extracted. In this article, we present Syntax transforMer (SyMer) based on the transformer architecture where we enhance it with position-related syntax complement (PSC) to better capture syntactic characteristics. PSC takes advantage of unambiguous relations among code tokens in abstract syntax tree (AST), as well as the gathered attention on crucial code tokens indicated by its syntactic structure. The experimental results demonstrate that SyMer outperforms state-of-the-art models by at least 2.4% bilingual evaluation understudy (BLEU), 1.0% metric for evaluation of translation with explicit ORdering (METEOR) on Java benchmark, and 4.8% (BLEU), 5.1% (METEOR), and 3.2% recall-oriented understudy for gisting evaluation - longest common subsequence (ROUGE-L) on Python benchmark.
{"title":"Improving Code Summarization With Tree Transformer Enhanced by Position-Related Syntax Complement","authors":"Jie Song;Zexin Zhang;Zirui Tang;Shi Feng;Yu Gu","doi":"10.1109/TAI.2024.3395231","DOIUrl":"https://doi.org/10.1109/TAI.2024.3395231","url":null,"abstract":"Code summarization aims to generate natural language (NL) summaries automatically given the source code snippet, which aids developers in understanding source code faster and improves software maintenance. Recent approaches using NL techniques in code summarization fall short of adequately capturing the syntactic characteristics of programming languages (PLs), particularly the position-related syntax, from which the semantics of the source code can be extracted. In this article, we present Syntax transforMer (SyMer) based on the transformer architecture where we enhance it with position-related syntax complement (PSC) to better capture syntactic characteristics. PSC takes advantage of unambiguous relations among code tokens in abstract syntax tree (AST), as well as the gathered attention on crucial code tokens indicated by its syntactic structure. The experimental results demonstrate that SyMer outperforms state-of-the-art models by at least 2.4% bilingual evaluation understudy (BLEU), 1.0% metric for evaluation of translation with explicit ORdering (METEOR) on Java benchmark, and 4.8% (BLEU), 5.1% (METEOR), and 3.2% recall-oriented understudy for gisting evaluation - longest common subsequence (ROUGE-L) on Python benchmark.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4776-4786"},"PeriodicalIF":0.0,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-29DOI: 10.1109/TAI.2024.3394797
Tasnim Ahmed;Naveed Ejaz;Salimur Choudhury
Road infrastructure is essential for transportation safety and efficiency. However, the current methods for assessing road conditions, crucial for effective planning and maintenance, suffer from high costs, time-intensive procedures, infrequent data collection, and limited real-time capabilities. This article presents an efficient lightweight system to analyze road quality from video feeds in real time. The backbone of the system is EdgeFusionViT, a novel vision transformer (ViT)-based architecture that uses an attention-based late fusion mechanism. The proposed architecture outperforms lightweight convolutional neural network (CNN)-based and ViT-based models. Its practicality is demonstrated by its deployment on an edge device, the Nvidia Jetson Orin Nano, enabling real-time road analysis at 12 frames per second. EdgeFusionViT outperforms existing benchmarks, achieving an impressive accuracy of 89.76% on the road surface condition dataset (RSCD). Notably, the model maintains a commendable accuracy of 76.89% even when trained with only 2% of the dataset, demonstrating its robustness and efficiency. These findings highlight the system's potential in road infrastructure management. It aids in creating safer, more efficient transport systems through timely, accurate road condition assessments. The study sets a new benchmark and opens up possibilities for advanced machine learning in infrastructure management.
道路基础设施对运输安全和效率至关重要。然而，目前评估道路状况的方法对有效规划和维护至关重要，但却存在成本高、程序耗时、数据收集不频繁、实时性有限等问题。本文介绍了一种高效的轻量级系统，可通过视频馈送实时分析道路质量。该系统的支柱是 EdgeFusionViT，它是一种基于视觉转换器（ViT）的新型架构，采用基于注意力的后期融合机制。所提出的架构优于基于卷积神经网络（CNN）的轻量级模型和基于 ViT 的模型。通过在边缘设备 Nvidia Jetson Orin Nano 上的部署，以每秒 12 帧的速度进行实时道路分析，证明了该架构的实用性。EdgeFusionViT 超越了现有基准，在路面状况数据集（RSCD）上实现了 89.76% 的惊人准确率。值得注意的是，即使只使用 2% 的数据集进行训练，该模型也能保持 76.89% 的准确率，这证明了它的鲁棒性和高效性。这些发现凸显了该系统在道路基础设施管理方面的潜力。通过及时、准确的道路状况评估，该系统有助于创建更安全、更高效的交通系统。这项研究树立了一个新的基准，为基础设施管理中的高级机器学习开辟了可能性。
{"title":"Redefining Real-Time Road Quality Analysis With Vision Transformers on Edge Devices","authors":"Tasnim Ahmed;Naveed Ejaz;Salimur Choudhury","doi":"10.1109/TAI.2024.3394797","DOIUrl":"https://doi.org/10.1109/TAI.2024.3394797","url":null,"abstract":"Road infrastructure is essential for transportation safety and efficiency. However, the current methods for assessing road conditions, crucial for effective planning and maintenance, suffer from high costs, time-intensive procedures, infrequent data collection, and limited real-time capabilities. This article presents an efficient lightweight system to analyze road quality from video feeds in real time. The backbone of the system is EdgeFusionViT, a novel vision transformer (ViT)-based architecture that uses an attention-based late fusion mechanism. The proposed architecture outperforms lightweight convolutional neural network (CNN)-based and ViT-based models. Its practicality is demonstrated by its deployment on an edge device, the Nvidia Jetson Orin Nano, enabling real-time road analysis at 12 frames per second. EdgeFusionViT outperforms existing benchmarks, achieving an impressive accuracy of 89.76% on the road surface condition dataset (RSCD). Notably, the model maintains a commendable accuracy of 76.89% even when trained with only 2% of the dataset, demonstrating its robustness and efficiency. These findings highlight the system's potential in road infrastructure management. It aids in creating safer, more efficient transport systems through timely, accurate road condition assessments. The study sets a new benchmark and opens up possibilities for advanced machine learning in infrastructure management.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"4972-4983"},"PeriodicalIF":0.0,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-29DOI: 10.1109/TAI.2024.3394795
Jing-Xiao Liao;Bo-Jian Hou;Hang-Cheng Dong;Hao Zhang;Xiaoge Zhang;Jinwei Sun;Shiping Zhang;Feng-Lei Fan
Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. To our best knowledge, it is the first heterogeneous autoencoder that is made of different types of neurons. Next, we apply the proposed heterogeneous autoencoder to unsupervised anomaly detection (AD) for tabular data and bearing fault signals. The AD faces difficulties such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability, which is suitable for the proposed heterogeneous autoencoder. Its high feature representation ability can characterize a variety of anomaly data (heterogeneity), discriminate the anomaly from the normal (unnoticeability), and accurately learn the distribution of normal samples (unknownness). Experiments show that heterogeneous autoencoders perform competitively compared with other state-of-the-art models.
{"title":"Quadratic Neuron-Empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection","authors":"Jing-Xiao Liao;Bo-Jian Hou;Hang-Cheng Dong;Hao Zhang;Xiaoge Zhang;Jinwei Sun;Shiping Zhang;Feng-Lei Fan","doi":"10.1109/TAI.2024.3394795","DOIUrl":"https://doi.org/10.1109/TAI.2024.3394795","url":null,"abstract":"Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. To our best knowledge, it is the first heterogeneous autoencoder that is made of different types of neurons. Next, we apply the proposed heterogeneous autoencoder to unsupervised anomaly detection (AD) for tabular data and bearing fault signals. The AD faces difficulties such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability, which is suitable for the proposed heterogeneous autoencoder. Its high feature representation ability can characterize a variety of anomaly data (heterogeneity), discriminate the anomaly from the normal (unnoticeability), and accurately learn the distribution of normal samples (unknownness). Experiments show that heterogeneous autoencoders perform competitively compared with other state-of-the-art models.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4723-4737"},"PeriodicalIF":0.0,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-18DOI: 10.1109/TAI.2024.3391230
Yan Wang;Xiaoyan Sun;Yong Zhang;Dunwei Gong;Hejuan Hu;Mingcheng Zuo
It is very challenging to autonomously generate algorithms suitable for constrained multiobjective optimization problems due to the diverse performance of existing algorithms. In this article, we propose a linear regression (LR)-based autonomous intelligent optimization method. It first extracts typical features of a constrained multiobjective optimization problem by focused sampling to form a feature vector. Then, a LR model is designed to learn the relationship between optimization problems and intelligent optimization algorithms (IOAs). Finally, the trained model autonomously generates a suitable IOA by inputting the feature vector. The proposed method is applied to six constrained multiobjective benchmark test sets with various characteristics and compared with seven popular optimization algorithms. The experimental results verify the effectiveness of the proposed method. In addition, the proposed method is used to solve the operation optimization problems of an integrated coal mine energy system, and the experimental results show its practicability.
由于现有算法的性能参差不齐，要自主生成适用于受限多目标优化问题的算法非常具有挑战性。本文提出了一种基于线性回归（LR）的自主智能优化方法。它首先通过集中采样提取约束多目标优化问题的典型特征，形成特征向量。然后，设计一个 LR 模型来学习优化问题与智能优化算法（IOA）之间的关系。最后，训练有素的模型通过输入特征向量自主生成合适的 IOA。所提出的方法被应用于六个具有不同特征的受限多目标基准测试集，并与七种流行的优化算法进行了比较。实验结果验证了所提方法的有效性。此外，还将所提方法用于解决煤矿综合能源系统的运行优化问题，实验结果表明了该方法的实用性。
{"title":"Linear Regression-Based Autonomous Intelligent Optimization for Constrained Multiobjective Problems","authors":"Yan Wang;Xiaoyan Sun;Yong Zhang;Dunwei Gong;Hejuan Hu;Mingcheng Zuo","doi":"10.1109/TAI.2024.3391230","DOIUrl":"https://doi.org/10.1109/TAI.2024.3391230","url":null,"abstract":"It is very challenging to autonomously generate algorithms suitable for constrained multiobjective optimization problems due to the diverse performance of existing algorithms. In this article, we propose a linear regression (LR)-based autonomous intelligent optimization method. It first extracts typical features of a constrained multiobjective optimization problem by focused sampling to form a feature vector. Then, a LR model is designed to learn the relationship between optimization problems and intelligent optimization algorithms (IOAs). Finally, the trained model autonomously generates a suitable IOA by inputting the feature vector. The proposed method is applied to six constrained multiobjective benchmark test sets with various characteristics and compared with seven popular optimization algorithms. The experimental results verify the effectiveness of the proposed method. In addition, the proposed method is used to solve the operation optimization problems of an integrated coal mine energy system, and the experimental results show its practicability.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4620-4634"},"PeriodicalIF":0.0,"publicationDate":"2024-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：