Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455311
Dengyong Zhang;Ruiyi He;Xin Liao;Feng Li;Jiaxin Chen;Gaobo Yang
Deepfake detection has gained increasing research attention in media forensics, and a variety of works have been produced. However, subtle artifacts might be eliminated by compression, and the convolutional neural networks (CNNs)-based detectors are invalidated for fake face images with compression. In this work, we propose a two-stream network for deepfake detection. We observed that high-frequency noise features and spatial features are inherently complementary to each other. Thus, both spatial features and high-frequency noise features are exploited for face forgery detection. Specifically, we design a double-frequency transformer module (DFTM) to guide the learning of spatial features from local artifact regions. To effectively fuse spatial features and high-frequency noise features, a dual-domain attention fusion module (DDAFM) is designed. We also introduce a local relationship constraint loss, which requires only image-level labels, for model training. We evaluate the proposed approach on five large-scale benchmark datasets, and extensive experimental results demonstrate the proposed approach outperforms most SOTA works.
在媒体取证领域，深度伪造检测受到越来越多的研究关注，各种研究成果层出不穷。然而，压缩可能会消除细微的伪影，基于卷积神经网络（CNN）的检测器在压缩后对假脸图像的检测无效。在这项工作中，我们提出了一种双流网络深度检假技术。我们发现，高频噪声特征和空间特征在本质上是互补的。因此，空间特征和高频噪声特征都可用于人脸伪造检测。具体来说，我们设计了一个双频变压器模块（DFTM）来引导从局部伪造区域学习空间特征。为了有效融合空间特征和高频噪声特征，我们设计了双域注意力融合模块（DDAFM）。我们还为模型训练引入了局部关系约束损失，它只需要图像级标签。我们在五个大型基准数据集上对所提出的方法进行了评估，大量实验结果表明所提出的方法优于大多数 SOTA 作品。
{"title":"Face Forgery Detection Based on Fine-Grained Clues and Noise Inconsistency","authors":"Dengyong Zhang;Ruiyi He;Xin Liao;Feng Li;Jiaxin Chen;Gaobo Yang","doi":"10.1109/TAI.2024.3455311","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455311","url":null,"abstract":"Deepfake detection has gained increasing research attention in media forensics, and a variety of works have been produced. However, subtle artifacts might be eliminated by compression, and the convolutional neural networks (CNNs)-based detectors are invalidated for fake face images with compression. In this work, we propose a two-stream network for deepfake detection. We observed that high-frequency noise features and spatial features are inherently complementary to each other. Thus, both spatial features and high-frequency noise features are exploited for face forgery detection. Specifically, we design a double-frequency transformer module (DFTM) to guide the learning of spatial features from local artifact regions. To effectively fuse spatial features and high-frequency noise features, a dual-domain attention fusion module (DDAFM) is designed. We also introduce a local relationship constraint loss, which requires only image-level labels, for model training. We evaluate the proposed approach on five large-scale benchmark datasets, and extensive experimental results demonstrate the proposed approach outperforms most SOTA works.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"144-158"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455313
Leena Heistrene;Juri Belikov;Dmitry Baimel;Liran Katzir;Ram Machlev;Kfir Levy;Shie Mannor;Yoash Levron
Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this article demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for nonexpert users such as bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios such as price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.
{"title":"An Improved and Explainable Electricity Price Forecasting Model via SHAP-Based Error Compensation Approach","authors":"Leena Heistrene;Juri Belikov;Dmitry Baimel;Liran Katzir;Ram Machlev;Kfir Levy;Shie Mannor;Yoash Levron","doi":"10.1109/TAI.2024.3455313","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455313","url":null,"abstract":"Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this article demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for nonexpert users such as bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios such as price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"159-168"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455308
Ashley S. Dale;Lauren Christopher
A trustworthy artificial intelligence (AI) model should be robust to perturbed data, where robustness correlates with the dimensionality and linearity of feature representations in the model latent space. Existing methods for evaluating feature representations in the latent space are restricted to white-box models. In this work, we introduce direct adversarial latent estimation (DALE) for evaluating the robustness of feature representations and decision boundaries for target black-box models. A surrogate latent space is created using a variational autoencoder (VAE) trained on a disjoint dataset from an object classification backbone, then the VAE latent space is traversed to create sets of adversarial images. An object classification model is trained using transfer learning on the VAE image reconstructions, then classifies instances in the adversarial image set. We propose that the number of times the classification changes in an image set indicates the complexity of the decision boundaries in the classifier latent space; more complex decision boundaries are found to be more robust. This is confirmed by comparing the DALE distributions to the degradation of the classifier F1 scores in the presence of adversarial attacks. This work enables the first comparisons of latent-space complexity between black box models by relating model robustness to complex decision boundaries.
{"title":"Direct Adversarial Latent Estimation to Evaluate Decision Boundary Complexity in Black Box Models","authors":"Ashley S. Dale;Lauren Christopher","doi":"10.1109/TAI.2024.3455308","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455308","url":null,"abstract":"A trustworthy artificial intelligence (AI) model should be robust to perturbed data, where robustness correlates with the dimensionality and linearity of feature representations in the model latent space. Existing methods for evaluating feature representations in the latent space are restricted to white-box models. In this work, we introduce \u0000<italic>direct adversarial latent estimation</i>\u0000 (DALE) for evaluating the robustness of feature representations and decision boundaries for target black-box models. A surrogate latent space is created using a variational autoencoder (VAE) trained on a disjoint dataset from an object classification backbone, then the VAE latent space is traversed to create sets of adversarial images. An object classification model is trained using transfer learning on the VAE image reconstructions, then classifies instances in the adversarial image set. We propose that the number of times the classification changes in an image set indicates the complexity of the decision boundaries in the classifier latent space; more complex decision boundaries are found to be more robust. This is confirmed by comparing the DALE distributions to the degradation of the classifier F1 scores in the presence of adversarial attacks. This work enables the first comparisons of latent-space complexity between black box models by relating model robustness to complex decision boundaries.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6043-6053"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multiple instance learning (MIL), a powerful strategy for weakly supervised learning, is able to perform various prediction tasks on gigapixel whole slide images (WSIs). However, the tens of thousands of patches in WSIs usually incur a vast computational burden for image augmentation, limiting the performance improvement in MIL. Currently, the feature augmentation-based MIL framework is a promising solution, while existing methods such as mixup often produce unrealistic features. To explore a more efficient and practical augmentation method, we introduce the diffusion model (DM) into MIL for the first time and propose a feature augmentation framework called AugDiff. The diverse generation capabilities of DM guarantee a various range of feature augmentations, while its iterative generation approach effectively preserves semantic integrity during these augmentations. We conduct extensive experiments over four distinct cancer datasets, two different feature extractors, and three prevalent MIL algorithms to evaluate the performance of AugDiff. Ablation study and visualization further verify the effectiveness. Moreover, we highlight AugDiff's higher quality augmented feature over image augmentation and its superiority over self-supervised learning. The generalization over external datasets indicates its broader applications. The code is open-sourced on https://github.com/szc19990412/AugDiff .
多实例学习（MIL）是一种强大的弱监督学习策略，能够在千兆像素的整张幻灯片图像（WSI）上执行各种预测任务。然而，WSI 中数以万计的斑块通常会给图像增强带来巨大的计算负担，从而限制了 MIL 性能的提高。目前，基于特征增强的 MIL 框架是一种很有前景的解决方案，而现有的方法（如 mixup）往往会产生不切实际的特征。为了探索一种更高效、更实用的增强方法，我们首次在 MIL 中引入了扩散模型（DM），并提出了名为 AugDiff 的特征增强框架。DM 多样化的生成能力保证了各种特征增强，而其迭代生成方法在这些增强过程中有效地保持了语义的完整性。我们在四个不同的癌症数据集、两种不同的特征提取器和三种流行的 MIL 算法上进行了广泛的实验，以评估 AugDiff 的性能。消融研究和可视化进一步验证了其有效性。此外，我们还强调了 AugDiff 比图像增强具有更高质量的增强特征，而且比自我监督学习更具优势。在外部数据集上的泛化表明其应用范围更广。代码开源于 https://github.com/szc19990412/AugDiff。
{"title":"AugDiff: Diffusion-Based Feature Augmentation for Multiple Instance Learning in Whole Slide Image","authors":"Zhuchen Shao;Liuxi Dai;Yifeng Wang;Haoqian Wang;Yongbing Zhang","doi":"10.1109/TAI.2024.3454591","DOIUrl":"https://doi.org/10.1109/TAI.2024.3454591","url":null,"abstract":"Multiple instance learning (MIL), a powerful strategy for weakly supervised learning, is able to perform various prediction tasks on gigapixel whole slide images (WSIs). However, the tens of thousands of patches in WSIs usually incur a vast computational burden for image augmentation, limiting the performance improvement in MIL. Currently, the feature augmentation-based MIL framework is a promising solution, while existing methods such as mixup often produce unrealistic features. To explore a more efficient and practical augmentation method, we introduce the diffusion model (DM) into MIL for the first time and propose a feature augmentation framework called AugDiff. The diverse generation capabilities of DM guarantee a various range of feature augmentations, while its iterative generation approach effectively preserves semantic integrity during these augmentations. We conduct extensive experiments over four distinct cancer datasets, two different feature extractors, and three prevalent MIL algorithms to evaluate the performance of AugDiff. Ablation study and visualization further verify the effectiveness. Moreover, we highlight AugDiff's higher quality augmented feature over image augmentation and its superiority over self-supervised learning. The generalization over external datasets indicates its broader applications. The code is open-sourced on \u0000<uri>https://github.com/szc19990412/AugDiff</uri>\u0000.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6617-6628"},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-05DOI: 10.1109/TAI.2024.3454566
Kristina Telegraph;Christos Kyrkou
This work presents advancements in multiclass vehicle detection using unmanned aerial vehicle (UAV) cameras through the development of spatiotemporal object detection models. The study introduces a spatiotemporal vehicle detection dataset (STVD) containing $6600$ annotated sequential frame images captured by UAVs, enabling comprehensive training and evaluation of algorithms for holistic spatiotemporal perception. A YOLO-based object detection algorithm is enhanced to incorporate temporal dynamics, resulting in improved performance over single frame models. The integration of attention mechanisms into spatiotemporal models is shown to further enhance performance. Experimental validation demonstrates significant progress, with the best spatiotemporal model exhibiting a 16.22% improvement over single frame models, while it is demonstrated that attention mechanisms hold the potential for additional performance gains.
{"title":"Spatiotemporal Object Detection for Improved Aerial Vehicle Detection in Traffic Monitoring","authors":"Kristina Telegraph;Christos Kyrkou","doi":"10.1109/TAI.2024.3454566","DOIUrl":"https://doi.org/10.1109/TAI.2024.3454566","url":null,"abstract":"This work presents advancements in multiclass vehicle detection using unmanned aerial vehicle (UAV) cameras through the development of spatiotemporal object detection models. The study introduces a spatiotemporal vehicle detection dataset (STVD) containing \u0000<inline-formula><tex-math>$6600$</tex-math></inline-formula>\u0000 annotated sequential frame images captured by UAVs, enabling comprehensive training and evaluation of algorithms for holistic spatiotemporal perception. A YOLO-based object detection algorithm is enhanced to incorporate temporal dynamics, resulting in improved performance over single frame models. The integration of attention mechanisms into spatiotemporal models is shown to further enhance performance. Experimental validation demonstrates significant progress, with the best spatiotemporal model exhibiting a 16.22% improvement over single frame models, while it is demonstrated that attention mechanisms hold the potential for additional performance gains.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6159-6171"},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent years have witnessed spiking neural networks (SNNs) gaining attention for their ultra-low energy consumption and high biological plausibility compared with traditional artificial neural networks (ANNs). Despite their distinguished properties, the application of SNNs in the computationally intensive field of image generation is still under exploration. In this article, we propose the spiking diffusion models (SDMs), an innovative family of SNN-based generative models that excel in producing high-quality samples with significantly reduced energy consumption. In particular, we propose a temporal-wise spiking mechanism (TSM) that allows SNNs to capture more temporal features from a bio-plasticity perspective. In addition, we propose a threshold-guided strategy that can further improve the performances by up to 16.7% without any additional training. We also make the first attempt to use the ANN-SNN approach for SNN-based generation tasks. Extensive experimental results reveal that our approach not only exhibits comparable performance to its ANN counterpart with few spiking time steps, but also outperforms previous SNN-based generative models by a large margin. Moreover, we also demonstrate the high-quality generation ability of SDM on large-scale datasets, e.g., LSUN bedroom. This development marks a pivotal advancement in the capabilities of SNN-based generation, paving the way for future research avenues to realize low-energy and low-latency generative applications.
{"title":"Spiking Diffusion Models","authors":"Jiahang Cao;Hanzhong Guo;Ziqing Wang;Deming Zhou;Hao Cheng;Qiang Zhang;Renjing Xu","doi":"10.1109/TAI.2024.3453229","DOIUrl":"https://doi.org/10.1109/TAI.2024.3453229","url":null,"abstract":"Recent years have witnessed spiking neural networks (SNNs) gaining attention for their ultra-low energy consumption and high biological plausibility compared with traditional artificial neural networks (ANNs). Despite their distinguished properties, the application of SNNs in the computationally intensive field of image generation is still under exploration. In this article, we propose the spiking diffusion models (SDMs), an innovative family of SNN-based generative models that excel in producing high-quality samples with significantly reduced energy consumption. In particular, we propose a temporal-wise spiking mechanism (TSM) that allows SNNs to capture more temporal features from a bio-plasticity perspective. In addition, we propose a threshold-guided strategy that can further improve the performances by up to 16.7% without any additional training. We also make the first attempt to use the ANN-SNN approach for SNN-based generation tasks. Extensive experimental results reveal that our approach not only exhibits comparable performance to its ANN counterpart with few spiking time steps, but also outperforms previous SNN-based generative models by a large margin. Moreover, we also demonstrate the high-quality generation ability of SDM on large-scale datasets, e.g., LSUN bedroom. This development marks a pivotal advancement in the capabilities of SNN-based generation, paving the way for future research avenues to realize low-energy and low-latency generative applications.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"132-143"},"PeriodicalIF":0.0,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Since most multiobjective optimization problems in real-world applications contain constraints, constraint-handling techniques (CHTs) are necessary for a multiobjective optimizer. However, existing CHTs give no relaxation to objectives, resulting in the elimination of infeasible dominated solutions that are promising (potentially useful but inferior) for detecting feasible regions and the constrained Pareto front (CPF). To overcome this drawback, in this work, we propose an objective relaxation technique that can preserve promising by relaxing objective function values, i.e., convergence, through an adaptively adjusted relaxation factor. Further, we develop a new constrained multiobjective optimization evolutionary algorithm (CMOEA) based on relaxations on both constraints and objectives. The proposed algorithm evolves one population by the constraint relaxation technique to preserve promising infeasible solutions and the other population by both objective and constraint relaxation techniques to preserve promising infeasible dominated solutions. In this way, our method can overcome the drawback of existing CHTs. Besides, an archive update strategy is designed to maintain encountered feasible solutions by the two populations to approximate the CPF. Experiments on challenging benchmark problems and real-world problems have demonstrated the superiority or at least competitiveness of our proposed CMOEA. Moreover, to verify the generality of the objective relaxation technique, we embed it into two existing CMOEA frameworks and the results show that it can significantly improve the performance in handling challenging problems.
{"title":"Constrained Multiobjective Optimization via Relaxations on Both Constraints and Objectives","authors":"Fei Ming;Bing Xue;Mengjie Zhang;Wenyin Gong;Huixiang Zhen","doi":"10.1109/TAI.2024.3454025","DOIUrl":"https://doi.org/10.1109/TAI.2024.3454025","url":null,"abstract":"Since most multiobjective optimization problems in real-world applications contain constraints, constraint-handling techniques (CHTs) are necessary for a multiobjective optimizer. However, existing CHTs give no relaxation to objectives, resulting in the elimination of infeasible dominated solutions that are promising (potentially useful but inferior) for detecting feasible regions and the constrained Pareto front (CPF). To overcome this drawback, in this work, we propose an objective relaxation technique that can preserve promising by relaxing objective function values, i.e., convergence, through an adaptively adjusted relaxation factor. Further, we develop a new constrained multiobjective optimization evolutionary algorithm (CMOEA) based on relaxations on both constraints and objectives. The proposed algorithm evolves one population by the constraint relaxation technique to preserve promising infeasible solutions and the other population by both objective and constraint relaxation techniques to preserve promising infeasible dominated solutions. In this way, our method can overcome the drawback of existing CHTs. Besides, an archive update strategy is designed to maintain encountered feasible solutions by the two populations to approximate the CPF. Experiments on challenging benchmark problems and real-world problems have demonstrated the superiority or at least competitiveness of our proposed CMOEA. Moreover, to verify the generality of the objective relaxation technique, we embed it into two existing CMOEA frameworks and the results show that it can significantly improve the performance in handling challenging problems.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6709-6722"},"PeriodicalIF":0.0,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-03DOI: 10.1109/TAI.2024.3453230
Cristiano da Costa Cunha;Wei Liu;Tim French;Ajmal Mian
We present Q-Cogni , an algorithmically integrated causal reinforcement learning framework that redesigns Q-Learning to improve the learning process with causal inference. Q-Cogni achieves improved policy quality and learning efficiency with a prelearned structural causal model of the environment, queried to guide the policy learning process with an understanding of cause-and-effect relationships in a state-action space. By doing so, we not only leverage the sample efficient techniques of reinforcement learning but also enable reasoning about a broader set of policies and bring higher degrees of interpretability to decisions made by the reinforcement learning agent. We apply Q-Cogni on vehicle routing problem (VRP) environments including a real-world dataset of taxis in New York City using the Taxi and Limousine Commission trip record data. We show Q-Cogni's capability to achieve an optimally guaranteed policy (total trip distance) in 76% of the cases when comparing to shortest-path-search methods and outperforming (shorter distances) state-of-the-art reinforcement learning algorithms in 66% of cases. Additionally, since Q-Cogni does not require a complete global map, we show that it can start efficiently routing with partial information and improve as more data is collected, such as traffic disruptions and changes in destination, making it ideal for deployment in real-world dynamic settings.
{"title":"Q-Cogni: An Integrated Causal Reinforcement Learning Framework","authors":"Cristiano da Costa Cunha;Wei Liu;Tim French;Ajmal Mian","doi":"10.1109/TAI.2024.3453230","DOIUrl":"https://doi.org/10.1109/TAI.2024.3453230","url":null,"abstract":"We present \u0000<italic>Q-Cogni</i>\u0000, an algorithmically integrated causal reinforcement learning framework that redesigns \u0000<italic>Q-Learning</i>\u0000 to improve the learning process with causal inference. \u0000<italic>Q-Cogni</i>\u0000 achieves improved policy quality and learning efficiency with a prelearned structural causal model of the environment, queried to guide the policy learning process with an understanding of cause-and-effect relationships in a state-action space. By doing so, we not only leverage the sample efficient techniques of reinforcement learning but also enable reasoning about a broader set of policies and bring higher degrees of interpretability to decisions made by the reinforcement learning agent. We apply \u0000<italic>Q-Cogni</i>\u0000 on vehicle routing problem (VRP) environments including a real-world dataset of taxis in New York City using the Taxi and Limousine Commission trip record data. We show \u0000<italic>Q-Cogni's</i>\u0000 capability to achieve an optimally guaranteed policy (total trip distance) in 76% of the cases when comparing to shortest-path-search methods and outperforming (shorter distances) state-of-the-art reinforcement learning algorithms in 66% of cases. Additionally, since \u0000<italic>Q-Cogni</i>\u0000 does not require a complete global map, we show that it can start efficiently routing with partial information and improve as more data is collected, such as traffic disruptions and changes in destination, making it ideal for deployment in real-world dynamic settings.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6186-6195"},"PeriodicalIF":0.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-30DOI: 10.1109/TAI.2024.3452678
Yifan Hu;Junjie Fu;Guanghui Wen;Changyin Sun
Learning efficient coordination policies over continuous state and action spaces remains a huge challenge for existing distributed multi-agent reinforcement learning (MARL) algorithms. In this article, the classic deterministic policy gradient (DPG) method is extended to the distributed MARL domain to handle the continuous control policy learning issue for a team of homogeneous agents connected through a directed graph. A theoretical on-policy distributed actor–critic algorithm is first proposed based on a local DPG theorem, which considers observation-based policies, and incorporates consensus updates for the critic and actor parameters. Stochastic approximation theory is then used to obtain asymptotic convergence results of the algorithm under standard assumptions. Thereafter, a practical distributed deterministic actor–critic algorithm is proposed by integrating the theoretical algorithm with the deep reinforcement learning training architecture, which achieves better scalability, exploration ability, and data efficiency. Simulations are carried out in standard MARL environments with continuous action spaces, where the results demonstrate that the proposed distributed algorithm achieves comparable learning performance to solid centralized trained baselines while demanding much less communication resources.
{"title":"Policy Consensus-Based Distributed Deterministic Multi-Agent Reinforcement Learning Over Directed Graphs","authors":"Yifan Hu;Junjie Fu;Guanghui Wen;Changyin Sun","doi":"10.1109/TAI.2024.3452678","DOIUrl":"https://doi.org/10.1109/TAI.2024.3452678","url":null,"abstract":"Learning efficient coordination policies over continuous state and action spaces remains a huge challenge for existing distributed multi-agent reinforcement learning (MARL) algorithms. In this article, the classic deterministic policy gradient (DPG) method is extended to the distributed MARL domain to handle the continuous control policy learning issue for a team of homogeneous agents connected through a directed graph. A theoretical on-policy distributed actor–critic algorithm is first proposed based on a local DPG theorem, which considers observation-based policies, and incorporates consensus updates for the critic and actor parameters. Stochastic approximation theory is then used to obtain asymptotic convergence results of the algorithm under standard assumptions. Thereafter, a practical distributed deterministic actor–critic algorithm is proposed by integrating the theoretical algorithm with the deep reinforcement learning training architecture, which achieves better scalability, exploration ability, and data efficiency. Simulations are carried out in standard MARL environments with continuous action spaces, where the results demonstrate that the proposed distributed algorithm achieves comparable learning performance to solid centralized trained baselines while demanding much less communication resources.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"118-131"},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Depth images are often used to improve the geometric understanding of scenes owing to their intuitive distance properties. Although there have been significant advancements in semantic segmentation tasks using red–green–blue-depth (RGB-D) images, the complexity of existing methods remains high. Furthermore, the requirement for high-quality depth images increases the model inference time, which limits the practicality of these methods. To address this issue, we propose a feature-implicit mapping knowledge distillation (FIMKD) method and a cross-modal knowledge distillation (KD) architecture to leverage deep modal information for training and reduce the model dependence on this information during inference. The approach comprises two networks: FIMKD-T, a teacher network that uses RGB-D data, and FIMKD-S, a student network that uses only RGB data. FIMKD-T extracts high-frequency information using the depth modality and compensates for the loss of RGB details due to a reduction in resolution during feature extraction by the high-frequency feature enhancement module, thereby enhancing the geometric perception of semantic features. In contrast, the FIMKD-S network does not employ deep learning techniques; instead, it uses a nonlearning approach to extract high-frequency information. To enable the FIMKD-S network to learn deep features, we propose a feature-implicit mapping KD for feature distillation. This mapping technique maps the features in channel and space to a low-dimensional hidden layer, which helps to avoid inefficient single-pattern student learning. We evaluated the proposed FIMKD-S* (FIMKD-S with KD) on the NYUv2 and SUN-RGBD datasets. The results demonstrate that both FIMKD-T and FIMKD-S* achieve state-of-the-art performance. Furthermore, FIMKD-S* provides the best performance balance.
深度图像由于其直观的距离属性，经常被用来提高对场景的几何理解。尽管使用红-绿-蓝-深（RGB-D）图像的语义分割任务已经取得了重大进展，但现有方法的复杂性仍然很高。此外，对高质量深度图像的要求增加了模型推理时间，限制了这些方法的实用性。为了解决这个问题，我们提出了一种特征隐式映射知识蒸馏（FIMKD）方法和一种跨模态知识蒸馏（KD）架构来利用深度模态信息进行训练，并在推理过程中减少模型对这些信息的依赖。该方法包括两个网络：FIMKD-T，一个使用RGB- d数据的教师网络，和FIMKD-S，一个只使用RGB数据的学生网络。FIMKD-T利用深度模态提取高频信息，补偿高频特征增强模块在特征提取过程中由于分辨率降低而导致的RGB细节损失，从而增强语义特征的几何感知。相比之下，FIMKD-S网络不采用深度学习技术；相反，它使用非学习方法来提取高频信息。为了使FIMKD-S网络能够学习深度特征，我们提出了一种用于特征蒸馏的特征隐式映射KD。这种映射技术将通道和空间中的特征映射到一个低维的隐藏层，有助于避免低效的单模式学生学习。我们在NYUv2和SUN-RGBD数据集上对提出的FIMKD-S* (FIMKD-S with KD)进行了评估。结果表明，FIMKD-T和FIMKD-S*都达到了最先进的性能。此外，FIMKD-S*提供了最佳的性能平衡。
{"title":"FIMKD: Feature-Implicit Mapping Knowledge Distillation for RGB-D Indoor Scene Semantic Segmentation","authors":"Wujie Zhou;Yuxiang Xiao;Yuanyuan Liu;Qiuping Jiang","doi":"10.1109/TAI.2024.3452052","DOIUrl":"https://doi.org/10.1109/TAI.2024.3452052","url":null,"abstract":"Depth images are often used to improve the geometric understanding of scenes owing to their intuitive distance properties. Although there have been significant advancements in semantic segmentation tasks using red–green–blue-depth (RGB-D) images, the complexity of existing methods remains high. Furthermore, the requirement for high-quality depth images increases the model inference time, which limits the practicality of these methods. To address this issue, we propose a feature-implicit mapping knowledge distillation (FIMKD) method and a cross-modal knowledge distillation (KD) architecture to leverage deep modal information for training and reduce the model dependence on this information during inference. The approach comprises two networks: FIMKD-T, a teacher network that uses RGB-D data, and FIMKD-S, a student network that uses only RGB data. FIMKD-T extracts high-frequency information using the depth modality and compensates for the loss of RGB details due to a reduction in resolution during feature extraction by the high-frequency feature enhancement module, thereby enhancing the geometric perception of semantic features. In contrast, the FIMKD-S network does not employ deep learning techniques; instead, it uses a nonlearning approach to extract high-frequency information. To enable the FIMKD-S network to learn deep features, we propose a feature-implicit mapping KD for feature distillation. This mapping technique maps the features in channel and space to a low-dimensional hidden layer, which helps to avoid inefficient single-pattern student learning. We evaluated the proposed FIMKD-S* (FIMKD-S with KD) on the NYUv2 and SUN-RGBD datasets. The results demonstrate that both FIMKD-T and FIMKD-S* achieve state-of-the-art performance. Furthermore, FIMKD-S* provides the best performance balance.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6488-6499"},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810286","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：