This article presents the results of an extensive study investigating the opinions on artificial intelligence (AI) of a sample of 4006 European citizens from eight distinct countries (France, Germany, Italy, Netherlands, Poland, Romania, Spain, and Sweden). The aim of the study is to gain a better understanding of people's views and perceptions within the European context, which is already marked by important policy actions and regulatory processes. To survey the perceptions of the citizens of Europe, we design and validate a new questionnaire (PAICE) structured around three dimensions: people's awareness, attitude, and trust. We observe that while awareness is characterized by a low level of self-assessed competency, the attitude toward AI is very positive for more than half of the population. Reflecting on the collected results, we highlight implicit contradictions and identify trends that may interfere with the creation of an ecosystem of trust and the development of inclusive AI policies. The introduction of rules that ensure legal and ethical standards, along with the activity of high-level educational entities, and the promotion of AI literacy are identified as key factors in supporting a trustworthy AI ecosystem. We make some recommendations for AI governance focused on the European context and conclude with suggestions for future work.
{"title":"Artificial Intelligence Across Europe: A Study on Awareness, Attitude and Trust","authors":"Teresa Scantamburlo;Atia Cortés;Francesca Foffano;Cristian Barrué;Veronica Distefano;Long Pham;Alessandro Fabris","doi":"10.1109/TAI.2024.3461633","DOIUrl":"https://doi.org/10.1109/TAI.2024.3461633","url":null,"abstract":"This article presents the results of an extensive study investigating the opinions on artificial intelligence (AI) of a sample of 4006 European citizens from eight distinct countries (France, Germany, Italy, Netherlands, Poland, Romania, Spain, and Sweden). The aim of the study is to gain a better understanding of people's views and perceptions within the European context, which is already marked by important policy actions and regulatory processes. To survey the perceptions of the citizens of Europe, we design and validate a new questionnaire (PAICE) structured around three dimensions: people's awareness, attitude, and trust. We observe that while awareness is characterized by a low level of self-assessed competency, the attitude toward AI is very positive for more than half of the population. Reflecting on the collected results, we highlight implicit contradictions and identify trends that may interfere with the creation of an ecosystem of trust and the development of inclusive AI policies. The introduction of rules that ensure legal and ethical standards, along with the activity of high-level educational entities, and the promotion of AI literacy are identified as key factors in supporting a trustworthy AI ecosystem. We make some recommendations for AI governance focused on the European context and conclude with suggestions for future work.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 2","pages":"477-490"},"PeriodicalIF":0.0,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10681325","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143535493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-11DOI: 10.1109/TAI.2024.3457931
Fei Teng;Jiaming Zhang;Kunyu Peng;Yaonan Wang;Rainer Stiefelhagen;Kailun Yang
Light field cameras are capable of capturing intricate angular and spatial details. This allows for acquiring complex light patterns and details from multiple angles, significantly enhancing the precision of image semantic segmentation. However, two significant issues arise: 1) The extensive angular information of light field cameras contains a large amount of redundant data, which is overwhelming for the limited hardware resources of intelligent agents. 2) A relative displacement difference exists in the data collected by different microlenses. To address these issues, we propose an omni-aperture fusion model (OAFuser) that leverages dense context from the central view and extracts the angular information from subaperture images to generate semantically consistent results. To simultaneously streamline the redundant information from the light field cameras and avoid feature loss during network propagation, we present a simple yet very effective subaperture fusion module (SAFM) . This module efficiently embeds subaperture images in angular features, allowing the network to process each subaperture image with a minimal computational demand of only ( ${sim}1rm GFlops$ ). Furthermore, to address the mismatched spatial information across viewpoints, we present a center angular rectification module (CARM) to realize feature resorting and prevent feature occlusion caused by misalignment. The proposed OAFuser achieves state-of-the-art performance on four UrbanLF datasets in terms of all evaluation metrics and sets a new record of $84.93%$ in mIoU on the UrbanLF-Real Extended dataset, with a gain of ${+}3.69%$ . The source code for OAFuser is available at https://github.com/FeiBryantkit/OAFuser .
{"title":"OAFuser: Toward Omni-Aperture Fusion for Light Field Semantic Segmentation","authors":"Fei Teng;Jiaming Zhang;Kunyu Peng;Yaonan Wang;Rainer Stiefelhagen;Kailun Yang","doi":"10.1109/TAI.2024.3457931","DOIUrl":"https://doi.org/10.1109/TAI.2024.3457931","url":null,"abstract":"Light field cameras are capable of capturing intricate angular and spatial details. This allows for acquiring complex light patterns and details from multiple angles, significantly enhancing the precision of image semantic segmentation. However, two significant issues arise: 1) The extensive angular information of light field cameras contains a large amount of redundant data, which is overwhelming for the limited hardware resources of intelligent agents. 2) A relative displacement difference exists in the data collected by different microlenses. To address these issues, we propose an \u0000<italic>omni-aperture fusion model (OAFuser)</i>\u0000 that leverages dense context from the central view and extracts the angular information from subaperture images to generate semantically consistent results. To simultaneously streamline the redundant information from the light field cameras and avoid feature loss during network propagation, we present a simple yet very effective \u0000<italic>subaperture fusion module (SAFM)</i>\u0000. This module efficiently embeds subaperture images in angular features, allowing the network to process each subaperture image with a minimal computational demand of only (\u0000<inline-formula><tex-math>${sim}1rm GFlops$</tex-math></inline-formula>\u0000). Furthermore, to address the mismatched spatial information across viewpoints, we present a \u0000<italic>center angular rectification module (CARM)</i>\u0000 to realize feature resorting and prevent feature occlusion caused by misalignment. The proposed OAFuser achieves state-of-the-art performance on four UrbanLF datasets in terms of \u0000<italic>all evaluation metrics</i>\u0000 and sets a new record of \u0000<inline-formula><tex-math>$84.93%$</tex-math></inline-formula>\u0000 in mIoU on the UrbanLF-Real Extended dataset, with a gain of \u0000<inline-formula><tex-math>${+}3.69%$</tex-math></inline-formula>\u0000. The source code for OAFuser is available at \u0000<uri>https://github.com/FeiBryantkit/OAFuser</uri>\u0000.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6225-6239"},"PeriodicalIF":0.0,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-10DOI: 10.1109/TAI.2024.3439048
Hussein Abbass;Keeley Crockett;Jonathan Garibaldi;Alexander Gegov;Uzay Kaymak;Joao Miguel C. Sousa
{"title":"Editorial: From Explainable Artificial Intelligence (xAI) to Understandable Artificial Intelligence (uAI)","authors":"Hussein Abbass;Keeley Crockett;Jonathan Garibaldi;Alexander Gegov;Uzay Kaymak;Joao Miguel C. Sousa","doi":"10.1109/TAI.2024.3439048","DOIUrl":"https://doi.org/10.1109/TAI.2024.3439048","url":null,"abstract":"","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4310-4314"},"PeriodicalIF":0.0,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10673750","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142165008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-09DOI: 10.1109/TAI.2024.3456086
Fangfang Zhang;Gaofeng Shi;Yi Mei;Mengjie Zhang
Dynamic flexible job shop scheduling is an important combinatorial optimization problem that has rich real-world applications such as product processing in manufacturing. Genetic programming has been successfully used to learn scheduling heuristics for dynamic flexible job shop scheduling. Intuitively, users prefer small and effective scheduling heuristics that can not only generate promising schedules but also are computationally efficient and easy to be understood. However, a scheduling heuristic with better effectiveness tends to have a larger size, and the effectiveness of rules and rule size are potentially conflicting objectives. With the traditional dominance relation-based multiobjective algorithms, there is a search bias toward rule size, since rule size is much easier to optimized than effectiveness, and larger rules are easily abandoned, resulting in the loss of effectiveness. To address this issue, this article develops a novel multiobjective genetic programming algorithm that takes size and effectiveness of scheduling heuristics for optimization via multitask learning mechanism. Specifically, we construct two tasks for the multiobjective optimization with biased objectives using different search mechanisms for each task. The focus of the proposed algorithm is to improve the effectiveness of learned small rules by knowledge sharing between constructed tasks which is implemented with the crossover operator. The results show that our proposed algorithm performs significantly better, i.e., with smaller and more effective scheduling heuristics, than the state-of-the-art algorithms in the examined scenarios. By analyzing the population diversity, we find that the proposed algorithm has a good balance between exploration and exploitation during the evolutionary process.
{"title":"Multiobjective Dynamic Flexible Job Shop Scheduling With Biased Objectives via Multitask Genetic Programming","authors":"Fangfang Zhang;Gaofeng Shi;Yi Mei;Mengjie Zhang","doi":"10.1109/TAI.2024.3456086","DOIUrl":"https://doi.org/10.1109/TAI.2024.3456086","url":null,"abstract":"Dynamic flexible job shop scheduling is an important combinatorial optimization problem that has rich real-world applications such as product processing in manufacturing. Genetic programming has been successfully used to learn scheduling heuristics for dynamic flexible job shop scheduling. Intuitively, users prefer small and effective scheduling heuristics that can not only generate promising schedules but also are computationally efficient and easy to be understood. However, a scheduling heuristic with better effectiveness tends to have a larger size, and the effectiveness of rules and rule size are potentially conflicting objectives. With the traditional dominance relation-based multiobjective algorithms, there is a search bias toward rule size, since rule size is much easier to optimized than effectiveness, and larger rules are easily abandoned, resulting in the loss of effectiveness. To address this issue, this article develops a novel multiobjective genetic programming algorithm that takes size and effectiveness of scheduling heuristics for optimization via multitask learning mechanism. Specifically, we construct two tasks for the multiobjective optimization with biased objectives using different search mechanisms for each task. The focus of the proposed algorithm is to improve the effectiveness of learned small rules by knowledge sharing between constructed tasks which is implemented with the crossover operator. The results show that our proposed algorithm performs significantly better, i.e., with smaller and more effective scheduling heuristics, than the state-of-the-art algorithms in the examined scenarios. By analyzing the population diversity, we find that the proposed algorithm has a good balance between exploration and exploitation during the evolutionary process.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"169-183"},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455311
Dengyong Zhang;Ruiyi He;Xin Liao;Feng Li;Jiaxin Chen;Gaobo Yang
Deepfake detection has gained increasing research attention in media forensics, and a variety of works have been produced. However, subtle artifacts might be eliminated by compression, and the convolutional neural networks (CNNs)-based detectors are invalidated for fake face images with compression. In this work, we propose a two-stream network for deepfake detection. We observed that high-frequency noise features and spatial features are inherently complementary to each other. Thus, both spatial features and high-frequency noise features are exploited for face forgery detection. Specifically, we design a double-frequency transformer module (DFTM) to guide the learning of spatial features from local artifact regions. To effectively fuse spatial features and high-frequency noise features, a dual-domain attention fusion module (DDAFM) is designed. We also introduce a local relationship constraint loss, which requires only image-level labels, for model training. We evaluate the proposed approach on five large-scale benchmark datasets, and extensive experimental results demonstrate the proposed approach outperforms most SOTA works.
在媒体取证领域，深度伪造检测受到越来越多的研究关注，各种研究成果层出不穷。然而，压缩可能会消除细微的伪影，基于卷积神经网络（CNN）的检测器在压缩后对假脸图像的检测无效。在这项工作中，我们提出了一种双流网络深度检假技术。我们发现，高频噪声特征和空间特征在本质上是互补的。因此，空间特征和高频噪声特征都可用于人脸伪造检测。具体来说，我们设计了一个双频变压器模块（DFTM）来引导从局部伪造区域学习空间特征。为了有效融合空间特征和高频噪声特征，我们设计了双域注意力融合模块（DDAFM）。我们还为模型训练引入了局部关系约束损失，它只需要图像级标签。我们在五个大型基准数据集上对所提出的方法进行了评估，大量实验结果表明所提出的方法优于大多数 SOTA 作品。
{"title":"Face Forgery Detection Based on Fine-Grained Clues and Noise Inconsistency","authors":"Dengyong Zhang;Ruiyi He;Xin Liao;Feng Li;Jiaxin Chen;Gaobo Yang","doi":"10.1109/TAI.2024.3455311","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455311","url":null,"abstract":"Deepfake detection has gained increasing research attention in media forensics, and a variety of works have been produced. However, subtle artifacts might be eliminated by compression, and the convolutional neural networks (CNNs)-based detectors are invalidated for fake face images with compression. In this work, we propose a two-stream network for deepfake detection. We observed that high-frequency noise features and spatial features are inherently complementary to each other. Thus, both spatial features and high-frequency noise features are exploited for face forgery detection. Specifically, we design a double-frequency transformer module (DFTM) to guide the learning of spatial features from local artifact regions. To effectively fuse spatial features and high-frequency noise features, a dual-domain attention fusion module (DDAFM) is designed. We also introduce a local relationship constraint loss, which requires only image-level labels, for model training. We evaluate the proposed approach on five large-scale benchmark datasets, and extensive experimental results demonstrate the proposed approach outperforms most SOTA works.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"144-158"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455313
Leena Heistrene;Juri Belikov;Dmitry Baimel;Liran Katzir;Ram Machlev;Kfir Levy;Shie Mannor;Yoash Levron
Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this article demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for nonexpert users such as bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios such as price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.
{"title":"An Improved and Explainable Electricity Price Forecasting Model via SHAP-Based Error Compensation Approach","authors":"Leena Heistrene;Juri Belikov;Dmitry Baimel;Liran Katzir;Ram Machlev;Kfir Levy;Shie Mannor;Yoash Levron","doi":"10.1109/TAI.2024.3455313","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455313","url":null,"abstract":"Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this article demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for nonexpert users such as bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios such as price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"159-168"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/TAI.2024.3455308
Ashley S. Dale;Lauren Christopher
A trustworthy artificial intelligence (AI) model should be robust to perturbed data, where robustness correlates with the dimensionality and linearity of feature representations in the model latent space. Existing methods for evaluating feature representations in the latent space are restricted to white-box models. In this work, we introduce direct adversarial latent estimation (DALE) for evaluating the robustness of feature representations and decision boundaries for target black-box models. A surrogate latent space is created using a variational autoencoder (VAE) trained on a disjoint dataset from an object classification backbone, then the VAE latent space is traversed to create sets of adversarial images. An object classification model is trained using transfer learning on the VAE image reconstructions, then classifies instances in the adversarial image set. We propose that the number of times the classification changes in an image set indicates the complexity of the decision boundaries in the classifier latent space; more complex decision boundaries are found to be more robust. This is confirmed by comparing the DALE distributions to the degradation of the classifier F1 scores in the presence of adversarial attacks. This work enables the first comparisons of latent-space complexity between black box models by relating model robustness to complex decision boundaries.
{"title":"Direct Adversarial Latent Estimation to Evaluate Decision Boundary Complexity in Black Box Models","authors":"Ashley S. Dale;Lauren Christopher","doi":"10.1109/TAI.2024.3455308","DOIUrl":"https://doi.org/10.1109/TAI.2024.3455308","url":null,"abstract":"A trustworthy artificial intelligence (AI) model should be robust to perturbed data, where robustness correlates with the dimensionality and linearity of feature representations in the model latent space. Existing methods for evaluating feature representations in the latent space are restricted to white-box models. In this work, we introduce \u0000<italic>direct adversarial latent estimation</i>\u0000 (DALE) for evaluating the robustness of feature representations and decision boundaries for target black-box models. A surrogate latent space is created using a variational autoencoder (VAE) trained on a disjoint dataset from an object classification backbone, then the VAE latent space is traversed to create sets of adversarial images. An object classification model is trained using transfer learning on the VAE image reconstructions, then classifies instances in the adversarial image set. We propose that the number of times the classification changes in an image set indicates the complexity of the decision boundaries in the classifier latent space; more complex decision boundaries are found to be more robust. This is confirmed by comparing the DALE distributions to the degradation of the classifier F1 scores in the presence of adversarial attacks. This work enables the first comparisons of latent-space complexity between black box models by relating model robustness to complex decision boundaries.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6043-6053"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multiple instance learning (MIL), a powerful strategy for weakly supervised learning, is able to perform various prediction tasks on gigapixel whole slide images (WSIs). However, the tens of thousands of patches in WSIs usually incur a vast computational burden for image augmentation, limiting the performance improvement in MIL. Currently, the feature augmentation-based MIL framework is a promising solution, while existing methods such as mixup often produce unrealistic features. To explore a more efficient and practical augmentation method, we introduce the diffusion model (DM) into MIL for the first time and propose a feature augmentation framework called AugDiff. The diverse generation capabilities of DM guarantee a various range of feature augmentations, while its iterative generation approach effectively preserves semantic integrity during these augmentations. We conduct extensive experiments over four distinct cancer datasets, two different feature extractors, and three prevalent MIL algorithms to evaluate the performance of AugDiff. Ablation study and visualization further verify the effectiveness. Moreover, we highlight AugDiff's higher quality augmented feature over image augmentation and its superiority over self-supervised learning. The generalization over external datasets indicates its broader applications. The code is open-sourced on https://github.com/szc19990412/AugDiff .
多实例学习（MIL）是一种强大的弱监督学习策略，能够在千兆像素的整张幻灯片图像（WSI）上执行各种预测任务。然而，WSI 中数以万计的斑块通常会给图像增强带来巨大的计算负担，从而限制了 MIL 性能的提高。目前，基于特征增强的 MIL 框架是一种很有前景的解决方案，而现有的方法（如 mixup）往往会产生不切实际的特征。为了探索一种更高效、更实用的增强方法，我们首次在 MIL 中引入了扩散模型（DM），并提出了名为 AugDiff 的特征增强框架。DM 多样化的生成能力保证了各种特征增强，而其迭代生成方法在这些增强过程中有效地保持了语义的完整性。我们在四个不同的癌症数据集、两种不同的特征提取器和三种流行的 MIL 算法上进行了广泛的实验，以评估 AugDiff 的性能。消融研究和可视化进一步验证了其有效性。此外，我们还强调了 AugDiff 比图像增强具有更高质量的增强特征，而且比自我监督学习更具优势。在外部数据集上的泛化表明其应用范围更广。代码开源于 https://github.com/szc19990412/AugDiff。
{"title":"AugDiff: Diffusion-Based Feature Augmentation for Multiple Instance Learning in Whole Slide Image","authors":"Zhuchen Shao;Liuxi Dai;Yifeng Wang;Haoqian Wang;Yongbing Zhang","doi":"10.1109/TAI.2024.3454591","DOIUrl":"https://doi.org/10.1109/TAI.2024.3454591","url":null,"abstract":"Multiple instance learning (MIL), a powerful strategy for weakly supervised learning, is able to perform various prediction tasks on gigapixel whole slide images (WSIs). However, the tens of thousands of patches in WSIs usually incur a vast computational burden for image augmentation, limiting the performance improvement in MIL. Currently, the feature augmentation-based MIL framework is a promising solution, while existing methods such as mixup often produce unrealistic features. To explore a more efficient and practical augmentation method, we introduce the diffusion model (DM) into MIL for the first time and propose a feature augmentation framework called AugDiff. The diverse generation capabilities of DM guarantee a various range of feature augmentations, while its iterative generation approach effectively preserves semantic integrity during these augmentations. We conduct extensive experiments over four distinct cancer datasets, two different feature extractors, and three prevalent MIL algorithms to evaluate the performance of AugDiff. Ablation study and visualization further verify the effectiveness. Moreover, we highlight AugDiff's higher quality augmented feature over image augmentation and its superiority over self-supervised learning. The generalization over external datasets indicates its broader applications. The code is open-sourced on \u0000<uri>https://github.com/szc19990412/AugDiff</uri>\u0000.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6617-6628"},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-05DOI: 10.1109/TAI.2024.3454566
Kristina Telegraph;Christos Kyrkou
This work presents advancements in multiclass vehicle detection using unmanned aerial vehicle (UAV) cameras through the development of spatiotemporal object detection models. The study introduces a spatiotemporal vehicle detection dataset (STVD) containing $6600$ annotated sequential frame images captured by UAVs, enabling comprehensive training and evaluation of algorithms for holistic spatiotemporal perception. A YOLO-based object detection algorithm is enhanced to incorporate temporal dynamics, resulting in improved performance over single frame models. The integration of attention mechanisms into spatiotemporal models is shown to further enhance performance. Experimental validation demonstrates significant progress, with the best spatiotemporal model exhibiting a 16.22% improvement over single frame models, while it is demonstrated that attention mechanisms hold the potential for additional performance gains.
{"title":"Spatiotemporal Object Detection for Improved Aerial Vehicle Detection in Traffic Monitoring","authors":"Kristina Telegraph;Christos Kyrkou","doi":"10.1109/TAI.2024.3454566","DOIUrl":"https://doi.org/10.1109/TAI.2024.3454566","url":null,"abstract":"This work presents advancements in multiclass vehicle detection using unmanned aerial vehicle (UAV) cameras through the development of spatiotemporal object detection models. The study introduces a spatiotemporal vehicle detection dataset (STVD) containing \u0000<inline-formula><tex-math>$6600$</tex-math></inline-formula>\u0000 annotated sequential frame images captured by UAVs, enabling comprehensive training and evaluation of algorithms for holistic spatiotemporal perception. A YOLO-based object detection algorithm is enhanced to incorporate temporal dynamics, resulting in improved performance over single frame models. The integration of attention mechanisms into spatiotemporal models is shown to further enhance performance. Experimental validation demonstrates significant progress, with the best spatiotemporal model exhibiting a 16.22% improvement over single frame models, while it is demonstrated that attention mechanisms hold the potential for additional performance gains.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6159-6171"},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：