Pub Date : 2024-10-10DOI: 10.1109/TAI.2024.3477457
Xiaotian Song;Xiangning Xie;Zeqiong Lv;Gary G. Yen;Weiping Ding;Jiancheng Lv;Yanan Sun
Neural architecture search (NAS) has received increasing attention because of its exceptional merits in automating the design of deep neural network (DNN) architectures. However, the performance evaluation process, as a key part of NAS, often requires training a large number of DNNs. This inevitably makes NAS computationally expensive. In past years, many efficient evaluation methods (EEMs) have been proposed to address this critical issue. In this article, we comprehensively survey these EEMs published up to date, and provide a detailed analysis to motivate the further development of this research direction. Specifically, we divide the existing EEMs into four categories based on the number of DNNs trained for constructing these EEMs. The categorization can reflect the degree of efficiency in principle, which can in turn help quickly grasp the methodological features. In surveying each category, we further discuss the design principles and analyze the strengths and weaknesses to clarify the landscape of existing EEMs, thus making easily understanding the research trends of EEMs. Furthermore, we also discuss the current challenges and issues to identify future research directions in this emerging topic. In summary, this survey provides a convenient overview of EEM for interested users, and they can easily select the proper EEM method for the tasks at hand. In addition, the researchers in the NAS field could continue exploring the future directions suggested in the article.
{"title":"Efficient Evaluation Methods for Neural Architecture Search: A Survey","authors":"Xiaotian Song;Xiangning Xie;Zeqiong Lv;Gary G. Yen;Weiping Ding;Jiancheng Lv;Yanan Sun","doi":"10.1109/TAI.2024.3477457","DOIUrl":"https://doi.org/10.1109/TAI.2024.3477457","url":null,"abstract":"Neural architecture search (NAS) has received increasing attention because of its exceptional merits in automating the design of deep neural network (DNN) architectures. However, the performance evaluation process, as a key part of NAS, often requires training a large number of DNNs. This inevitably makes NAS computationally expensive. In past years, many efficient evaluation methods (EEMs) have been proposed to address this critical issue. In this article, we comprehensively survey these EEMs published up to date, and provide a detailed analysis to motivate the further development of this research direction. Specifically, we divide the existing EEMs into four categories based on the number of DNNs trained for constructing these EEMs. The categorization can reflect the degree of efficiency in principle, which can in turn help quickly grasp the methodological features. In surveying each category, we further discuss the design principles and analyze the strengths and weaknesses to clarify the landscape of existing EEMs, thus making easily understanding the research trends of EEMs. Furthermore, we also discuss the current challenges and issues to identify future research directions in this emerging topic. In summary, this survey provides a convenient overview of EEM for interested users, and they can easily select the proper EEM method for the tasks at hand. In addition, the researchers in the NAS field could continue exploring the future directions suggested in the article.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"5990-6011"},"PeriodicalIF":0.0,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-10DOI: 10.1109/TAI.2024.3477425
Arya Agrawal;Teena Sharma;Nishchal K. Verma
Real-time 3-D view reconstruction in an unfamiliar environment poses complexity for various applications due to varying conditions such as occlusion, latency, precision, etc. This article thoroughly examines and tests contemporary methodologies addressing challenges in 3-D view reconstruction. The methods being explored in this article are categorized into volumetric and mesh, generative adversarial network based, and open source library based methods. The exploration of these methods undergoes detailed discussions, encompassing methods, advantages, limitations, and empirical results. The real-time testing of each method is done on benchmarked datasets, including ShapeNet, Pascal 3D+, Pix3D, etc. The narrative highlights the crucial role of 3-D view reconstruction in domains such as robotics, virtual and augmented reality, medical imaging, cultural heritage preservation, etc. The article also anticipates future scopes by exploring generative models, unsupervised learning, and advanced sensor fusion to increase the robustness of the algorithms.
{"title":"A Comprehensive Exploration of Real-Time 3-D View Reconstruction Methods","authors":"Arya Agrawal;Teena Sharma;Nishchal K. Verma","doi":"10.1109/TAI.2024.3477425","DOIUrl":"https://doi.org/10.1109/TAI.2024.3477425","url":null,"abstract":"Real-time 3-D view reconstruction in an unfamiliar environment poses complexity for various applications due to varying conditions such as occlusion, latency, precision, etc. This article thoroughly examines and tests contemporary methodologies addressing challenges in 3-D view reconstruction. The methods being explored in this article are categorized into volumetric and mesh, generative adversarial network based, and open source library based methods. The exploration of these methods undergoes detailed discussions, encompassing methods, advantages, limitations, and empirical results. The real-time testing of each method is done on benchmarked datasets, including ShapeNet, Pascal 3D+, Pix3D, etc. The narrative highlights the crucial role of 3-D view reconstruction in domains such as robotics, virtual and augmented reality, medical imaging, cultural heritage preservation, etc. The article also anticipates future scopes by exploring generative models, unsupervised learning, and advanced sensor fusion to increase the robustness of the algorithms.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"5915-5927"},"PeriodicalIF":0.0,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-10DOI: 10.1109/TAI.2024.3478191
Kaixiang Yang;Wuxing Chen;Yifan Shi;Zhiwen Yu;C. L. Philip Chen
In the field of intelligent manufacturing, tackling the classification challenges caused by imbalanced data is crucial. Although the broad learning system (BLS) has been recognized as an effective and efficient method, its performance wanes with imbalanced datasets. Therefore, this article proposes a novel approach named simplified kernel-based cost-sensitive broad learning system (SKCSBLS) to address these issues. Based on the framework of cost-sensitive broad learning system (CSBLS) that assigns distinctive adjustment costs for individual classes, SKCSBLS emphasizes the importance of the minority class while mitigating the impact of data imbalance. Additionally, considering the complexity introduced by noisy or overlapping data points, we incorporate kernel mapping into the CSBLS. This improvement not only improves the system's capability to handle overlapping classes of samples, but also improves the overall classification effectiveness. Our experimental results highlight the potential of SKCSBLS in overcoming the challenges inherent in unbalanced data, providing a robust solution for advanced fault diagnosis in intelligent systems.
{"title":"Simplified Kernel-Based Cost-Sensitive Broad Learning System for Imbalanced Fault Diagnosis","authors":"Kaixiang Yang;Wuxing Chen;Yifan Shi;Zhiwen Yu;C. L. Philip Chen","doi":"10.1109/TAI.2024.3478191","DOIUrl":"https://doi.org/10.1109/TAI.2024.3478191","url":null,"abstract":"In the field of intelligent manufacturing, tackling the classification challenges caused by imbalanced data is crucial. Although the broad learning system (BLS) has been recognized as an effective and efficient method, its performance wanes with imbalanced datasets. Therefore, this article proposes a novel approach named simplified kernel-based cost-sensitive broad learning system (SKCSBLS) to address these issues. Based on the framework of cost-sensitive broad learning system (CSBLS) that assigns distinctive adjustment costs for individual classes, SKCSBLS emphasizes the importance of the minority class while mitigating the impact of data imbalance. Additionally, considering the complexity introduced by noisy or overlapping data points, we incorporate kernel mapping into the CSBLS. This improvement not only improves the system's capability to handle overlapping classes of samples, but also improves the overall classification effectiveness. Our experimental results highlight the potential of SKCSBLS in overcoming the challenges inherent in unbalanced data, providing a robust solution for advanced fault diagnosis in intelligent systems.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6629-6644"},"PeriodicalIF":0.0,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-10DOI: 10.1109/TAI.2024.3477436
Xiaojing Zhang;Shuangrong Liu;Lin Wang;Bo Yang;Jiawei Fan
In this study, the adaptive hypersphere nearest neighbors (ASNN) method is proposed as an optimization framework to enhance the generalization performance of neural network classifiers. In terms of the classification task, the neural network draws decision boundaries by constructing the discriminative features of samples. To learn those features, attributed to the flexibility and separability, the pair-wise constraint-based methods that consist of the pair-wise loss and an embedding space (e.g., hypersphere space) have gained considerable attention over the last decade. Despite their success, pair-wise constraint-based methods still suffer from premature convergence or divergence problems, driven by two main challenges. 1) The poor scalability of the embedding space constrains the variety of the distribution of embedded samples, thereby increasing the optimization difficulty. 2) It is hard to select suitable positive/negative pairs during the training. In order to address the aforementioned problems, we propose an adaptive hypersphere nearest neighbors method. On the one hand, we improve the scalability of features via a scale-adaptive hypersphere embedding space. On the other hand, we introduce a neighborhood-based probability loss, which magnifies the difference between pairs and enhances the discriminative power of features generated by the neural networks based on the nearest neighbor-based pairing strategy. Experiments on UCI datasets and image recognition tasks demonstrate that the proposed ASNN not only achieves improved intraclass consistency and interclass separability of samples, but also outperforms its competitive counterparts.
{"title":"Learning Neural Network Classifiers by Distributing Nearest Neighbors on Adaptive Hypersphere","authors":"Xiaojing Zhang;Shuangrong Liu;Lin Wang;Bo Yang;Jiawei Fan","doi":"10.1109/TAI.2024.3477436","DOIUrl":"https://doi.org/10.1109/TAI.2024.3477436","url":null,"abstract":"In this study, the adaptive hypersphere nearest neighbors (ASNN) method is proposed as an optimization framework to enhance the generalization performance of neural network classifiers. In terms of the classification task, the neural network draws decision boundaries by constructing the discriminative features of samples. To learn those features, attributed to the flexibility and separability, the pair-wise constraint-based methods that consist of the pair-wise loss and an embedding space (e.g., hypersphere space) have gained considerable attention over the last decade. Despite their success, pair-wise constraint-based methods still suffer from premature convergence or divergence problems, driven by two main challenges. 1) The poor scalability of the embedding space constrains the variety of the distribution of embedded samples, thereby increasing the optimization difficulty. 2) It is hard to select suitable positive/negative pairs during the training. In order to address the aforementioned problems, we propose an adaptive hypersphere nearest neighbors method. On the one hand, we improve the scalability of features via a scale-adaptive hypersphere embedding space. On the other hand, we introduce a neighborhood-based probability loss, which magnifies the difference between pairs and enhances the discriminative power of features generated by the neural networks based on the nearest neighbor-based pairing strategy. Experiments on UCI datasets and image recognition tasks demonstrate that the proposed ASNN not only achieves improved intraclass consistency and interclass separability of samples, but also outperforms its competitive counterparts.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"234-249"},"PeriodicalIF":0.0,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-08DOI: 10.1109/TAI.2024.3476434
Jing Li;Jichen Wang;Zerui Li;Yu Kang;Wenjun Lv
Lithology identification plays a pivotal role in stratigraphic characterization and reservoir exploration. The promising field of intelligent logging lithology identification, which employs machine learning algorithms to infer lithology from logging curves, is gaining significant attention. However, models trained on labeled wells currently face challenges in accurately predicting the lithologies of new unlabeled wells due to significant discrepancies in data distribution among different wells caused by the complex sedimentary environment and variations in logging equipment. Additionally, there is no guarantee that newly drilled wells share the same lithology classes as previously explored ones. Therefore, our research aims to leverage source logging and lithology data along with target logging data to train a model capable of directly discerning the lithologies of target wells. The challenges are centered around the disparities in data distribution and the lack of prior knowledge regarding potential lithology classes in the target well. To tackle these concerns, we have made concerted efforts: 1) proposing a novel lithology identification framework, sample transferability weighting based partial domain adaptation (ST-PDA), to effectively address the practical scenario of encountering an unknown label space in target wells; 2) designing a sample transferability weighting module to assign higher weights to shared-class samples, thus effectively mitigating the negative transfer caused by unshared-class source samples; 3) developing a module, convolutional neural network with integrated channel attention mechanism (CG ${}^{2}$ CA), to serve as the backbone network for feature extraction; and 4) incorporating a target sample reconstruction module to enhance the feature representation and further facilitating positive transfer. Extensive experiments on 16 real-world wells demonstrated the strong performance of ST-PDA and highlighted the necessity of each component in the framework.
{"title":"Partial Domain Adaptation for Building Borehole Lithology Model Under Weaker Geological Prior","authors":"Jing Li;Jichen Wang;Zerui Li;Yu Kang;Wenjun Lv","doi":"10.1109/TAI.2024.3476434","DOIUrl":"https://doi.org/10.1109/TAI.2024.3476434","url":null,"abstract":"Lithology identification plays a pivotal role in stratigraphic characterization and reservoir exploration. The promising field of intelligent logging lithology identification, which employs machine learning algorithms to infer lithology from logging curves, is gaining significant attention. However, models trained on labeled wells currently face challenges in accurately predicting the lithologies of new unlabeled wells due to significant discrepancies in data distribution among different wells caused by the complex sedimentary environment and variations in logging equipment. Additionally, there is no guarantee that newly drilled wells share the same lithology classes as previously explored ones. Therefore, our research aims to leverage source logging and lithology data along with target logging data to train a model capable of directly discerning the lithologies of target wells. The challenges are centered around the disparities in data distribution and the lack of prior knowledge regarding potential lithology classes in the target well. To tackle these concerns, we have made concerted efforts: 1) proposing a novel lithology identification framework, sample transferability weighting based partial domain adaptation (ST-PDA), to effectively address the practical scenario of encountering an unknown label space in target wells; 2) designing a sample transferability weighting module to assign higher weights to shared-class samples, thus effectively mitigating the negative transfer caused by unshared-class source samples; 3) developing a module, convolutional neural network with integrated channel attention mechanism (CG\u0000<inline-formula><tex-math>${}^{2}$</tex-math></inline-formula>\u0000CA), to serve as the backbone network for feature extraction; and 4) incorporating a target sample reconstruction module to enhance the feature representation and further facilitating positive transfer. Extensive experiments on 16 real-world wells demonstrated the strong performance of ST-PDA and highlighted the necessity of each component in the framework.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6645-6658"},"PeriodicalIF":0.0,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-07DOI: 10.1109/TAI.2024.3474648
Wan Shen;Junye Jiang;Minghan Li;Shuanglong Liu
Recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, have emerged as standard tools for tackling a wide range of time series applications, such as natural language processing. However, deploying these models on edge devices presents great challenges due to limited computational resources. Additionally, the implementation of RNN activation functions on low-end hardware devices significantly impacts the overall network performance, as activations constitute the dominant part of execution time. In this work, we propose an efficient approach for implementing commonly used RNN activations, leveraging an optimized coordinate rotation digital computer algorithm (CORDIC). Moreover, we propose a unified hardware architecture for mapping the CORDIC-based method onto field-programmable gate arrays (FPGAs), which can be configured to implement multiple nonlinear activation functions. Our architecture reduces the computational time with fewer iterations in CORDIC compared with existing methods, rendering it particularly suitable for resource-constrained edge devices. Our design is implemented on a Xilinx Zynq-7000 device and evaluated across three RNNs and benchmark datasets. Experimental results demonstrate that our design achieves up to a 2$boldsymbol{times}$ speedup while maintaining model accuracy compared with the state-of-the-art designs.
{"title":"Efficient CORDIC-Based Activation Functions for RNN Acceleration on FPGAs","authors":"Wan Shen;Junye Jiang;Minghan Li;Shuanglong Liu","doi":"10.1109/TAI.2024.3474648","DOIUrl":"https://doi.org/10.1109/TAI.2024.3474648","url":null,"abstract":"Recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, have emerged as standard tools for tackling a wide range of time series applications, such as natural language processing. However, deploying these models on edge devices presents great challenges due to limited computational resources. Additionally, the implementation of RNN activation functions on low-end hardware devices significantly impacts the overall network performance, as activations constitute the dominant part of execution time. In this work, we propose an efficient approach for implementing commonly used RNN activations, leveraging an optimized coordinate rotation digital computer algorithm (CORDIC). Moreover, we propose a unified hardware architecture for mapping the CORDIC-based method onto field-programmable gate arrays (FPGAs), which can be configured to implement multiple nonlinear activation functions. Our architecture reduces the computational time with fewer iterations in CORDIC compared with existing methods, rendering it particularly suitable for resource-constrained edge devices. Our design is implemented on a Xilinx Zynq-7000 device and evaluated across three RNNs and benchmark datasets. Experimental results demonstrate that our design achieves up to a 2<inline-formula><tex-math>$boldsymbol{times}$</tex-math></inline-formula> speedup while maintaining model accuracy compared with the state-of-the-art designs.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"199-210"},"PeriodicalIF":0.0,"publicationDate":"2024-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975727","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-07DOI: 10.1109/TAI.2024.3474650
Jingkai Ma;Shuang Bai;Wenchao Pan
Few-shot semantic segmentation (FSS) focuses on segmenting objects of novel classes with only a small number of annotated samples and has achieved great development. However, compared with general semantic segmentation, inaccurate boundary predictions remain a serious problem in FSS. This is because, in scenarios with few samples, the extracted query features by the model struggle to contain sufficient detailed information to focus on the boundary of the target. To address this issue, we propose a prior-driven edge feature enhancement network (PDEFE) that utilizes the prior information of the object edges to enhance the query feature, thereby promoting the accurate segmentation of the target. Specifically, we first design an edge feature enhancement module (EFEM) that can utilize object edges to enhance the feature of the query object's boundaries. Furthermore, we also propose an edge prior mask generator (EPMG) to generate prior masks for edges based on the gradient information of the image, which can guide the model to pay more attention to the boundaries of the target in the query image. Extensive experiments on PASCAL-$5^{i}$ and COCO-$20^{i}$ demonstrate that PDEFE significantly improves upon two baseline detectors (up to 2.7$sim$4.2% mIoU in average), achieving state-of-the-art performance.
{"title":"Boosting Few-Shot Semantic Segmentation With Prior-Driven Edge Feature Enhancement Network","authors":"Jingkai Ma;Shuang Bai;Wenchao Pan","doi":"10.1109/TAI.2024.3474650","DOIUrl":"https://doi.org/10.1109/TAI.2024.3474650","url":null,"abstract":"Few-shot semantic segmentation (FSS) focuses on segmenting objects of novel classes with only a small number of annotated samples and has achieved great development. However, compared with general semantic segmentation, inaccurate boundary predictions remain a serious problem in FSS. This is because, in scenarios with few samples, the extracted query features by the model struggle to contain sufficient detailed information to focus on the boundary of the target. To address this issue, we propose a prior-driven edge feature enhancement network (PDEFE) that utilizes the prior information of the object edges to enhance the query feature, thereby promoting the accurate segmentation of the target. Specifically, we first design an edge feature enhancement module (EFEM) that can utilize object edges to enhance the feature of the query object's boundaries. Furthermore, we also propose an edge prior mask generator (EPMG) to generate prior masks for edges based on the gradient information of the image, which can guide the model to pay more attention to the boundaries of the target in the query image. Extensive experiments on PASCAL-<inline-formula><tex-math>$5^{i}$</tex-math></inline-formula> and COCO-<inline-formula><tex-math>$20^{i}$</tex-math></inline-formula> demonstrate that PDEFE significantly improves upon two baseline detectors (up to 2.7<inline-formula><tex-math>$sim$</tex-math></inline-formula>4.2% mIoU in average), achieving state-of-the-art performance.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"211-220"},"PeriodicalIF":0.0,"publicationDate":"2024-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975726","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-07DOI: 10.1109/TAI.2024.3474654
Yang Yang;Chao Wang;Lei Gong;Min Wu;Zhenghua Chen;Xiang Li;Xianglan Chen;Xuehai Zhou
Ensemble object detectors have demonstrated remarkable effectiveness in enhancing prediction accuracy and uncertainty quantification. However, their widespread adoption is hindered by significant computational and storage demands, limiting their feasibility in resource-constrained settings. To overcome this, researchers have focused on distilling the knowledge from ensemble object detectors into a single model. In this article, we introduce probabilization based ensemble distillation (ProbED), an innovative ensemble distillation framework that consolidates knowledge from multiple object detectors into a single, resource-efficient model. Unlike traditional ensemble distillation methods that average the outputs of subteachers, ProbED captures comprehensive outcome distributions from all subteachers, providing a more nuanced approach to knowledge transfer. ProbED employs knowledge probabilization to achieve a sophisticated and refined aggregation of teacher knowledge, including feature knowledge, semantic knowledge, and localization knowledge, resulting in dual improvements in prediction accuracy and uncertainty quantification for the student model. In particular, ProED's novel knowledge probabilization-based approach to aggregating teacher knowledge is inspired by our empirical observations, which demonstrate that knowledge probabilization excels in effectively representing uncertainty, improving prediction, and facilitating robust knowledge transfer. Furthermore, we introduce a random smoothing perturbation technique to modify inputs within ProbED, further enhancing the distillation process. Extensive experiments highlight ProbED's ability to significantly enhance the prediction accuracy and uncertainty quantification of various object detectors, demonstrating its superior performance compared to other state-of-the-art techniques.
{"title":"Knowledge Probabilization in Ensemble Distillation: Improving Accuracy and Uncertainty Quantification for Object Detectors","authors":"Yang Yang;Chao Wang;Lei Gong;Min Wu;Zhenghua Chen;Xiang Li;Xianglan Chen;Xuehai Zhou","doi":"10.1109/TAI.2024.3474654","DOIUrl":"https://doi.org/10.1109/TAI.2024.3474654","url":null,"abstract":"Ensemble object detectors have demonstrated remarkable effectiveness in enhancing prediction accuracy and uncertainty quantification. However, their widespread adoption is hindered by significant computational and storage demands, limiting their feasibility in resource-constrained settings. To overcome this, researchers have focused on distilling the knowledge from ensemble object detectors into a single model. In this article, we introduce probabilization based ensemble distillation (ProbED), an innovative ensemble distillation framework that consolidates knowledge from multiple object detectors into a single, resource-efficient model. Unlike traditional ensemble distillation methods that average the outputs of subteachers, ProbED captures comprehensive outcome distributions from all subteachers, providing a more nuanced approach to knowledge transfer. ProbED employs knowledge probabilization to achieve a sophisticated and refined aggregation of teacher knowledge, including feature knowledge, semantic knowledge, and localization knowledge, resulting in dual improvements in prediction accuracy and uncertainty quantification for the student model. In particular, ProED's novel knowledge probabilization-based approach to aggregating teacher knowledge is inspired by our empirical observations, which demonstrate that knowledge probabilization excels in effectively representing uncertainty, improving prediction, and facilitating robust knowledge transfer. Furthermore, we introduce a random smoothing perturbation technique to modify inputs within ProbED, further enhancing the distillation process. Extensive experiments highlight ProbED's ability to significantly enhance the prediction accuracy and uncertainty quantification of various object detectors, demonstrating its superior performance compared to other state-of-the-art techniques.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"221-233"},"PeriodicalIF":0.0,"publicationDate":"2024-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the growing popularity of the Internet, Web applications have become increasingly essential in our daily lives. Web application programming interfaces (Web APIs) play a crucial role in facilitating interaction between applications. However, most Web service platforms are suffering from the imbalance of Web services now, many services of good quality but low popularity are difficult to be invoked even once and do not create direct connections with the users. Some graph-based Web service recommendation methods also often present a long-tailed distribution of recommended Web services due to limited Mashup–API invocation relationships. To relieve this problem and promote service recommendation, in this article, we propose a community structure and popularity-based approach by constructing an evolving cooperation network for Web APIs. We leverage the Louvain algorithm in community detection to assign community structure to each Web API and consider both the popularity and community structure in constructing the network. By optimizing the Barabάsi–Albert (BA) evolving network model, we demonstrate that our approach outperforms the BA, Bianconi–Barabάsi (BB), and popularity-similarity optimization (PSO) models in Web service clustering. Based on our proposed evolutionary network model for the evolutionary extension of API cooperation network and used for downstream Web service recommendation tasks, the experimental results also show that our recommended approach outperforms some other baseline models for Web service recommendation.
随着Internet的日益普及，Web应用程序在我们的日常生活中变得越来越重要。Web应用程序编程接口（Web api）在促进应用程序之间的交互方面起着至关重要的作用。然而，目前大多数Web服务平台都存在着Web服务不均衡的问题，许多质量好的但知名度不高的服务甚至很难被调用一次，也无法与用户建立直接连接。由于Mashup-API调用关系有限，一些基于图的Web服务推荐方法也经常呈现推荐的Web服务的长尾分布。为了缓解这一问题并促进服务推荐，本文通过构建Web api的演进合作网络，提出了一种基于社区结构和流行度的方法。我们利用社区检测中的Louvain算法为每个Web API分配社区结构，并在构建网络时考虑流行度和社区结构。通过优化barab si - albert （BA）进化网络模型，我们证明了我们的方法在Web服务聚类中优于BA、bianconi - barab si （BB）和流行度相似度优化（PSO）模型。基于我们提出的API协作网络进化扩展的进化网络模型，并将其用于下游Web服务推荐任务，实验结果还表明，我们的推荐方法优于其他一些Web服务推荐基线模型。
{"title":"Evolution of Web API Cooperation Network via Exploring Community Structure and Popularity","authors":"Guosheng Kang;Yang Wang;Jianxun Liu;Buqing Cao;Yong Xiao;Yu Xu","doi":"10.1109/TAI.2024.3472614","DOIUrl":"https://doi.org/10.1109/TAI.2024.3472614","url":null,"abstract":"With the growing popularity of the Internet, Web applications have become increasingly essential in our daily lives. Web application programming interfaces (Web APIs) play a crucial role in facilitating interaction between applications. However, most Web service platforms are suffering from the imbalance of Web services now, many services of good quality but low popularity are difficult to be invoked even once and do not create direct connections with the users. Some graph-based Web service recommendation methods also often present a long-tailed distribution of recommended Web services due to limited Mashup–API invocation relationships. To relieve this problem and promote service recommendation, in this article, we propose a community structure and popularity-based approach by constructing an evolving cooperation network for Web APIs. We leverage the Louvain algorithm in community detection to assign community structure to each Web API and consider both the popularity and community structure in constructing the network. By optimizing the Barabάsi–Albert (BA) evolving network model, we demonstrate that our approach outperforms the BA, Bianconi–Barabάsi (BB), and popularity-similarity optimization (PSO) models in Web service clustering. Based on our proposed evolutionary network model for the evolutionary extension of API cooperation network and used for downstream Web service recommendation tasks, the experimental results also show that our recommended approach outperforms some other baseline models for Web service recommendation.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6659-6671"},"PeriodicalIF":0.0,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：