The coordination of population structure is the foundation for the effective functioning of evolutionary algorithms. An efficient population evolution structure can guide individuals to engage in successful and robust exploitative and exploratory behaviors. However, due to the black-box property of the search process, it is challenging to assess the current state of the population and implement targeted measures. In this paper, we propose a dynamic population structures-based differential evolution algorithm (DPSDE) to uncover the real-time state of population continuous optimization. According to the exploitation and exploration state of population, we introduce four structural modules to address the premature convergence and search stagnation issues of the current population. To effectively utilize these modules, we propose a real-time discernment mechanism to judge the population's current state. Based on the feedback information, suitable structural modules are dynamically invoked, ensuring that the population undergoes continuous and beneficial evolution, ultimately exploring the optimal population structure. The comparative outcomes with numerous cutting-edge algorithms on the IEEE Congress on Evolutionary Computation (CEC) 2017 benchmark functions and 2011 real-world problems verify the superiority of DPSDE. Furthermore, parameters, population state, and ablation study of modules are discussed.
{"title":"Dynamic Population Structures-Based Differential Evolution Algorithm","authors":"Jiaru Yang;Kaiyu Wang;Yirui Wang;Jiahai Wang;Zhenyu Lei;Shangce Gao","doi":"10.1109/TETCI.2024.3367809","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3367809","url":null,"abstract":"The coordination of population structure is the foundation for the effective functioning of evolutionary algorithms. An efficient population evolution structure can guide individuals to engage in successful and robust exploitative and exploratory behaviors. However, due to the black-box property of the search process, it is challenging to assess the current state of the population and implement targeted measures. In this paper, we propose a dynamic population structures-based differential evolution algorithm (DPSDE) to uncover the real-time state of population continuous optimization. According to the exploitation and exploration state of population, we introduce four structural modules to address the premature convergence and search stagnation issues of the current population. To effectively utilize these modules, we propose a real-time discernment mechanism to judge the population's current state. Based on the feedback information, suitable structural modules are dynamically invoked, ensuring that the population undergoes continuous and beneficial evolution, ultimately exploring the optimal population structure. The comparative outcomes with numerous cutting-edge algorithms on the IEEE Congress on Evolutionary Computation (CEC) 2017 benchmark functions and 2011 real-world problems verify the superiority of DPSDE. Furthermore, parameters, population state, and ablation study of modules are discussed.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The conventional dendritic neuron model (DNM) is a single-neuron model inspired by biological dendritic neurons that has been applied successfully in various fields. However, an increasing number of input features results in inefficient learning and gradient vanishing problems in the DNM. Thus, the DNM struggles to handle more complex tasks, including multiclass classification and multivariate time-series forecasting problems. In this study, we extended the conventional DNM to overcome these limitations. In the proposed dendritic neural network (DNN), the flexibility of both synapses and dendritic branches is considered and formulated, which can improve the model's nonlinear capabilities on high-dimensional problems. Then, multiple output layers are stacked to accommodate the various loss functions of complex tasks, and a dropout mechanism is implemented to realize a better balance between the underfitting and overfitting problems, which enhances the network's generalizability. The performance and computational efficiency of the proposed DNN compared to state-of-the-art machine learning algorithms were verified on 10 multiclass classification and 2 high-dimensional binary classification datasets. The experimental results demonstrate that the proposed DNN is a promising and practical neural network architecture.
{"title":"Dendritic Neural Network: A Novel Extension of Dendritic Neuron Model","authors":"Cheng Tang;Junkai Ji;Yuki Todo;Atsushi Shimada;Weiping Ding;Akimasa Hirata","doi":"10.1109/TETCI.2024.3367819","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3367819","url":null,"abstract":"The conventional dendritic neuron model (DNM) is a single-neuron model inspired by biological dendritic neurons that has been applied successfully in various fields. However, an increasing number of input features results in inefficient learning and gradient vanishing problems in the DNM. Thus, the DNM struggles to handle more complex tasks, including multiclass classification and multivariate time-series forecasting problems. In this study, we extended the conventional DNM to overcome these limitations. In the proposed dendritic neural network (DNN), the flexibility of both synapses and dendritic branches is considered and formulated, which can improve the model's nonlinear capabilities on high-dimensional problems. Then, multiple output layers are stacked to accommodate the various loss functions of complex tasks, and a dropout mechanism is implemented to realize a better balance between the underfitting and overfitting problems, which enhances the network's generalizability. The performance and computational efficiency of the proposed DNN compared to state-of-the-art machine learning algorithms were verified on 10 multiclass classification and 2 high-dimensional binary classification datasets. The experimental results demonstrate that the proposed DNN is a promising and practical neural network architecture.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10460122","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-08DOI: 10.1109/TETCI.2024.3369949
Rung-Tzuo Liaw;Yu-Wei Wen
Evolutionary machine learning has drawn much attentions on solving data-driven learning problem in the past decades, where classification is a major branch of data-driven learning problem. To improve the quality of obtained classifier, ensemble is a simple yet powerful strategy. However, gathering classifiers for ensemble requires multiple runs of learning process which bring additional cost at evaluation on the data. This study proposes an innovative framework for ensemble learning through evolutionary multitasking, i.e., the evolutionary multitasking for ensemble learning (EMTEL). There are four main features in the EMTEL. First, the EMTEL formulates a classification problem as a dynamic multitask optimization problem. Second, the EMTEL utilizes evolutionary multitasking to resolve the dynamic multitask optimization problem for better convergence through the synergy of common properties hidden in the tasks. Third, the EMTEL incorporates evolutionary instance selection for saving the cost at evaluation. Finally, the EMTEL formulates the ensemble learning problem as a numerical optimization problem and proposes an online ensemble aggregation approach to simultaneously select appropriate ensemble candidates from learning history and optimize ensemble weights for aggregating predictions. A case study is investigated by integrating two state-of-the-art methods for evolutionary multitasking and evolutionary instance selection respectively, i.e., the symbiosis in biocoenosis optimization and cooperative evolutionary learning and instance selection. For online ensemble aggregation, this study adopts the well-known covariance matrix adaptation evolution strategy. Experiments validate the effectiveness of the EMTEL over conventional and advanced evolutionary machine learning algorithms, including genetic programming, self-learning gene expression programming, and multi-dimensional genetic programming. Experimental results show that the proposed framework ameliorates state-of-the-art methods, and the improvements on quality for multiclass classification are at 8.48% at least and 56.35% at most in relation to the macro F-score. For convergence speed, the speedups achieved by the proposed framework are 7.85 at least and 100.53 at most on multiclass classification.
{"title":"Ensemble Learning Through Evolutionary Multitasking: A Formulation and Case Study","authors":"Rung-Tzuo Liaw;Yu-Wei Wen","doi":"10.1109/TETCI.2024.3369949","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369949","url":null,"abstract":"Evolutionary machine learning has drawn much attentions on solving data-driven learning problem in the past decades, where classification is a major branch of data-driven learning problem. To improve the quality of obtained classifier, ensemble is a simple yet powerful strategy. However, gathering classifiers for ensemble requires multiple runs of learning process which bring additional cost at evaluation on the data. This study proposes an innovative framework for ensemble learning through evolutionary multitasking, i.e., the evolutionary multitasking for ensemble learning (EMTEL). There are four main features in the EMTEL. First, the EMTEL formulates a classification problem as a dynamic multitask optimization problem. Second, the EMTEL utilizes evolutionary multitasking to resolve the dynamic multitask optimization problem for better convergence through the synergy of common properties hidden in the tasks. Third, the EMTEL incorporates evolutionary instance selection for saving the cost at evaluation. Finally, the EMTEL formulates the ensemble learning problem as a numerical optimization problem and proposes an online ensemble aggregation approach to simultaneously select appropriate ensemble candidates from learning history and optimize ensemble weights for aggregating predictions. A case study is investigated by integrating two state-of-the-art methods for evolutionary multitasking and evolutionary instance selection respectively, i.e., the symbiosis in biocoenosis optimization and cooperative evolutionary learning and instance selection. For online ensemble aggregation, this study adopts the well-known covariance matrix adaptation evolution strategy. Experiments validate the effectiveness of the EMTEL over conventional and advanced evolutionary machine learning algorithms, including genetic programming, self-learning gene expression programming, and multi-dimensional genetic programming. Experimental results show that the proposed framework ameliorates state-of-the-art methods, and the improvements on quality for multiclass classification are at 8.48% at least and 56.35% at most in relation to the macro F-score. For convergence speed, the speedups achieved by the proposed framework are 7.85 at least and 100.53 at most on multiclass classification.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-07DOI: 10.1109/TETCI.2024.3369629
Songbai Liu;Jun Li;Qiuzhen Lin;Ye Tian;Jianqiang Li;Kay Chen Tan
Addressing the challenge of efficiently handling high-dimensional search spaces in solving large-scale multiobjective optimization problems (LMOPs) becomes an emerging research topic in evolutionary computation. In response, this paper proposes a new evolutionary optimizer with a tactic of autoencoder-based problem transformation (APT). The APT involves creating an autoencoder to learn the relative importance of each variable by competitively reconstructing the dominated and non-dominated solutions. Using the learned importance, all variables are divided into multiple groups without consuming any function evaluations. The number of groups dynamically increases according to the population's evolutionary status. Each variable group has an associated autoencoder, transforming the search space into an adaptable small-scale representation space. Thus, the search process occurs within these dynamic representation spaces, leading to effective production of offspring solutions. To assess the effectiveness of APT, extensive testing is performed on benchmark suites and real-world LMOPs, encompassing variable sizes ranging from 10 3 to 10 4 . The comparative results demonstrate the advantages of our proposed optimizer in solving these LMOPs with a limited budget of 10 5 function evaluations.
{"title":"Evolutionary Large-Scale Multiobjective Optimization via Autoencoder-Based Problem Transformation","authors":"Songbai Liu;Jun Li;Qiuzhen Lin;Ye Tian;Jianqiang Li;Kay Chen Tan","doi":"10.1109/TETCI.2024.3369629","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369629","url":null,"abstract":"Addressing the challenge of efficiently handling high-dimensional search spaces in solving large-scale multiobjective optimization problems (LMOPs) becomes an emerging research topic in evolutionary computation. In response, this paper proposes a new evolutionary optimizer with a tactic of autoencoder-based problem transformation (APT). The APT involves creating an autoencoder to learn the relative importance of each variable by competitively reconstructing the dominated and non-dominated solutions. Using the learned importance, all variables are divided into multiple groups without consuming any function evaluations. The number of groups dynamically increases according to the population's evolutionary status. Each variable group has an associated autoencoder, transforming the search space into an adaptable small-scale representation space. Thus, the search process occurs within these dynamic representation spaces, leading to effective production of offspring solutions. To assess the effectiveness of APT, extensive testing is performed on benchmark suites and real-world LMOPs, encompassing variable sizes ranging from 10\u0000<sup>3</sup>\u0000 to 10\u0000<sup>4</sup>\u0000. The comparative results demonstrate the advantages of our proposed optimizer in solving these LMOPs with a limited budget of 10\u0000<sup>5</sup>\u0000 function evaluations.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-07DOI: 10.1109/TETCI.2024.3369855
Debashis Sen;V. K. Vivekraj
A novel measure to evaluate a dynamic video summary against multiple reference summaries is proposed in this paper. To this end, concepts of rough set and granular computing are leveraged to theoretically design the measure that captures the inherent (dis)agreement among the multiple references and the resulting clustering tendency. In our design, multiple F-measures are used to represent the similarities between the dynamic video summary being evaluated and the multiple references. The clustering tendency among the multiple references induces granulation, which allows the computation of degrees of appropriateness of the multiple F-measures. These degrees of appropriateness are then used to combine the multiple F-measures resulting in our novel measure, which we refer to as the granule-aware F-measure or the GF-measure. Along with a few attributes of our proposed evaluation measure, it is theoretically shown that the average F-measure is a special case of our GF-measure. Two specific GF-measures called the GF(mad)-measure and GF(sat)-measure corresponding to judicious parameter choices are also discussed. Experiments including statistical, correlation and user studies are performed on the GF-measure to demonstrate its significance, distinguishing it from the popular average and maximum F-measures. The experiments are performed on summaries generated by multiple dynamic video summarization approaches for videos from a few standard datasets.
本文提出了一种新颖的方法，用于根据多个参考摘要对动态视频摘要进行评估。为此，我们利用粗糙集和粒度计算的概念，从理论上设计了一种能捕捉多个参考资料之间固有（不）一致性以及由此产生的聚类趋势的测量方法。在我们的设计中，使用多个 F 度量来表示被评估的动态视频摘要与多个参考之间的相似性。多个参考文献之间的聚类趋势会导致颗粒化，从而允许计算多个 F 度量的适当度。然后，利用这些适当度来组合多个 F-度量，从而形成我们的新度量，我们称之为粒度感知 F-度量或 GF-度量。除了我们提出的评估指标的一些属性外，理论上还证明了平均 F 指标是我们的 GF 指标的一个特例。此外，还讨论了与明智的参数选择相对应的两种特定 GF 测量，即 GF（mad）测量和 GF（sat）测量。对 GF 度量进行了包括统计、相关性和用户研究在内的实验，以证明其重要性，并将其与流行的平均和最大 F 度量区分开来。实验是在多个动态视频摘要方法针对几个标准数据集的视频生成的摘要上进行的。
{"title":"Multi-Reference Evaluation of Dynamic Video Summaries Using Granule-Aware F-Measure","authors":"Debashis Sen;V. K. Vivekraj","doi":"10.1109/TETCI.2024.3369855","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369855","url":null,"abstract":"A novel measure to evaluate a dynamic video summary against multiple reference summaries is proposed in this paper. To this end, concepts of rough set and granular computing are leveraged to theoretically design the measure that captures the inherent (dis)agreement among the multiple references and the resulting clustering tendency. In our design, multiple F-measures are used to represent the similarities between the dynamic video summary being evaluated and the multiple references. The clustering tendency among the multiple references induces granulation, which allows the computation of degrees of appropriateness of the multiple F-measures. These degrees of appropriateness are then used to combine the multiple F-measures resulting in our novel measure, which we refer to as the granule-aware F-measure or the GF-measure. Along with a few attributes of our proposed evaluation measure, it is theoretically shown that the average F-measure is a special case of our GF-measure. Two specific GF-measures called the GF(mad)-measure and GF(sat)-measure corresponding to judicious parameter choices are also discussed. Experiments including statistical, correlation and user studies are performed on the GF-measure to demonstrate its significance, distinguishing it from the popular average and maximum F-measures. The experiments are performed on summaries generated by multiple dynamic video summarization approaches for videos from a few standard datasets.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Scaling up the capacity of deep neural networks (DNN) is one of the effective approaches to improve the model quality for several different DNN-based applications, making the DNN models continuously grow. To promote the execution efficiency of large and complex models, the devices are becoming increasingly heterogeneous with CPUs and domain-specific hardware accelerators. In many cases, the capacity of large-scale models is beyond the memory limit of a single accelerator. Recent work has shown that model parallelism, which aims to partition a DNN's computational graph on multiple devices, can not only address this problem while also provide significant performance improvements. In this work, we focus on optimizing model parallelism for timely inference of large-scale DNNs on heterogeneous processors. We transform the computation graphs of DNNs into directed acyclic graphs (DAGs) and propose to utilize heterogeneous scheduling methods to determine the model partition plan. Nevertheless, we have found that current efficient DAG scheduling methods have a lot of room for improvement to process large-scale DAGs and have high computation complexity. To this end, we propose a scalable DAG partition assisted scheduling method for heterogeneous processors to address these problems. Our approach takes the execution time of DNN models, high scalability, and memory constraints into consideration. We demonstrate the effectiveness of our approaches using both small- and large-scale DNN models. To the best of our knowledge, it is the first work that explores DAG scheduling and partitioning methods for model parallelism, and provides new avenues for accelerating large-scale DNN inference.
扩展深度神经网络（DNN）的容量是提高基于 DNN 的多个不同应用的模型质量的有效方法之一，这使得 DNN 模型不断增长。为了提高大型复杂模型的执行效率，CPU 和特定领域硬件加速器等设备正变得越来越异构。在许多情况下，大规模模型的容量超出了单个加速器的内存限制。最近的研究表明，旨在将 DNN 计算图分割到多个设备上的模型并行化不仅能解决这一问题，还能显著提高性能。在这项工作中，我们重点优化模型并行性，以便在异构处理器上及时推断大规模 DNN。我们将 DNN 的计算图转化为有向无环图（DAG），并建议利用异构调度方法来确定模型分区计划。然而，我们发现，当前高效的 DAG 调度方法在处理大规模 DAG 方面还有很大的改进空间，而且计算复杂度较高。为此，我们提出了一种针对异构处理器的可扩展 DAG 分区辅助调度方法来解决这些问题。我们的方法考虑到了 DNN 模型的执行时间、高可扩展性和内存限制。我们使用小型和大型 DNN 模型证明了我们方法的有效性。据我们所知，这是第一项探索模型并行性的 DAG 调度和分区方法的研究，为加速大规模 DNN 推断提供了新途径。
{"title":"Scalable Heterogeneous Scheduling Based Model Parallelism for Real-Time Inference of Large-Scale Deep Neural Networks","authors":"Xiaofeng Zou;Cen Chen;Peiying Lin;Luochuan Zhang;Yanwu Xu;Wenjie Zhang","doi":"10.1109/TETCI.2024.3369628","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369628","url":null,"abstract":"Scaling up the capacity of deep neural networks (DNN) is one of the effective approaches to improve the model quality for several different DNN-based applications, making the DNN models continuously grow. To promote the execution efficiency of large and complex models, the devices are becoming increasingly heterogeneous with CPUs and domain-specific hardware accelerators. In many cases, the capacity of large-scale models is beyond the memory limit of a single accelerator. Recent work has shown that model parallelism, which aims to partition a DNN's computational graph on multiple devices, can not only address this problem while also provide significant performance improvements. In this work, we focus on optimizing model parallelism for timely inference of large-scale DNNs on heterogeneous processors. We transform the computation graphs of DNNs into directed acyclic graphs (DAGs) and propose to utilize heterogeneous scheduling methods to determine the model partition plan. Nevertheless, we have found that current efficient DAG scheduling methods have a lot of room for improvement to process large-scale DAGs and have high computation complexity. To this end, we propose a scalable DAG partition assisted scheduling method for heterogeneous processors to address these problems. Our approach takes the execution time of DNN models, high scalability, and memory constraints into consideration. We demonstrate the effectiveness of our approaches using both small- and large-scale DNN models. To the best of our knowledge, it is the first work that explores DAG scheduling and partitioning methods for model parallelism, and provides new avenues for accelerating large-scale DNN inference.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-07DOI: 10.1109/TETCI.2024.3369331
Rahul Umesh Mhapsekar;Norah O'Shea;Steven Davy;Lizy Abraham
There has been an increase in the implementation of Artificial Intelligence (AI) in the dairy industry for Milk Quality Analysis (MQA). However, traditional Machine Learning (ML) algorithms may not be effective due to non-linearity in milk spectral data and the requirement of pre-processing. Important features from the spectral data may be lost during the pre-processing stage, which is a severe problem. Deep Learning (DL) can help by eliminating the need for pre-processing, thereby avoiding the loss of information. Although traditional DL methods have been used in dairy farming applications, fewer studies indicate the use of DL for MQA. Therefore, there is a need to develop novel DL models for MQA to improve the classification accuracy for milk quality monitoring. This study proposes a Hybrid Blended Deep Learning (HyBDL) approach for better classification accuracy and lower prediction errors. The proposed model outperformed classical DL and Blended DL models in terms of overall accuracy, loss, and class-wise accuracy used in this study. The model achieved 98.03% accuracy and lower Mean Squared Error (MSE) scores for each iteration, and its power consumption, energy consumption, and training time were evaluated. To support our work, we calculated the reproducibility score for all the models, representing how consistent the results are when repeated multiple times. Time complexity analysis of the models is performed to compare the resource consumption and training times for the base learners and HyBDL model. To further validate the performance of our model, we have trained it on different resource-intensive edge devices, such as the NVIDIA Jetson Nano and a low-end device. Edge devices can be used in dairy processing plants to provide real-time milk quality predictions making it essential to this field of research. Our proposed HyBDL model outperformed all the other models by having a low deviation score of 0.0037 for ten iterations and 0.0077 for 100 iterations showing high reproducibility.
{"title":"Hybrid Blended Deep Learning Approach for Milk Quality Analysis","authors":"Rahul Umesh Mhapsekar;Norah O'Shea;Steven Davy;Lizy Abraham","doi":"10.1109/TETCI.2024.3369331","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369331","url":null,"abstract":"There has been an increase in the implementation of Artificial Intelligence (AI) in the dairy industry for Milk Quality Analysis (MQA). However, traditional Machine Learning (ML) algorithms may not be effective due to non-linearity in milk spectral data and the requirement of pre-processing. Important features from the spectral data may be lost during the pre-processing stage, which is a severe problem. Deep Learning (DL) can help by eliminating the need for pre-processing, thereby avoiding the loss of information. Although traditional DL methods have been used in dairy farming applications, fewer studies indicate the use of DL for MQA. Therefore, there is a need to develop novel DL models for MQA to improve the classification accuracy for milk quality monitoring. This study proposes a Hybrid Blended Deep Learning (HyBDL) approach for better classification accuracy and lower prediction errors. The proposed model outperformed classical DL and Blended DL models in terms of overall accuracy, loss, and class-wise accuracy used in this study. The model achieved 98.03% accuracy and lower Mean Squared Error (MSE) scores for each iteration, and its power consumption, energy consumption, and training time were evaluated. To support our work, we calculated the reproducibility score for all the models, representing how consistent the results are when repeated multiple times. Time complexity analysis of the models is performed to compare the resource consumption and training times for the base learners and HyBDL model. To further validate the performance of our model, we have trained it on different resource-intensive edge devices, such as the NVIDIA Jetson Nano and a low-end device. Edge devices can be used in dairy processing plants to provide real-time milk quality predictions making it essential to this field of research. Our proposed HyBDL model outperformed all the other models by having a low deviation score of 0.0037 for ten iterations and 0.0077 for 100 iterations showing high reproducibility.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-06DOI: 10.1109/TETCI.2024.3393367
Ruokai Yin;Youngeun Kim;Yuhang Li;Abhishek Moitra;Nitin Satpute;Anna Hambitzer;Priyadarshini Panda
Pruning for Spiking Neural Networks (SNNs) has emerged as a fundamental methodology for deploying deep SNNs on resource-constrained edge devices. Though the existing pruning methods can provide extremely high weight sparsity for deep SNNs, the high weight sparsity brings a workload imbalance problem. Specifically, the workload imbalance happens when a different number of non-zero weights are assigned to hardware units running in parallel. This results in low hardware utilization and thus imposes longer latency and higher energy costs. In preliminary experiments, we show that sparse SNNs ( $sim$ 98% weight sparsity) can suffer as low as $sim$ 59% utilization. To alleviate the workload imbalance problem, we propose u-Ticket, where we monitor and adjust the weight connections of the SNN during Lottery Ticket Hypothesis (LTH) based pruning, thus guaranteeing the final ticket gets optimal utilization when deployed onto the hardware. Experiments indicate that our u-Ticket can guarantee up to 100% hardware utilization, thus reducing up to 76.9% latency and 63.8% energy cost compared to the non-utilization-aware LTH method.
{"title":"Workload-Balanced Pruning for Sparse Spiking Neural Networks","authors":"Ruokai Yin;Youngeun Kim;Yuhang Li;Abhishek Moitra;Nitin Satpute;Anna Hambitzer;Priyadarshini Panda","doi":"10.1109/TETCI.2024.3393367","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3393367","url":null,"abstract":"Pruning for Spiking Neural Networks (SNNs) has emerged as a fundamental methodology for deploying deep SNNs on resource-constrained edge devices. Though the existing pruning methods can provide extremely high weight sparsity for deep SNNs, the high weight sparsity brings a workload imbalance problem. Specifically, the workload imbalance happens when a different number of non-zero weights are assigned to hardware units running in parallel. This results in low hardware utilization and thus imposes longer latency and higher energy costs. In preliminary experiments, we show that sparse SNNs (\u0000<inline-formula><tex-math>$sim$</tex-math></inline-formula>\u000098% weight sparsity) can suffer as low as \u0000<inline-formula><tex-math>$sim$</tex-math></inline-formula>\u000059% utilization. To alleviate the workload imbalance problem, we propose u-Ticket, where we monitor and adjust the weight connections of the SNN during Lottery Ticket Hypothesis (LTH) based pruning, thus guaranteeing the final ticket gets optimal utilization when deployed onto the hardware. Experiments indicate that our u-Ticket can guarantee up to 100% hardware utilization, thus reducing up to 76.9% latency and 63.8% energy cost compared to the non-utilization-aware LTH method.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965866","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-05DOI: 10.1109/TETCI.2024.3367821
Xuebin Yue;Lin Meng
Recently, object detection witnessed vast progress with the rapid development of Convolutional Neural Networks (CNNs). However, object detection is mainly for multi-class tasks, and few networks are used to detect single-class multi-deformation objects. This paper aims to develop a lightweight object detection network for single-class multi-deformation objects to promote the practical application of object detection networks. First, we design a Densely Connected Multi-scale (DCM) module to augment the semantic information extraction of deformation objects. With the DCM module and other strategies incorporated, we design a lightweight backbone structure for object detection, namely, DCMNet. Then, we construct a lightweight Neck structure Ghost Multi-scale Feature (GMF) module for feature fusion using a feature linear generation strategy. Finally, with the DCMNet and GMF module, we propose the object detection network YOLO-SM for single-class multi-deformation objects. Extensive experiments demonstrate that our proposed backbone structure, DCMNet, significantly outperforms the state-of-the-art models. YOLO-SM achieves 97.66% mean Average Precision ( $mAP$ ) on the Barcode public dataset, which is higher than other state-of-the-art object detection models, and achieves an inference time of 55.45 frames per second (FPS), proving that the YOLO-SM has a good performance tradeoff between speed and accuracy in detecting single-class multi-deformation objects. Furthermore, in the single-class multi-deformation Crack public dataset, the $mAP$ of 86.11% is achieved, and an $mAP$ of 99.84% is obtained in the multi-class dataset Dish20, which is much higher than other state-of-the-art object detection models, proving that the YOLO-SM has good generalization ability.
{"title":"YOLO-SM: A Lightweight Single-Class Multi-Deformation Object Detection Network","authors":"Xuebin Yue;Lin Meng","doi":"10.1109/TETCI.2024.3367821","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3367821","url":null,"abstract":"Recently, object detection witnessed vast progress with the rapid development of Convolutional Neural Networks (CNNs). However, object detection is mainly for multi-class tasks, and few networks are used to detect single-class multi-deformation objects. This paper aims to develop a lightweight object detection network for single-class multi-deformation objects to promote the practical application of object detection networks. First, we design a Densely Connected Multi-scale (DCM) module to augment the semantic information extraction of deformation objects. With the DCM module and other strategies incorporated, we design a lightweight backbone structure for object detection, namely, DCMNet. Then, we construct a lightweight Neck structure Ghost Multi-scale Feature (GMF) module for feature fusion using a feature linear generation strategy. Finally, with the DCMNet and GMF module, we propose the object detection network YOLO-SM for single-class multi-deformation objects. Extensive experiments demonstrate that our proposed backbone structure, DCMNet, significantly outperforms the state-of-the-art models. YOLO-SM achieves 97.66% mean Average Precision (\u0000<inline-formula><tex-math>$mAP$</tex-math></inline-formula>\u0000) on the Barcode public dataset, which is higher than other state-of-the-art object detection models, and achieves an inference time of 55.45 frames per second (FPS), proving that the YOLO-SM has a good performance tradeoff between speed and accuracy in detecting single-class multi-deformation objects. Furthermore, in the single-class multi-deformation Crack public dataset, the \u0000<inline-formula><tex-math>$mAP$</tex-math></inline-formula>\u0000 of 86.11% is achieved, and an \u0000<inline-formula><tex-math>$mAP$</tex-math></inline-formula>\u0000 of 99.84% is obtained in the multi-class dataset Dish20, which is much higher than other state-of-the-art object detection models, proving that the YOLO-SM has good generalization ability.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-05DOI: 10.1109/TETCI.2024.3369909
Mert Can Kurucu;Ercan Atam;Müjde Güzelkaya;İbrahim Eksin
Seismic protection of multi-story buildings using friction dampers (FDs) is a cheap and effective passive structural control solution. For optimization of system response, optimal distribution of FDs between floors is required, which is a challenging problem. In this paper, we propose novel intelligent computational methods based on reinforcement learning for the distribution of $n$ FDs for $m$ -story buildings. In order to demonstrate the effectiveness of the proposed methods, a case study of optimally distributing 16 FDs in a 3-story building is considered, and the results are compared with the optimal solution found from a statistical analysis based on a large number of earthquake accelerations.
{"title":"Intelligent Computational Methods for Optimal Distribution of Friction Dampers in Seismic Protection of Buildings","authors":"Mert Can Kurucu;Ercan Atam;Müjde Güzelkaya;İbrahim Eksin","doi":"10.1109/TETCI.2024.3369909","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369909","url":null,"abstract":"Seismic protection of multi-story buildings using friction dampers (FDs) is a cheap and effective passive structural control solution. For optimization of system response, optimal distribution of FDs between floors is required, which is a challenging problem. In this paper, we propose novel intelligent computational methods based on reinforcement learning for the distribution of \u0000<inline-formula><tex-math>$n$</tex-math></inline-formula>\u0000 FDs for \u0000<inline-formula><tex-math>$m$</tex-math></inline-formula>\u0000-story buildings. In order to demonstrate the effectiveness of the proposed methods, a case study of optimally distributing 16 FDs in a 3-story building is considered, and the results are compared with the optimal solution found from a statistical analysis based on a large number of earthquake accelerations.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：