This paper is concerned with the double sensitive fault detection filter for positive Markovian jump systems. A new hybrid adaptive event-triggered mechanism is proposed by introducing a non-monotonic adaptive law. A linear adaptive event-triggered threshold is established by virtue of 1-norm inequality. Under such a triggering strategy, the original system can be transformed into an interval uncertain system. By using a stochastic copositive Lyapunov function, an asynchronous fault detection filter is designed for positive Markovian jump systems (PMJSs) in terms of linear programming. The presented filter satisfies both $L_{-}$ -gain ( $ell_{-}$ -gain) fault sensitivity and $L_{1} (ell_{1})$ internal differential privacy sensitivity. The proposed approach is also extended to the discrete-time case. Finally, two examples are provided to illustrate the effectiveness of the proposed design.
{"title":"A Double Sensitive Fault Detection Filter for Positive Markovian Jump Systems with A Hybrid Event-Triggered Mechanism","authors":"Junfeng Zhang;Baozhu Du;Suhuan Zhang;Shihong Ding","doi":"10.1109/JAS.2024.124677","DOIUrl":"https://doi.org/10.1109/JAS.2024.124677","url":null,"abstract":"This paper is concerned with the double sensitive fault detection filter for positive Markovian jump systems. A new hybrid adaptive event-triggered mechanism is proposed by introducing a non-monotonic adaptive law. A linear adaptive event-triggered threshold is established by virtue of 1-norm inequality. Under such a triggering strategy, the original system can be transformed into an interval uncertain system. By using a stochastic copositive Lyapunov function, an asynchronous fault detection filter is designed for positive Markovian jump systems (PMJSs) in terms of linear programming. The presented filter satisfies both \u0000<tex>$L_{-}$</tex>\u0000-gain (\u0000<tex>$ell_{-}$</tex>\u0000-gain) fault sensitivity and \u0000<tex>$L_{1} (ell_{1})$</tex>\u0000 internal differential privacy sensitivity. The proposed approach is also extended to the discrete-time case. Finally, two examples are provided to illustrate the effectiveness of the proposed design.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this article, a general Lyapunov stability theory of nonlinear systems is put forward and it contains asymptotic/finite-time/fast finite-time/fixed-time stability. Especially, a more accurate estimate of the settling-time function is exhibited for fixed-time stability, and it is still extraneous to the initial conditions. This can be applied to obtain less conservative convergence time of the practical systems without the information of the initial conditions. As an application, the given fixed-time stability theorem is used to resolve time-varying (TV) convex optimization problem. By the Newton's method, two classes of new dynamical systems are constructed to guarantee that the solution of the dynamic system can track to the optimal trajectory of the unconstrained and equality constrained TV convex optimization problems in fixed time, respectively. Without the exact knowledge of the time derivative of the cost function gradient, a fixed-time dynamical non-smooth system is established to overcome the issue of robust TV convex optimization. Two examples are provided to illustrate the effectiveness of the proposed TV convex optimization algorithms. Subsequently, the fixed-time stability theory is extended to the theories of predefined-time/practical predefined-time stability whose bound of convergence time can be arbitrarily given in advance, without tuning the system parameters. Under which, TV convex optimization problem is solved. The previous two examples are used to demonstrate the validity of the predefined-time TV convex optimization algorithms.
本文提出了非线性系统的一般李雅普诺夫稳定性理论，它包含渐近/有限时间/快速有限时间/固定时间稳定性。特别是在定时稳定性方面，展示了对沉降时间函数更精确的估计，而且它仍然与初始条件无关。这可以用于在不考虑初始条件信息的情况下，获得实际系统的较少保守收敛时间。在应用中，给出的定时稳定性定理被用于解决时变（TV）凸优化问题。通过牛顿法，构建了两类新的动力系统，分别保证动力系统的解能在固定时间内跟踪到无约束和等约束 TV 凸优化问题的最优轨迹。在不确切知道代价函数梯度时间导数的情况下，建立了固定时间动态非光滑系统，从而克服了鲁棒 TV 凸优化问题。本文举了两个例子来说明所提出的 TV 凸优化算法的有效性。随后，固定时间稳定性理论被扩展到预定时间/实用预定时间稳定性理论，其收敛时间的边界可以事先任意给出，无需调整系统参数。在此基础上，TV 凸优化问题得以求解。前面两个例子用来证明预定义时间 TV 凸优化算法的有效性。
{"title":"General Lyapunov Stability and its Application to Time-Varying Convex Optimization","authors":"Zhibao Song;Ping Li","doi":"10.1109/JAS.2024.124374","DOIUrl":"https://doi.org/10.1109/JAS.2024.124374","url":null,"abstract":"In this article, a general Lyapunov stability theory of nonlinear systems is put forward and it contains asymptotic/finite-time/fast finite-time/fixed-time stability. Especially, a more accurate estimate of the settling-time function is exhibited for fixed-time stability, and it is still extraneous to the initial conditions. This can be applied to obtain less conservative convergence time of the practical systems without the information of the initial conditions. As an application, the given fixed-time stability theorem is used to resolve time-varying (TV) convex optimization problem. By the Newton's method, two classes of new dynamical systems are constructed to guarantee that the solution of the dynamic system can track to the optimal trajectory of the unconstrained and equality constrained TV convex optimization problems in fixed time, respectively. Without the exact knowledge of the time derivative of the cost function gradient, a fixed-time dynamical non-smooth system is established to overcome the issue of robust TV convex optimization. Two examples are provided to illustrate the effectiveness of the proposed TV convex optimization algorithms. Subsequently, the fixed-time stability theory is extended to the theories of predefined-time/practical predefined-time stability whose bound of convergence time can be arbitrarily given in advance, without tuning the system parameters. Under which, TV convex optimization problem is solved. The previous two examples are used to demonstrate the validity of the predefined-time TV convex optimization algorithms.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, This letter presents a new transfer learning framework for the deep multi-agent reinforcement learning (DMARL) to reduce the convergence difficulty and training time when applying DMARL to a new scenario [1], [2]. The proposed transfer learning framework includes the design of neural network architecture, curriculum transfer learning (CTL) and strategy distillation. Experimental results demonstrate that our framework enables DMARL models to converge faster while improving the final performance.
{"title":"A Transfer Learning Framework for Deep Multi-Agent Reinforcement Learning","authors":"Yi Liu;Xiang Wu;Yuming Bo;Jiacun Wang;Lifeng Ma","doi":"10.1109/JAS.2023.124173","DOIUrl":"https://doi.org/10.1109/JAS.2023.124173","url":null,"abstract":"Dear Editor, This letter presents a new transfer learning framework for the deep multi-agent reinforcement learning (DMARL) to reduce the convergence difficulty and training time when applying DMARL to a new scenario [1], [2]. The proposed transfer learning framework includes the design of neural network architecture, curriculum transfer learning (CTL) and strategy distillation. Experimental results demonstrate that our framework enables DMARL models to converge faster while improving the final performance.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10707687","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, This letter investigates the output tracking control issue of networked control systems (NCSs) with communication constraints and denial-of-service (DoS) attacks in the sensor-to-controller channel, both of which would induce random network delays. A dual-prediction-based compensation control (DPCC) scheme, consisting of a predictive observer and a predictive controller, is proposed to actively compensate for the adverse effect of network delays on NCSs. Compared with existing networked predictive control (NPC) methods, the DPCC scheme only requires the sensor to send a single measurement output to the controller at each sampling instant, and also does not need to know the upper bound of random network delays in advance. The stability condition of the closed-loop system is derived. Finally, numerical simulations are carried out to validate the effectiveness of the proposed scheme.
{"title":"Prediction-Based State Estimation and Compensation Control for Networked Systems with Communication Constraints and DoS Attacks","authors":"Zhong-Hua Pang;Qian Cao;Haibin Guo;Zhe Dong","doi":"10.1109/JAS.2024.124605","DOIUrl":"https://doi.org/10.1109/JAS.2024.124605","url":null,"abstract":"Dear Editor, This letter investigates the output tracking control issue of networked control systems (NCSs) with communication constraints and denial-of-service (DoS) attacks in the sensor-to-controller channel, both of which would induce random network delays. A dual-prediction-based compensation control (DPCC) scheme, consisting of a predictive observer and a predictive controller, is proposed to actively compensate for the adverse effect of network delays on NCSs. Compared with existing networked predictive control (NPC) methods, the DPCC scheme only requires the sensor to send a single measurement output to the controller at each sampling instant, and also does not need to know the upper bound of random network delays in advance. The stability condition of the closed-loop system is derived. Finally, numerical simulations are carried out to validate the effectiveness of the proposed scheme.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10707690","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kui Jiang;Ruoxi Wang;Yi Xiao;Junjun Jiang;Xin Xu;Tao Lu
Degradation under challenging conditions such as rain, haze, and low light not only diminishes content visibility, but also results in additional degradation side effects, including detail occlusion and color distortion. However, current technologies have barely explored the correlation between perturbation removal and background restoration, consequently struggling to generate high-naturalness content in challenging scenarios. In this paper, we rethink the image enhancement task from the perspective of joint optimization: Perturbation removal and texture reconstruction. To this end, we advise an efficient yet effective image enhancement model, termed the perturbation-guided texture reconstruction network (PerTeRNet). It contains two sub-networks designed for the perturbation elimination and texture reconstruction tasks, respectively. To facilitate texture recovery, we develop a novel perturbation-guided texture enhancement module (PerTEM) to connect these two tasks, where informative background features are extracted from the input with the guidance of predicted perturbation priors. To alleviate the learning burden and computational cost, we suggest performing perturbation removal in a sub-space and exploiting super-resolution to infer high-frequency background details. Our PerTeRNet has demonstrated significant superiority over typical methods in both quantitative and qualitative measures, as evidenced by extensive experimental results on popular image enhancement and joint detection tasks. The source code is available at https://github.com/kuijiang94/PerTeRNet.
{"title":"Image Enhancement via Associated Perturbation Removal and Texture Reconstruction Learning","authors":"Kui Jiang;Ruoxi Wang;Yi Xiao;Junjun Jiang;Xin Xu;Tao Lu","doi":"10.1109/JAS.2024.124521","DOIUrl":"https://doi.org/10.1109/JAS.2024.124521","url":null,"abstract":"Degradation under challenging conditions such as rain, haze, and low light not only diminishes content visibility, but also results in additional degradation side effects, including detail occlusion and color distortion. However, current technologies have barely explored the correlation between perturbation removal and background restoration, consequently struggling to generate high-naturalness content in challenging scenarios. In this paper, we rethink the image enhancement task from the perspective of joint optimization: Perturbation removal and texture reconstruction. To this end, we advise an efficient yet effective image enhancement model, termed the perturbation-guided texture reconstruction network (PerTeRNet). It contains two sub-networks designed for the perturbation elimination and texture reconstruction tasks, respectively. To facilitate texture recovery, we develop a novel perturbation-guided texture enhancement module (PerTEM) to connect these two tasks, where informative background features are extracted from the input with the guidance of predicted perturbation priors. To alleviate the learning burden and computational cost, we suggest performing perturbation removal in a sub-space and exploiting super-resolution to infer high-frequency background details. Our PerTeRNet has demonstrated significant superiority over typical methods in both quantitative and qualitative measures, as evidenced by extensive experimental results on popular image enhancement and joint detection tasks. The source code is available at https://github.com/kuijiang94/PerTeRNet.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the development of linear quadratic regulator (LQR) algorithms, the Riccati equation approach offers two important characteristics —it is recursive and readily meets the existence condition. However, these attributes are applicable only to transformed singular systems, and the efficiency of the regulator may be undermined if constraints are violated in nonsingular versions. To address this gap, we introduce a direct approach to the LQR problem for linear singular systems, avoiding the need for any transformations and eliminating the need for regularity assumptions. To achieve this goal, we begin by formulating a quadratic cost function to derive the LQR algorithm through a penalized and weighted regression framework and then connect it to a constrained minimization problem using the Bellman's criterion. Then, we employ a dynamic programming strategy in a backward approach within a finite horizon to develop an LQR algorithm for the original system. To accomplish this, we address the stability and convergence analysis under the reachability and observability assumptions of a hypothetical system constructed by the pencil of augmented matrices and connected using the Hamiltonian diagonalization technique.
{"title":"Revisiting the LQR Problem of Singular Systems","authors":"Komeil Nosrati;Juri Belikov;Aleksei Tepljakov;Eduard Petlenkov","doi":"10.1109/JAS.2024.124665","DOIUrl":"https://doi.org/10.1109/JAS.2024.124665","url":null,"abstract":"In the development of linear quadratic regulator (LQR) algorithms, the Riccati equation approach offers two important characteristics —it is recursive and readily meets the existence condition. However, these attributes are applicable only to transformed singular systems, and the efficiency of the regulator may be undermined if constraints are violated in nonsingular versions. To address this gap, we introduce a direct approach to the LQR problem for linear singular systems, avoiding the need for any transformations and eliminating the need for regularity assumptions. To achieve this goal, we begin by formulating a quadratic cost function to derive the LQR algorithm through a penalized and weighted regression framework and then connect it to a constrained minimization problem using the Bellman's criterion. Then, we employ a dynamic programming strategy in a backward approach within a finite horizon to develop an LQR algorithm for the original system. To accomplish this, we address the stability and convergence analysis under the reachability and observability assumptions of a hypothetical system constructed by the pencil of augmented matrices and connected using the Hamiltonian diagonalization technique.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Deep reinforcement learning (DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management. However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata, which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications. First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units (PDMUs), and a reverse breadth-first search (BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.
{"title":"Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems","authors":"Min Yang;Guanjun Liu;Ziyuan Zhou;Jiacun Wang","doi":"10.1109/JAS.2024.124818","DOIUrl":"https://doi.org/10.1109/JAS.2024.124818","url":null,"abstract":"Deep reinforcement learning (DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management. However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata, which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications. First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units (PDMUs), and a reverse breadth-first search (BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuanxin Lin;Zhiwen Yu;Kaixiang Yang;Ziwei Fan;C. L. Philip Chen
Multi-label classification is a challenging problem that has attracted significant attention from researchers, particularly in the domain of image and text attribute annotation. However, multi-label datasets are prone to serious intra-class and inter-class imbalance problems, which can significantly degrade the classification performance. To address the above issues, we propose the multi-label weighted broad learning system (MLW-BLS) from the perspective of label imbalance weighting and label correlation mining. Further, we propose the multi-label adaptive weighted broad learning system (MLAW-BLS) to adaptively adjust the specific weights and values of labels of MLW-BLS and construct an efficient imbalanced classifier set. Extensive experiments are conducted on various datasets to evaluate the effectiveness of the proposed model, and the results demonstrate its superiority over other advanced approaches.
{"title":"Boosting Adaptive Weighted Broad Learning System for Multi-Label Learning","authors":"Yuanxin Lin;Zhiwen Yu;Kaixiang Yang;Ziwei Fan;C. L. Philip Chen","doi":"10.1109/JAS.2024.124557","DOIUrl":"https://doi.org/10.1109/JAS.2024.124557","url":null,"abstract":"Multi-label classification is a challenging problem that has attracted significant attention from researchers, particularly in the domain of image and text attribute annotation. However, multi-label datasets are prone to serious intra-class and inter-class imbalance problems, which can significantly degrade the classification performance. To address the above issues, we propose the multi-label weighted broad learning system (MLW-BLS) from the perspective of label imbalance weighting and label correlation mining. Further, we propose the multi-label adaptive weighted broad learning system (MLAW-BLS) to adaptively adjust the specific weights and values of labels of MLW-BLS and construct an efficient imbalanced classifier set. Extensive experiments are conducted on various datasets to evaluate the effectiveness of the proposed model, and the results demonstrate its superiority over other advanced approaches.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
High-dimensional and incomplete (HDI) matrices are primarily generated in all kinds of big-data-related practical applications. A latent factor analysis (LFA) model is capable of conducting efficient representation learning to an HDI matrix, whose hyper-parameter adaptation can be implemented through a particle swarm optimizer (PSO) to meet scalable requirements. However, conventional PSO is limited by its premature issues, which leads to the accuracy loss of a resultant LFA model. To address this thorny issue, this study merges the information of each particle's state migration into its evolution process following the principle of a generalized momentum method for improving its search ability, thereby building a state-migration particle swarm optimizer (SPSO), whose theoretical convergence is rigorously proved in this study. It is then incorporated into an LFA model for implementing efficient hyper-parameter adaptation without accuracy loss. Experiments on six HDI matrices indicate that an SPSO-incorporated LFA model outperforms state-of-the-art LFA models in terms of prediction accuracy for missing data of an HDI matrix with competitive computational efficiency. Hence, SPSO's use ensures efficient and reliable hyper-parameter adaptation in an LFA model, thus ensuring practicality and accurate representation learning for HDI matrices.
{"title":"A State-Migration Particle Swarm Optimizer for Adaptive Latent Factor Analysis of High-Dimensional and Incomplete Data","authors":"Jiufang Chen;Kechen Liu;Xin Luo;Ye Yuan;Khaled Sedraoui;Yusuf Al-Turki;MengChu Zhou","doi":"10.1109/JAS.2024.124575","DOIUrl":"https://doi.org/10.1109/JAS.2024.124575","url":null,"abstract":"High-dimensional and incomplete (HDI) matrices are primarily generated in all kinds of big-data-related practical applications. A latent factor analysis (LFA) model is capable of conducting efficient representation learning to an HDI matrix, whose hyper-parameter adaptation can be implemented through a particle swarm optimizer (PSO) to meet scalable requirements. However, conventional PSO is limited by its premature issues, which leads to the accuracy loss of a resultant LFA model. To address this thorny issue, this study merges the information of each particle's state migration into its evolution process following the principle of a generalized momentum method for improving its search ability, thereby building a state-migration particle swarm optimizer (SPSO), whose theoretical convergence is rigorously proved in this study. It is then incorporated into an LFA model for implementing efficient hyper-parameter adaptation without accuracy loss. Experiments on six HDI matrices indicate that an SPSO-incorporated LFA model outperforms state-of-the-art LFA models in terms of prediction accuracy for missing data of an HDI matrix with competitive computational efficiency. Hence, SPSO's use ensures efficient and reliable hyper-parameter adaptation in an LFA model, thus ensuring practicality and accurate representation learning for HDI matrices.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Knowledge distillation (KD) enhances student network generalization by transferring dark knowledge from a complex teacher network. To optimize computational expenditure and memory utilization, self-knowledge distillation (SKD) extracts dark knowledge from the model itself rather than an external teacher network. However, previous SKD methods performed distillation indiscriminately on full datasets, overlooking the analysis of representative samples. In this work, we present a novel two-stage approach to providing targeted knowledge on specific samples, named two-stage approach self-knowledge distillation (TOAST). We first soften the hard targets using class medoids generated based on logit vectors per class. Then, we iteratively distill the under-trained data with past predictions of half the batch size. The two-stage knowledge is linearly combined, efficiently enhancing model performance. Extensive experiments conducted on five backbone architectures show our method is model-agnostic and achieves the best generalization performance. Besides, TOAST is strongly compatible with existing augmentation-based regularization methods. Our method also obtains a speedup of up to 2.95x compared with a recent state-of-the-art method.
{"title":"Two-Stage Approach for Targeted Knowledge Transfer in Self-Knowledge Distillation","authors":"Zimo Yin;Jian Pu;Yijie Zhou;Xiangyang Xue","doi":"10.1109/JAS.2024.124629","DOIUrl":"https://doi.org/10.1109/JAS.2024.124629","url":null,"abstract":"Knowledge distillation (KD) enhances student network generalization by transferring dark knowledge from a complex teacher network. To optimize computational expenditure and memory utilization, self-knowledge distillation (SKD) extracts dark knowledge from the model itself rather than an external teacher network. However, previous SKD methods performed distillation indiscriminately on full datasets, overlooking the analysis of representative samples. In this work, we present a novel two-stage approach to providing targeted knowledge on specific samples, named two-stage approach self-knowledge distillation (TOAST). We first soften the hard targets using class medoids generated based on logit vectors per class. Then, we iteratively distill the under-trained data with past predictions of half the batch size. The two-stage knowledge is linearly combined, efficiently enhancing model performance. Extensive experiments conducted on five backbone architectures show our method is model-agnostic and achieves the best generalization performance. Besides, TOAST is strongly compatible with existing augmentation-based regularization methods. Our method also obtains a speedup of up to 2.95x compared with a recent state-of-the-art method.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：