Educational data mining (EDM) offers an effective solution to predict students’ course grades in the next term. Conventional grade prediction methods can be viewed as regressing an expectation of the probability distribution of the student's grade, typically called single-value grade prediction. The reliable prediction outcomes of these methods depend on the complete input information related to students. However, next-term grade prediction often encounters the challenge of incomplete input information due to the inaccessibility of future data and the privacy of data. In this scenario, single-value grade prediction struggles to assess students’ academic status, as it may not be represented and assessed by relying on a singular expectation value. This limitation increases the risk of misjudgment, and may lead to errors in educational decision-making. Considering the challenge of collecting complete input information, we shift from traditional single-value predictions to forecasting the explicit probability distribution of the course grade. The probability distribution of the grade can assess the students’ academic status by providing probabilities corresponding to all possible grade values rather than relying solely on an expectation value, which offers the foundation to support the educators’ decision-making. In this article, the course grade distribution prediction (CGDP) model is proposed, aiming to estimate an explicit conditional probability distribution of course grades in the next term. This model can identify at-risk students, offering comprehensive decision-making information for educators and students. To ensure precise distribution predictions, a calibration method is also employed to improve the alignment between predicted and actual probabilities. Experimental results verify the effectiveness of the proposed model in early grade warning for undergraduates, based on real university data.
{"title":"A Novel Grades Prediction Method for Undergraduate Students by Learning Explicit Conditional Distribution","authors":"Na Zhang;Ming Liu;Lin Wang;Shuangrong Liu;Runyuan Sun;Bo Yang;Shenghui Zhu;Chengdong Li;Cheng Yang;Yuhu Cheng","doi":"10.1109/TAI.2024.3416077","DOIUrl":"https://doi.org/10.1109/TAI.2024.3416077","url":null,"abstract":"Educational data mining (EDM) offers an effective solution to predict students’ course grades in the next term. Conventional grade prediction methods can be viewed as regressing an expectation of the probability distribution of the student's grade, typically called single-value grade prediction. The reliable prediction outcomes of these methods depend on the complete input information related to students. However, next-term grade prediction often encounters the challenge of incomplete input information due to the inaccessibility of future data and the privacy of data. In this scenario, single-value grade prediction struggles to assess students’ academic status, as it may not be represented and assessed by relying on a singular expectation value. This limitation increases the risk of misjudgment, and may lead to errors in educational decision-making. Considering the challenge of collecting complete input information, we shift from traditional single-value predictions to forecasting the explicit probability distribution of the course grade. The probability distribution of the grade can assess the students’ academic status by providing probabilities corresponding to all possible grade values rather than relying solely on an expectation value, which offers the foundation to support the educators’ decision-making. In this article, the course grade distribution prediction (CGDP) model is proposed, aiming to estimate an explicit conditional probability distribution of course grades in the next term. This model can identify at-risk students, offering comprehensive decision-making information for educators and students. To ensure precise distribution predictions, a calibration method is also employed to improve the alignment between predicted and actual probabilities. Experimental results verify the effectiveness of the proposed model in early grade warning for undergraduates, based on real university data.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 9","pages":"4837-4848"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142169721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-18DOI: 10.1109/TAI.2024.3415549
Huai-Ning Wu;Xiao-Yan Jiang;Mi Wang
Since human beings are of limited reasoning ability as well as the machines do not usually know human intentions, how to learn human cognitive levels in shared control to enhance the machines’ intelligence is a challenging issue. In this study, this issue is addressed in the context of human–machine shared control for a class of human-in-the-loop (HiTL) systems based on a differential game with bounded rationality and incomplete information. Initially, we formulate the human–machine shared control problem as a two-player nonzero-sum linear quadratic dynamic game (LQDG), where the weighting matrix of the cost function representing the human intention is unknown for the machine. To model the human bounded rationality, the level- $boldsymbol{k}$ (LK) approach is employed to set up the LK control policies of two players performing the corresponding steps of strategic thinking. To infer the human intention, an online adaptive inverse optimal control (IOC) algorithm is then developed by using the system state data, so that the control policies of different cognitive levels can be computed. In addition, a reinforcement learning method is proposed for the machine to identify the distribution of the human cognitive levels while providing a proactive collaborative control to assist the human in a probabilistic switching way. Finally, simulation results on a cooperative shared control driver assistance system (DAS) illustrate the efficacy of the proposed approach.
{"title":"Human Cognitive Learning in Shared Control via Differential Game With Bounded Rationality and Incomplete Information","authors":"Huai-Ning Wu;Xiao-Yan Jiang;Mi Wang","doi":"10.1109/TAI.2024.3415549","DOIUrl":"https://doi.org/10.1109/TAI.2024.3415549","url":null,"abstract":"Since human beings are of limited reasoning ability as well as the machines do not usually know human intentions, how to learn human cognitive levels in shared control to enhance the machines’ intelligence is a challenging issue. In this study, this issue is addressed in the context of human–machine shared control for a class of human-in-the-loop (HiTL) systems based on a differential game with bounded rationality and incomplete information. Initially, we formulate the human–machine shared control problem as a two-player nonzero-sum linear quadratic dynamic game (LQDG), where the weighting matrix of the cost function representing the human intention is unknown for the machine. To model the human bounded rationality, the level-\u0000<inline-formula><tex-math>$boldsymbol{k}$</tex-math></inline-formula>\u0000 (LK) approach is employed to set up the LK control policies of two players performing the corresponding steps of strategic thinking. To infer the human intention, an online adaptive inverse optimal control (IOC) algorithm is then developed by using the system state data, so that the control policies of different cognitive levels can be computed. In addition, a reinforcement learning method is proposed for the machine to identify the distribution of the human cognitive levels while providing a proactive collaborative control to assist the human in a probabilistic switching way. Finally, simulation results on a cooperative shared control driver assistance system (DAS) illustrate the efficacy of the proposed approach.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"5141-5152"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142443001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The addition of dendritic inhibition has been shown to significantly enhance the computational and representational capabilities of neurons. However, this inhibitory mechanism is mostly ignored in the existing artificial neural networks (ANNs). In this article, we propose the alternating excitatory and inhibitory mechanisms and use them to construct an ANN-based dendritic neuron, the alternating excitation–inhibition dendritic neuron model (ADNM). Subsequently, a comprehensive multilayer neural system named the alternating excitation–inhibition dendritic neuron system (ADNS) is constructed by networking multiple ADNMs. To evaluate the performance of ADNS, a series of extensive experiments are implemented to compare it with other state-of-the-art networks on a diverse set consisting of 47 feature-based classification datasets and two image-based classification datasets. The experimental results demonstrate that ADNS outperforms its competitors in classification tasks. In addition, the impact of different hyperparameters on the performance of the neural model is analyzed and discussed. In summary, the study provides a novel dendritic neuron model (DNM) with better performance and interpretability for practical classification tasks.
{"title":"Alternating Excitation–Inhibition Dendritic Computing for Classification","authors":"Jiayi Li;Zhenyu Lei;Zhiming Zhang;Haotian Li;Yuki Todo;Shangce Gao","doi":"10.1109/TAI.2024.3416236","DOIUrl":"https://doi.org/10.1109/TAI.2024.3416236","url":null,"abstract":"The addition of dendritic inhibition has been shown to significantly enhance the computational and representational capabilities of neurons. However, this inhibitory mechanism is mostly ignored in the existing artificial neural networks (ANNs). In this article, we propose the alternating excitatory and inhibitory mechanisms and use them to construct an ANN-based dendritic neuron, the alternating excitation–inhibition dendritic neuron model (ADNM). Subsequently, a comprehensive multilayer neural system named the alternating excitation–inhibition dendritic neuron system (ADNS) is constructed by networking multiple ADNMs. To evaluate the performance of ADNS, a series of extensive experiments are implemented to compare it with other state-of-the-art networks on a diverse set consisting of 47 feature-based classification datasets and two image-based classification datasets. The experimental results demonstrate that ADNS outperforms its competitors in classification tasks. In addition, the impact of different hyperparameters on the performance of the neural model is analyzed and discussed. In summary, the study provides a novel dendritic neuron model (DNM) with better performance and interpretability for practical classification tasks.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5431-5441"},"PeriodicalIF":0.0,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-13DOI: 10.1109/TAI.2024.3410928
Zeenat Firdosh Quadri;M. Saqib Akhoon;Sajad A. Loan
In this work, we propose a novel hybrid architecture for epileptic seizure prediction, utilizing a deep learning approach by stacking the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) layers. The proposed approach employs a series of 1-D convolution layers, each with several filters with lengths varying exponentially. The deep Bi-LSTM layers are subsequently integrated to the design to create a densely connected feed-forward structure. The model effectively prioritizes spatiotemporal information, thus extracting key insights for identification of interictal and preictal features. The Boston Children’s Hospital–MIT datasets (Children’s Hospital Boston-Massachusetts Institute of Technology (CHB-MIT)) are utilized and fivefold cross validation is applied for training the model. The proposed model has undergone comprehensive evaluations, with sensitivity of 97.63%, precision of 98.30%, F1-Score of 98.25%, and an area under curve (AUC)-receiver operating characteristic (ROC) of 0.9 across six patients. It can predict seizures 30 min before their onset, allowing individuals ample time to take preventive measures. Compared to the state-of-the-art approach, our model achieves a higher accuracy by 3.44% and demonstrating improved prediction times.
{"title":"Epileptic Seizure Prediction Using Stacked CNN-BiLSTM: A Novel Approach","authors":"Zeenat Firdosh Quadri;M. Saqib Akhoon;Sajad A. Loan","doi":"10.1109/TAI.2024.3410928","DOIUrl":"https://doi.org/10.1109/TAI.2024.3410928","url":null,"abstract":"In this work, we propose a novel hybrid architecture for epileptic seizure prediction, utilizing a deep learning approach by stacking the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) layers. The proposed approach employs a series of 1-D convolution layers, each with several filters with lengths varying exponentially. The deep Bi-LSTM layers are subsequently integrated to the design to create a densely connected feed-forward structure. The model effectively prioritizes spatiotemporal information, thus extracting key insights for identification of interictal and preictal features. The Boston Children’s Hospital–MIT datasets (Children’s Hospital Boston-Massachusetts Institute of Technology (CHB-MIT)) are utilized and fivefold cross validation is applied for training the model. The proposed model has undergone comprehensive evaluations, with sensitivity of 97.63%, precision of 98.30%, F1-Score of 98.25%, and an area under curve (AUC)-receiver operating characteristic (ROC) of 0.9 across six patients. It can predict seizures 30 min before their onset, allowing individuals ample time to take preventive measures. Compared to the state-of-the-art approach, our model achieves a higher accuracy by 3.44% and demonstrating improved prediction times.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5553-5560"},"PeriodicalIF":0.0,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-13DOI: 10.1109/TAI.2024.3413692
Zijian Gao;Kele Xu;Yuanzhao Zhai;Bo Ding;Dawei Feng;Xinjun Mao;Huaimin Wang
In sparse extrinsic reward settings, reinforcement learning remains a challenge despite increasing interest in this field. Existing approaches suggest that intrinsic rewards can alleviate issues caused by reward sparsity. However, many studies overlook the critical role of temporal information, essential for human curiosity. This article introduces a novel intrinsic reward mechanism inspired by human learning processes, where curiosity is evaluated by comparing current observations with historical knowledge. Our method involves training a self-supervised prediction model, periodically saving snapshots of the model parameters, and employing the nuclear norm to assess the temporal inconsistency between predictions from different snapshots as intrinsic rewards. Additionally, we propose a variational weighting mechanism to adaptively assign weights to the snapshots, enhancing the model's robustness and performance. Experimental results across various benchmark environments demonstrate the efficacy of our approach, which outperforms other state-of-the-art methods without incurring additional training costs and exhibits higher noise tolerance. Our findings indicate that leveraging temporal information in intrinsic rewards can significantly improve exploration performance, motivating future research to develop more robust and accurate reward systems for reinforcement learning.
{"title":"Self-Supervised Exploration via Temporal Inconsistency in Reinforcement Learning","authors":"Zijian Gao;Kele Xu;Yuanzhao Zhai;Bo Ding;Dawei Feng;Xinjun Mao;Huaimin Wang","doi":"10.1109/TAI.2024.3413692","DOIUrl":"https://doi.org/10.1109/TAI.2024.3413692","url":null,"abstract":"In sparse extrinsic reward settings, reinforcement learning remains a challenge despite increasing interest in this field. Existing approaches suggest that intrinsic rewards can alleviate issues caused by reward sparsity. However, many studies overlook the critical role of temporal information, essential for human curiosity. This article introduces a novel intrinsic reward mechanism inspired by human learning processes, where curiosity is evaluated by comparing current observations with historical knowledge. Our method involves training a self-supervised prediction model, periodically saving snapshots of the model parameters, and employing the nuclear norm to assess the temporal inconsistency between predictions from different snapshots as intrinsic rewards. Additionally, we propose a variational weighting mechanism to adaptively assign weights to the snapshots, enhancing the model's robustness and performance. Experimental results across various benchmark environments demonstrate the efficacy of our approach, which outperforms other state-of-the-art methods without incurring additional training costs and exhibits higher noise tolerance. Our findings indicate that leveraging temporal information in intrinsic rewards can significantly improve exploration performance, motivating future research to develop more robust and accurate reward systems for reinforcement learning.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5530-5539"},"PeriodicalIF":0.0,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600432","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ridge regression (RR)-based methods aim to obtain a low-dimensional subspace for feature extraction. However, the subspace's dimensionality does not exceed the number of data categories, hence compromising its capability of feature representation. Moreover, these methods with $L_{2}$ -norm metric and regularization cannot extract highly robust features from data with corruption. To address these problems, in this article, we propose generalized jointly sparse linear discriminant regression (GOAL), a novel regression method based on joint $L_{2,1}$ -norm and capped- $L_{2}$ -norm, which can integrate sparsity, locality, and discriminability into one model to learn a full-rank robust feature extractor. The sparsely selected discriminative features are robust enough to characterize the decision boundary between classes. Locality is related to manifold structure and Laplacian smoothing, which can enhance the robustness of the model. By using the multinorm metric and regularization regression framework, the proposed method obtains the projection with joint sparsity and guarantees that the rank of the projection matrix will not be limited by the number of classes. An iterative algorithm is proposed to compute the optimal solution. Complexity analysis and proofs of convergence are also given in the article. Experiments on well-known datasets demonstrate our model's superiority and generalization ability.
{"title":"GOAL: Generalized Jointly Sparse Linear Discriminant Regression for Feature Extraction","authors":"Haoquan Lu;Zhihui Lai;Junhong Zhang;Zhuozhen Yu;Jiajun Wen","doi":"10.1109/TAI.2024.3412862","DOIUrl":"https://doi.org/10.1109/TAI.2024.3412862","url":null,"abstract":"Ridge regression (RR)-based methods aim to obtain a low-dimensional subspace for feature extraction. However, the subspace's dimensionality does not exceed the number of data categories, hence compromising its capability of feature representation. Moreover, these methods with \u0000<inline-formula><tex-math>$L_{2}$</tex-math></inline-formula>\u0000-norm metric and regularization cannot extract highly robust features from data with corruption. To address these problems, in this article, we propose generalized jointly sparse linear discriminant regression (GOAL), a novel regression method based on joint \u0000<inline-formula><tex-math>$L_{2,1}$</tex-math></inline-formula>\u0000-norm and capped-\u0000<inline-formula><tex-math>$L_{2}$</tex-math></inline-formula>\u0000-norm, which can integrate sparsity, locality, and discriminability into one model to learn a full-rank robust feature extractor. The sparsely selected discriminative features are robust enough to characterize the decision boundary between classes. Locality is related to manifold structure and Laplacian smoothing, which can enhance the robustness of the model. By using the multinorm metric and regularization regression framework, the proposed method obtains the projection with joint sparsity and guarantees that the rank of the projection matrix will not be limited by the number of classes. An iterative algorithm is proposed to compute the optimal solution. Complexity analysis and proofs of convergence are also given in the article. Experiments on well-known datasets demonstrate our model's superiority and generalization ability.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"4959-4971"},"PeriodicalIF":0.0,"publicationDate":"2024-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142443106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-10DOI: 10.1109/TAI.2024.3411596
J. Bhattacharya;A. Gupta;M. N. Dretsch;T. S. Denney;G. Deshpande
In recent years, there has been an upsurge in artificial intelligence (AI) systems. These systems, along with efficient performance and predictability, also need to incorporate the power of explainability and interpretability. This can significantly aid clinical decision support by providing explainable predictions to assist clinicians. Explainability generally involves uncovering key input features important for classification. However, characterizing the uncertainty underlying the decisions of the AI system is an important aspect needed for interpreting the decisions. This is especially important in clinical decision support systems, given considerations of medical ethics such as nonmaleficence and beneficence. In this study, we develop methods for characterizing the decision certainty of machine learning (ML)-based clinical decision support systems. As an illustrative example, we introduce a framework for ML-based posttraumatic stress disorder (PTSD) diagnostic classification that classifies the subjects into pure and mixed classes. Accordingly, a clinician can have very high confidence ( $geq$ 95% probability) about the diagnosis of a subject in a pure PTSD or combat control class. Remaining sample points for which the AI classification tool does not have very high confidence ( $<$ 95% probability) are grouped into a mixed class. Such a scheme will address ethical considerations of nonmaleficence and beneficence since the clinicians can use the AI system to identify those subjects whose diagnosis has a very high degree of confidence (and proceed with treatment accordingly), and refer those in the uncertain/mixed group to further tests. This is a novel approach, in contrast to the existing framework which aims to maximize classification.
近年来，人工智能（AI）系统激增。这些系统除了具有高效的性能和可预测性外，还需要具备可解释性和可解释性。这可以通过提供可解释的预测来协助临床医生，从而极大地帮助临床决策支持。可解释性通常包括发现对分类非常重要的关键输入特征。然而，表征人工智能系统决策背后的不确定性也是解释决策所需的一个重要方面。考虑到非渎职和受益等医学伦理因素，这一点在临床决策支持系统中尤为重要。在本研究中，我们开发了描述基于机器学习（ML）的临床决策支持系统决策确定性的方法。作为一个示例，我们介绍了基于 ML 的创伤后应激障碍（PTSD）诊断分类框架，该框架将受试者分为纯类和混合类。因此，临床医生可以有很高的信心（95% 的概率）将受试者诊断为纯创伤后应激障碍或战斗控制类。剩下的样本点，如果人工智能分类工具没有很高的可信度（95% 的概率），则被归入混合类。这样的方案可以解决非公益性和公益性的伦理问题，因为临床医生可以使用人工智能系统来识别那些诊断可信度非常高的受试者（并据此进行治疗），并将那些不确定/混合组的受试者转入进一步的测试。这是一种新颖的方法，与旨在最大化分类的现有框架形成鲜明对比。
{"title":"A Reliable Clinical Decision Support System for Posttraumatic Stress Disorder Using Functional Magnetic Resonance Imaging Data","authors":"J. Bhattacharya;A. Gupta;M. N. Dretsch;T. S. Denney;G. Deshpande","doi":"10.1109/TAI.2024.3411596","DOIUrl":"https://doi.org/10.1109/TAI.2024.3411596","url":null,"abstract":"In recent years, there has been an upsurge in artificial intelligence (AI) systems. These systems, along with efficient performance and predictability, also need to incorporate the power of explainability and interpretability. This can significantly aid clinical decision support by providing explainable predictions to assist clinicians. Explainability generally involves uncovering key input features important for classification. However, characterizing the uncertainty underlying the decisions of the AI system is an important aspect needed for interpreting the decisions. This is especially important in clinical decision support systems, given considerations of medical ethics such as nonmaleficence and beneficence. In this study, we develop methods for characterizing the decision certainty of machine learning (ML)-based clinical decision support systems. As an illustrative example, we introduce a framework for ML-based posttraumatic stress disorder (PTSD) diagnostic classification that classifies the subjects into pure and mixed classes. Accordingly, a clinician can have very high confidence (\u0000<inline-formula><tex-math>$geq$</tex-math></inline-formula>\u000095% probability) about the diagnosis of a subject in a pure PTSD or combat control class. Remaining sample points for which the AI classification tool does not have very high confidence (\u0000<inline-formula><tex-math>$<$</tex-math></inline-formula>\u000095% probability) are grouped into a mixed class. Such a scheme will address ethical considerations of nonmaleficence and beneficence since the clinicians can use the AI system to identify those subjects whose diagnosis has a very high degree of confidence (and proceed with treatment accordingly), and refer those in the uncertain/mixed group to further tests. This is a novel approach, in contrast to the existing framework which aims to maximize classification.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5605-5615"},"PeriodicalIF":0.0,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-07DOI: 10.1109/TAI.2024.3410934
Shusen Ma;Yun-Bo Zhao;Yu Kang;Peng Bai
Many real-world scenarios require accurate predictions of time series, especially in the case of long sequence time-series forecasting (LSTF), such as predicting traffic flow and electricity consumption. However, existing time-series prediction models encounter certain limitations. First, they struggle with mapping the multidimensional information present in each time step to high dimensions, resulting in information coupling and increased prediction difficulty. Second, these models fail to effectively decompose the intertwined temporal patterns within the time series, which hinders their ability to learn more predictable features. To overcome these challenges, we propose a novel end-to-end LSTF model with parallelized convolution and decomposed sparse-Transformer (PCDformer). PCDformer achieves the decoupling of input sequences by parallelizing the convolutional layers, enabling the simultaneous processing of different variables within the input sequence. To decompose distinct temporal patterns, PCDformer incorporates a temporal decomposition module within the encoder–decoder structure, effectively separating the input sequence into predictable seasonal and trend components. Additionally, to capture the correlation between variables and mitigate the impact of irrelevant information, PCDformer utilizes a sparse self-attention mechanism. Extensive experimentation conducted on five diverse datasets demonstrates the superior performance of PCDformer in LSTF tasks compared to existing approaches, particularly outperforming encoder–decoder-based models.
{"title":"Multivariate Time-Series Modeling and Forecasting With Parallelized Convolution and Decomposed Sparse-Transformer","authors":"Shusen Ma;Yun-Bo Zhao;Yu Kang;Peng Bai","doi":"10.1109/TAI.2024.3410934","DOIUrl":"https://doi.org/10.1109/TAI.2024.3410934","url":null,"abstract":"Many real-world scenarios require accurate predictions of time series, especially in the case of long sequence time-series forecasting (LSTF), such as predicting traffic flow and electricity consumption. However, existing time-series prediction models encounter certain limitations. First, they struggle with mapping the multidimensional information present in each time step to high dimensions, resulting in information coupling and increased prediction difficulty. Second, these models fail to effectively decompose the intertwined temporal patterns within the time series, which hinders their ability to learn more predictable features. To overcome these challenges, we propose a novel end-to-end LSTF model with parallelized convolution and decomposed sparse-Transformer (PCDformer). PCDformer achieves the decoupling of input sequences by parallelizing the convolutional layers, enabling the simultaneous processing of different variables within the input sequence. To decompose distinct temporal patterns, PCDformer incorporates a temporal decomposition module within the encoder–decoder structure, effectively separating the input sequence into predictable seasonal and trend components. Additionally, to capture the correlation between variables and mitigate the impact of irrelevant information, PCDformer utilizes a sparse self-attention mechanism. Extensive experimentation conducted on five diverse datasets demonstrates the superior performance of PCDformer in LSTF tasks compared to existing approaches, particularly outperforming encoder–decoder-based models.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"5232-5243"},"PeriodicalIF":0.0,"publicationDate":"2024-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-05DOI: 10.1109/TAI.2024.3409673
Jipeng Li;Xueqiong Yuan;Ercan Engin Kuruoglu
The fundamental use of neural networks is in providing a nonlinear mapping between input and output data with possibly a high number of parameters that can be learned from data directly. Consequently, studying the model's parameters, particularly the weights, is of paramount importance. The distribution and interdependencies of these weights have a direct impact on the model's generalizability, compressibility, initialization, and convergence speed. By fitting the weights of pretrained neural networks using the $alpha$ -stable distributions and conducting statistical tests, we discover widespread heavy-tailed phenomena in neural network weights, with a few layers exhibiting asymmetry. Additionally, we employ a multivariate $alpha$ -stable distribution to model the weights and explore the relationship between weights within and across layers by calculating the signed symmetric covariation coefficient. The results reveal a strong dependence among certain weights. Our findings indicate that the Gaussian assumption, symmetry assumption, and independence assumption commonly used in neural network research might be inconsistent with reality. In conclusion, our research shows three properties observed in neural network weights: heavy-tailed phenomena, asymmetry, and dependence on certain weights.
{"title":"Exploring Weight Distributions and Dependence in Neural Networks With $alpha$-Stable Distributions","authors":"Jipeng Li;Xueqiong Yuan;Ercan Engin Kuruoglu","doi":"10.1109/TAI.2024.3409673","DOIUrl":"https://doi.org/10.1109/TAI.2024.3409673","url":null,"abstract":"The fundamental use of neural networks is in providing a nonlinear mapping between input and output data with possibly a high number of parameters that can be learned from data directly. Consequently, studying the model's parameters, particularly the weights, is of paramount importance. The distribution and interdependencies of these weights have a direct impact on the model's generalizability, compressibility, initialization, and convergence speed. By fitting the weights of pretrained neural networks using the \u0000<inline-formula><tex-math>$alpha$</tex-math></inline-formula>\u0000-stable distributions and conducting statistical tests, we discover widespread heavy-tailed phenomena in neural network weights, with a few layers exhibiting asymmetry. Additionally, we employ a multivariate \u0000<inline-formula><tex-math>$alpha$</tex-math></inline-formula>\u0000-stable distribution to model the weights and explore the relationship between weights within and across layers by calculating the signed symmetric covariation coefficient. The results reveal a strong dependence among certain weights. Our findings indicate that the Gaussian assumption, symmetry assumption, and independence assumption commonly used in neural network research might be inconsistent with reality. In conclusion, our research shows three properties observed in neural network weights: heavy-tailed phenomena, asymmetry, and dependence on certain weights.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 11","pages":"5519-5529"},"PeriodicalIF":0.0,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-04DOI: 10.1109/TAI.2024.3409516
Suman De;Pabitra Mitra
Manufacturing plants are highly dependent on machines and involve a significant number of equipment to produce a finished product. Industry 4.0 helps structure the processes involved in such setups and enables the functionalities of how the equipment and machines interact with each other. With the advancement of visualizing these types of equipment as digital twins, multiple opportunities have developed for automating processes and optimizing various aspects of the assembly, especially for original equipment manufacturers (OEMs). One problem that concerns a network of manufacturers is the availability of equipment and spare parts data which are sometimes confidential but are required by a new member in the network for several analytical applications. This article looks at this problem statement to turn this into an opportunity by introducing a novel concept of AASGAN that combines the knowledge representation of a digital twin data in the asset administration shell (AAS) and a synthetic data generation technique of generative adversarial network (GAN) to generate fake data that is identical to real data. This article also explains how this concept helps perform analytical operations using industry grade solutions for the automotive industry available for managing digital twins and other scenarios for industrial automation.
{"title":"A Novel Technique of Synthetic Data Generation for Asset Administration Shells in Industry 4.0 Scenarios","authors":"Suman De;Pabitra Mitra","doi":"10.1109/TAI.2024.3409516","DOIUrl":"https://doi.org/10.1109/TAI.2024.3409516","url":null,"abstract":"Manufacturing plants are highly dependent on machines and involve a significant number of equipment to produce a finished product. Industry 4.0 helps structure the processes involved in such setups and enables the functionalities of how the equipment and machines interact with each other. With the advancement of visualizing these types of equipment as digital twins, multiple opportunities have developed for automating processes and optimizing various aspects of the assembly, especially for original equipment manufacturers (OEMs). One problem that concerns a network of manufacturers is the availability of equipment and spare parts data which are sometimes confidential but are required by a new member in the network for several analytical applications. This article looks at this problem statement to turn this into an opportunity by introducing a novel concept of AASGAN that combines the knowledge representation of a digital twin data in the asset administration shell (AAS) and a synthetic data generation technique of generative adversarial network (GAN) to generate fake data that is identical to real data. This article also explains how this concept helps perform analytical operations using industry grade solutions for the automotive industry available for managing digital twins and other scenarios for industrial automation.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 10","pages":"5258-5266"},"PeriodicalIF":0.0,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142443114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：