Pub Date : 2024-03-15DOI: 10.1109/TETCI.2024.3398024
Thanveer Shaik;Xiaohui Tao;Haoran Xie;Lin Li;Jianming Yong;Yuefeng Li
Reinforcement learning (RL) is renowned for its proficiency in modeling sequential tasks and adaptively learning latent data patterns. Deep learning models have been extensively explored and adopted in regression and classification tasks. However, deep learning has limitations, such as the assumption of equally spaced and ordered data, and the inability to incorporate graph structure in time-series prediction. Graph Neural Network (GNN) can overcome these challenges by capturing the temporal dependencies in time-series data effectively. In this study, we propose a novel approach for predicting time-series data using GNN, augmented with Reinforcement Learning(GraphRL) for monitoring. GNNs explicitly integrate the graph structure of the data into the model, enabling them to naturally capture temporal dependencies. This approach facilitates more accurate predictions in complex temporal structures, as encountered in healthcare, traffic, and weather forecasting domains. We further enhance our GraphRL model's performance through fine-tuning with a Bayesian optimization technique. The proposed framework surpasses baseline models in time-series forecasting and monitoring. This study's contributions include introducing a novel GraphRL framework for time-series prediction and demonstrating GNNs' efficacy compared to traditional deep learning models, such as Recurrent Neural Networks (RNN) and Long Short-Term Memory Networks(LSTM). Overall, this study underscores the potential of GraphRL in yielding accurate and efficient predictions within dynamic RL environments.
{"title":"Graph-Enabled Reinforcement Learning for Time Series Forecasting With Adaptive Intelligence","authors":"Thanveer Shaik;Xiaohui Tao;Haoran Xie;Lin Li;Jianming Yong;Yuefeng Li","doi":"10.1109/TETCI.2024.3398024","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3398024","url":null,"abstract":"Reinforcement learning (RL) is renowned for its proficiency in modeling sequential tasks and adaptively learning latent data patterns. Deep learning models have been extensively explored and adopted in regression and classification tasks. However, deep learning has limitations, such as the assumption of equally spaced and ordered data, and the inability to incorporate graph structure in time-series prediction. Graph Neural Network (GNN) can overcome these challenges by capturing the temporal dependencies in time-series data effectively. In this study, we propose a novel approach for predicting time-series data using GNN, augmented with Reinforcement Learning(GraphRL) for monitoring. GNNs explicitly integrate the graph structure of the data into the model, enabling them to naturally capture temporal dependencies. This approach facilitates more accurate predictions in complex temporal structures, as encountered in healthcare, traffic, and weather forecasting domains. We further enhance our GraphRL model's performance through fine-tuning with a Bayesian optimization technique. The proposed framework surpasses baseline models in time-series forecasting and monitoring. This study's contributions include introducing a novel GraphRL framework for time-series prediction and demonstrating GNNs' efficacy compared to traditional deep learning models, such as Recurrent Neural Networks (RNN) and Long Short-Term Memory Networks(LSTM). Overall, this study underscores the potential of GraphRL in yielding accurate and efficient predictions within dynamic RL environments.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/TETCI.2024.3369858
Bo Peng;Jia Zhang;Zhe Zhang;Qingming Huang;Liqun Chen;Jianjun Lei
Enhancing low-light images in an unsupervised manner has become a popular topic due to the challenge of obtaining paired real-world low/normal-light images. Driven by massive available normal-light images, learning a low-light image enhancement network from unpaired data is more practical and valuable. This paper presents an unsupervised low-light image enhancement method (DeULLE) via luminance mask and luminance-independent representation decoupling based on unpaired data. Specifically, by estimating a luminance mask from low-light image, a luminance mask-guided low-light image generation (LMLIG) module is presented to darken reference normal-light image. In addition, a luminance-independent representation-based low-light image enhancement (LRLIE) module is developed to enhance low-light image by learning luminance-independent representation and incorporating the luminance cue of reference normal-light image. With the LMLIG and LRLIE modules, a bidirectional mapping-based cycle supervision (BMCS) is constructed to facilitate the decoupling of the luminance mask and luminance-independent representation, which further promotes unsupervised low-light enhancement learning with unpaired data. Comprehensive experiments on various challenging benchmark datasets demonstrate that the proposed DeULLE exhibits superior performance.
{"title":"Unsupervised Low-Light Image Enhancement via Luminance Mask and Luminance-Independent Representation Decoupling","authors":"Bo Peng;Jia Zhang;Zhe Zhang;Qingming Huang;Liqun Chen;Jianjun Lei","doi":"10.1109/TETCI.2024.3369858","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369858","url":null,"abstract":"Enhancing low-light images in an unsupervised manner has become a popular topic due to the challenge of obtaining paired real-world low/normal-light images. Driven by massive available normal-light images, learning a low-light image enhancement network from unpaired data is more practical and valuable. This paper presents an unsupervised low-light image enhancement method (DeULLE) via luminance mask and luminance-independent representation decoupling based on unpaired data. Specifically, by estimating a luminance mask from low-light image, a luminance mask-guided low-light image generation (LMLIG) module is presented to darken reference normal-light image. In addition, a luminance-independent representation-based low-light image enhancement (LRLIE) module is developed to enhance low-light image by learning luminance-independent representation and incorporating the luminance cue of reference normal-light image. With the LMLIG and LRLIE modules, a bidirectional mapping-based cycle supervision (BMCS) is constructed to facilitate the decoupling of the luminance mask and luminance-independent representation, which further promotes unsupervised low-light enhancement learning with unpaired data. Comprehensive experiments on various challenging benchmark datasets demonstrate that the proposed DeULLE exhibits superior performance.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/TETCI.2024.3369866
Qing Song;Zilong Jia;Wenhe Jia;Wenyi Zhao;Mengjie Hu;Chun Liu
In complex long-term news videos, the fundamental component is the news excerpt which consists of many studio and interview screens. Spotting and identifying the correct news excerpt from such a complex long-term video is a challenging task. Apart from the inherent temporal semantics and the complex generic events interactions, the varied richness of semantics within the text and visual modalities further complicates matters. In this paper, we delve into the nuanced realm of video temporal understanding, examining it through a multimodal and multitask perspective. Our research involves presenting a more fine-grained challenge, which we refer to as M ultimodal News E xcerpt S egmentation and I dentification. The objective is to segment news videos into individual frame-level excerpts while accurately assigning elaborate tags to each segment by utilizing multimodal semantics. As there is an absence of multimodal fine-grained temporal segmentation dataset at present, we set up a new benchmark called News-MESI to support our research. News-MESI comprises over 150 high-quality news videos sourced from digital media, with approximately 150 hours in total and encompassing more than 2000 news excerpts. By annotating it with frame-level excerpt boundaries and an elaborate categorization hierarchy, this collection offers a valuable chance for multi-modal semantic understanding of these distinctive videos. We also present a novel algorithm employing coarse-to-fine multimodal fusion and hierarchical classification to address this problem. Extensive experiments are executed on our benchmark to show how the news content is temporally evolving in nature. Further analysis shows that multi-modal solutions are significantly superior to the single-modal solution.
{"title":"News-MESI: A Dataset for Multimodal News Excerpt Segmentation and Identification","authors":"Qing Song;Zilong Jia;Wenhe Jia;Wenyi Zhao;Mengjie Hu;Chun Liu","doi":"10.1109/TETCI.2024.3369866","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369866","url":null,"abstract":"In complex long-term news videos, the fundamental component is the news excerpt which consists of many studio and interview screens. Spotting and identifying the correct news excerpt from such a complex long-term video is a challenging task. Apart from the inherent temporal semantics and the complex generic events interactions, the varied richness of semantics within the text and visual modalities further complicates matters. In this paper, we delve into the nuanced realm of video temporal understanding, examining it through a multimodal and multitask perspective. Our research involves presenting a more fine-grained challenge, which we refer to as \u0000<bold>M</b>\u0000ultimodal News \u0000<bold>E</b>\u0000xcerpt \u0000<bold>S</b>\u0000egmentation and \u0000<bold>I</b>\u0000dentification. The objective is to segment news videos into individual frame-level excerpts while accurately assigning elaborate tags to each segment by utilizing multimodal semantics. As there is an absence of multimodal fine-grained temporal segmentation dataset at present, we set up a new benchmark called News-MESI to support our research. News-MESI comprises over 150 high-quality news videos sourced from digital media, with approximately 150 hours in total and encompassing more than 2000 news excerpts. By annotating it with frame-level excerpt boundaries and an elaborate categorization hierarchy, this collection offers a valuable chance for multi-modal semantic understanding of these distinctive videos. We also present a novel algorithm employing coarse-to-fine multimodal fusion and hierarchical classification to address this problem. Extensive experiments are executed on our benchmark to show how the news content is temporally evolving in nature. Further analysis shows that multi-modal solutions are significantly superior to the single-modal solution.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multi-view graph clustering can divide similar objects into the same category through learning the relationship among samples. To improve clustering efficiency, instead of all sample-based graph learning, the bipartite graph learning method can achieve efficient clustering by establishing the graph between data points and a few anchors, so it becomes an important research topic. However, most these bipartite graph-based multi-view clustering approaches focused on consistent information learning among views, ignored the diversity information of each view, which is not conductive to improve clustering precision. To address this issue, a diversity-induced bipartite graph fusion for multiview graph clustering (DiBGF-MGC) is proposed to simultaneously consider the consistency and diversity of multiple views. In our method, the constraint of diversity is achieved via minimizing the diversity of each view and minimizing the inconsistency of diversity in different views. The former ensures the sparse of diversity information, and the later ensures the diversity information is private information of each view. Specifically, we separate the bipartite graph to the consistent part and the divergent part in order to remove the diversity parts while preserving the consistency among multiple views. The consistent parts are used to learn the consensus bipartite graph, which can obtain a clear clustering structure due to eliminating diversity part from original bipartite graph. The diversity part is formulated by intra-view constraint and inter-views inconsistent constraint, which can better distinguish diversity part from original bipartite graph. The consistent learning and diversity learning can be improved iteratively via leveraging the results of the other one. Experiment shows that the proposed DiBGF-MGC method obtains better clustering results than state-of-the-art methods on several benchmark datasets.
{"title":"Diversity-Induced Bipartite Graph Fusion for Multiview Graph Clustering","authors":"Weiqing Yan;Xinying Zhao;Guanghui Yue;Jinlai Ren;Jindong Xu;Zhaowei Liu;Chang Tang","doi":"10.1109/TETCI.2024.3369316","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369316","url":null,"abstract":"Multi-view graph clustering can divide similar objects into the same category through learning the relationship among samples. To improve clustering efficiency, instead of all sample-based graph learning, the bipartite graph learning method can achieve efficient clustering by establishing the graph between data points and a few anchors, so it becomes an important research topic. However, most these bipartite graph-based multi-view clustering approaches focused on consistent information learning among views, ignored the diversity information of each view, which is not conductive to improve clustering precision. To address this issue, a diversity-induced bipartite graph fusion for multiview graph clustering (DiBGF-MGC) is proposed to simultaneously consider the consistency and diversity of multiple views. In our method, the constraint of diversity is achieved via minimizing the diversity of each view and minimizing the inconsistency of diversity in different views. The former ensures the sparse of diversity information, and the later ensures the diversity information is private information of each view. Specifically, we separate the bipartite graph to the consistent part and the divergent part in order to remove the diversity parts while preserving the consistency among multiple views. The consistent parts are used to learn the consensus bipartite graph, which can obtain a clear clustering structure due to eliminating diversity part from original bipartite graph. The diversity part is formulated by intra-view constraint and inter-views inconsistent constraint, which can better distinguish diversity part from original bipartite graph. The consistent learning and diversity learning can be improved iteratively via leveraging the results of the other one. Experiment shows that the proposed DiBGF-MGC method obtains better clustering results than state-of-the-art methods on several benchmark datasets.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/TETCI.2024.3369320
Tao Zhang;Tao Li;Zaifa Xue;Xin Lu;Le Gao
Random forest (RF) is a classical machine learning model, and many variants have been proposed to improve the performance or interpretability in recent years. To improve the classification performance and interpretability of RF under the premise of consistency, a novel RF variant named intervention correlation ratio random forest (ICR 2 F) is proposed. First, intervention correlation ratio (ICR) is proposed as a novel causality evaluation method by the ratio of pre- and post intervention on features which is used to select features and thresholds to divide a non-leaf node when building a decision tree. And then, decision trees are built based on ICR to construct ICR 2 F through ensemble learning. In addition, ICR 2 F is proven to satisfy consistency in exploring random forest in theory. Finally, experimental results on 20 UCI datasets have shown that ICR 2 F surpasses classical classifiers and the latest RF variants in classification performance under the premise of consistency and interpretability.
{"title":"A Novel Random Forest Variant Based on Intervention Correlation Ratio","authors":"Tao Zhang;Tao Li;Zaifa Xue;Xin Lu;Le Gao","doi":"10.1109/TETCI.2024.3369320","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369320","url":null,"abstract":"Random forest (RF) is a classical machine learning model, and many variants have been proposed to improve the performance or interpretability in recent years. To improve the classification performance and interpretability of RF under the premise of consistency, a novel RF variant named intervention correlation ratio random forest (ICR\u0000<sup>2</sup>\u0000F) is proposed. First, intervention correlation ratio (ICR) is proposed as a novel causality evaluation method by the ratio of pre- and post intervention on features which is used to select features and thresholds to divide a non-leaf node when building a decision tree. And then, decision trees are built based on ICR to construct ICR\u0000<sup>2</sup>\u0000F through ensemble learning. In addition, ICR\u0000<sup>2</sup>\u0000F is proven to satisfy consistency in exploring random forest in theory. Finally, experimental results on 20 UCI datasets have shown that ICR\u0000<sup>2</sup>\u0000F surpasses classical classifiers and the latest RF variants in classification performance under the premise of consistency and interpretability.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/TETCI.2024.3370005
Xiao-Kai Cao;Man-Sheng Chen;Chang-Dong Wang;Jian-Huang Lai;Qiong Huang;C. L. Philip Chen
Distributed computing as a widely concerned research direction needs to use the data training model of users, making the security of users' private data become a challenging problem to be solved. At present, federated learning is the mainstream research method to solve this problem. However, federated learning is not good at distributed training on streaming data. In real scenarios, the client's data is usually continuously updated streaming data. In this paper, we propose Dynamic Secure Multi Broad Network (DSMBN), which is a novel privacy computing framework completely different from federated learning. In DSMBN, we design three interactive communication protocols to handle streaming data in different scenarios. The function of the protocol is to use random mapping to encrypt data during the interaction. The protocol ensures that the client's original data does not leave the local server when generating mapped features. The central server uses the resulting mapped features (essentially encrypted data) instead of the original data to train machine learning models. In theoretical analysis, we analyze the first protocol's security, communication costs, and computational complexity. In the experiment, we design seven experimental scenarios, including quantity balance, Non-IID data distribution and streaming data, and compare them with several mainstream privacy protection machine learning methods. The experimental results show that compared with centralized training without privacy protection, DSMBN can achieve the same test accuracy under the premise of protecting private data security. Compared with mainstream federated learning methods, DSMBN can achieve higher accuracy in the Non-IID scenarios and save computing time and communication resources.
{"title":"Dynamic Secure Multi Broad Network for Privacy Preserving of Streaming Data","authors":"Xiao-Kai Cao;Man-Sheng Chen;Chang-Dong Wang;Jian-Huang Lai;Qiong Huang;C. L. Philip Chen","doi":"10.1109/TETCI.2024.3370005","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3370005","url":null,"abstract":"Distributed computing as a widely concerned research direction needs to use the data training model of users, making the security of users' private data become a challenging problem to be solved. At present, federated learning is the mainstream research method to solve this problem. However, federated learning is not good at distributed training on streaming data. In real scenarios, the client's data is usually continuously updated streaming data. In this paper, we propose Dynamic Secure Multi Broad Network (DSMBN), which is a novel privacy computing framework completely different from federated learning. In DSMBN, we design three interactive communication protocols to handle streaming data in different scenarios. The function of the protocol is to use random mapping to encrypt data during the interaction. The protocol ensures that the client's original data does not leave the local server when generating mapped features. The central server uses the resulting mapped features (essentially encrypted data) instead of the original data to train machine learning models. In theoretical analysis, we analyze the first protocol's security, communication costs, and computational complexity. In the experiment, we design seven experimental scenarios, including quantity balance, Non-IID data distribution and streaming data, and compare them with several mainstream privacy protection machine learning methods. The experimental results show that compared with centralized training without privacy protection, DSMBN can achieve the same test accuracy under the premise of protecting private data security. Compared with mainstream federated learning methods, DSMBN can achieve higher accuracy in the Non-IID scenarios and save computing time and communication resources.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/TETCI.2024.3369447
Wenhao Zhang;He Xiao;Dirui Xie;Yue Zhou;Shukai Duan;Xiaofang Hu
Recently, using the idea of non-local operations, various non-local networks and the Vision Transformer have been proposed to model the long-range pixel dependencies, addressing the limitation of Convolutional neural networks(CNNs). However, most of these models cannot adaptively process images with different resolutions, and their large number of parameters and computational complexity make them unfavorable for edge devices. In this paper, we propose an efficient Global Self-Attention Memristive Neural Network (GSA-MNN) for image restoration and present a memristive circuits implementation scheme for GSA-MNN. GSA-MNN can both extract global and local information from images, which can be flexibly applied to different resolution images. Specifically, the Global Spatial Attention Module (GSAM) and the Global Channel Attention Module (GCAM) are designed to complete the modeling and inference of global relations. The GSAM is used to model global spatial relations between the pixels of the feature maps, while the GCAM explores global relations across the channels. Moreover, a multi-scale local information extraction module is proposed to deal with image regions with complex textures. Furthermore, we provide a modular designed circuit implementation scheme for these three modules and the entire GSA-MNN. Benefiting from the programmability of the memristor crossbars, three kinds of image restoration tasks: image deraining, low-light image enhancement, and image dehazing are realized on the same circuit framework by adjusting the configuration parameters. Experimental comparisons with over 20 state-of-the-art methods on 10 public datasets show that our proposed GSA-MNN has superiority.
{"title":"A Global Self-Attention Memristive Neural Network for Image Restoration","authors":"Wenhao Zhang;He Xiao;Dirui Xie;Yue Zhou;Shukai Duan;Xiaofang Hu","doi":"10.1109/TETCI.2024.3369447","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369447","url":null,"abstract":"Recently, using the idea of non-local operations, various non-local networks and the Vision Transformer have been proposed to model the long-range pixel dependencies, addressing the limitation of Convolutional neural networks(CNNs). However, most of these models cannot adaptively process images with different resolutions, and their large number of parameters and computational complexity make them unfavorable for edge devices. In this paper, we propose an efficient Global Self-Attention Memristive Neural Network (GSA-MNN) for image restoration and present a memristive circuits implementation scheme for GSA-MNN. GSA-MNN can both extract global and local information from images, which can be flexibly applied to different resolution images. Specifically, the Global Spatial Attention Module (GSAM) and the Global Channel Attention Module (GCAM) are designed to complete the modeling and inference of global relations. The GSAM is used to model global spatial relations between the pixels of the feature maps, while the GCAM explores global relations across the channels. Moreover, a multi-scale local information extraction module is proposed to deal with image regions with complex textures. Furthermore, we provide a modular designed circuit implementation scheme for these three modules and the entire GSA-MNN. Benefiting from the programmability of the memristor crossbars, three kinds of image restoration tasks: image deraining, low-light image enhancement, and image dehazing are realized on the same circuit framework by adjusting the configuration parameters. Experimental comparisons with over 20 state-of-the-art methods on 10 public datasets show that our proposed GSA-MNN has superiority.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-13DOI: 10.1109/TETCI.2024.3369937
Yiming Huang;Chunsheng Liu;Faliang Chang;Yansha Lu
Driver fatigue is one of the main causes of traffic accidents. Current vision-based methods for detecting driver fatigue lack robustness in the presence of interfering images, and exhibit insufficient ability to focus on frames containing crucial information. To address these issues, we propose a Self-supervised Multi-granularity Graph Attention Network (SMGA-Net) for driver fatigue detection. The network mainly contains the following contributions: Firstly, with the multi-task self-supervised learning strategy, a novel method called Image Restoration based Self-supervised Learning (IRS-Learning) is proposed to enhance the network's robustness when processing interfering images. Secondly, with the graph attention mechanism, a Multi-head Graph Attention (MG-Attention) module is designed to concentrate on frames that contain crucial information by assigning importance weights to each frame. In addition, a Cross Attention Feature Fusion (CAF-Fusion) method is proposed to adaptively merge the multi-granularity features and emphasize effective information contained therein. Experiments performed on the National TsingHua University Drowsy Driver Detection (NTHU-DDD) dataset show that the proposed SMGA-Net based driver fatigue detection method outperforms the state-of-art methods.
{"title":"Self-Supervised Multi-Granularity Graph Attention Network for Vision-Based Driver Fatigue Detection","authors":"Yiming Huang;Chunsheng Liu;Faliang Chang;Yansha Lu","doi":"10.1109/TETCI.2024.3369937","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369937","url":null,"abstract":"Driver fatigue is one of the main causes of traffic accidents. Current vision-based methods for detecting driver fatigue lack robustness in the presence of interfering images, and exhibit insufficient ability to focus on frames containing crucial information. To address these issues, we propose a \u0000<italic>Self-supervised Multi-granularity Graph Attention Network</i>\u0000 (SMGA-Net) for driver fatigue detection. The network mainly contains the following contributions: Firstly, with the multi-task self-supervised learning strategy, a novel method called \u0000<italic>Image Restoration based Self-supervised Learning</i>\u0000 (IRS-Learning) is proposed to enhance the network's robustness when processing interfering images. Secondly, with the graph attention mechanism, a \u0000<italic>Multi-head Graph Attention</i>\u0000 (MG-Attention) module is designed to concentrate on frames that contain crucial information by assigning importance weights to each frame. In addition, a \u0000<italic>Cross Attention Feature Fusion</i>\u0000 (CAF-Fusion) method is proposed to adaptively merge the multi-granularity features and emphasize effective information contained therein. Experiments performed on the National TsingHua University Drowsy Driver Detection (NTHU-DDD) dataset show that the proposed SMGA-Net based driver fatigue detection method outperforms the state-of-art methods.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-13DOI: 10.1109/TETCI.2024.3369999
Junxin Chen;Xu Xu;Gwanggil Jeon;David Camacho;Ben-Guo He
Water leakage recognition plays a significant role in ensuring the safety of shield tunnel lining. However, current models cannot meet the engineering requirements because the tunnel environment is complex. In this concern, a one-stage deep learning model is developed for water leakage recognition. First, we design an attention module to reduce background noise interference. Second, an edge refinement algorithm is proposed to refine the mask of water leakage region. Furthermore, a mixed data augmentation is developed to enhance the robustness of model. Experimental results indicate an average precision (AP) is up to 60%, and a recognition speed is 26 frames per second (FPS). This determines that our proposed network is lightweight and has advantages over peer methods.
{"title":"WLR-Net: An Improved YOLO-V7 With Edge Constraints and Attention Mechanism for Water Leakage Recognition in the Tunnel","authors":"Junxin Chen;Xu Xu;Gwanggil Jeon;David Camacho;Ben-Guo He","doi":"10.1109/TETCI.2024.3369999","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369999","url":null,"abstract":"Water leakage recognition plays a significant role in ensuring the safety of shield tunnel lining. However, current models cannot meet the engineering requirements because the tunnel environment is complex. In this concern, a one-stage deep learning model is developed for water leakage recognition. First, we design an attention module to reduce background noise interference. Second, an edge refinement algorithm is proposed to refine the mask of water leakage region. Furthermore, a mixed data augmentation is developed to enhance the robustness of model. Experimental results indicate an average precision (AP) is up to 60%, and a recognition speed is 26 frames per second (FPS). This determines that our proposed network is lightweight and has advantages over peer methods.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-12DOI: 10.1109/TETCI.2024.3360331
Hao Li;Tianshi Luo;Liwen Liu;Maoguo Gong;Wenyuan Qiao;Fei Xie;A. K. Qin
Change detection in multitemporal remote sensing images aims to generate a difference image (DI) and then analyze it to identify the unchanged/changed areas. The current change detection techniques always investigate a single change detection task of two images from the image series one by one and may ignore the relevant information across the different tasks. Furthermore, theoretical results have demonstrated that the distribution of DI can be interpreted by a Rayleigh-Rice mixture model (RRMM). The parameters of RRMM are usually estimated by the expectation-maximization (EM) algorithm, which is easy to be trapped into local minima. In order to address these issues, a selective transfer based evolutionary multitasking change detection method is proposed to deal with multiple change detection tasks concurrently. For each change detection task, the log-likelihood function and centroid distance function are considered as two objectives to be optimized simultaneously. In the proposed method, a RRMM parameter estimation driven initialization method with random partition of the data is designed by maximum likelihood estimates of the parameters. Then the next population is generated by the intra-task and inter-task genetic transfer operators. A selective knowledge transfer based local search strategy is proposed to further improve the population by applying EM algorithm. In this strategy, the samples in the unchanged class of multiple tasks are utilized to estimate the parameters to acquire knowledge transferred from the other task. Experiments on three real remote sensing data sets demonstrate that the selective transfer based evolutionary multitasking change detection method is able to accelerate the convergence and achieve superior performance in terms of accuracy.
多时遥感图像中的变化检测旨在生成差分图像（DI），然后对其进行分析，以识别未改变/已改变的区域。目前的变化检测技术总是逐一研究图像系列中两幅图像的单一变化检测任务，可能会忽略不同任务中的相关信息。此外，理论研究结果表明，DI 的分布可以用 Rayleigh-Rice 混合物模型（RRMM）来解释。RRMM 的参数通常采用期望最大化（EM）算法进行估计，这种算法很容易陷入局部最小值。为了解决这些问题，我们提出了一种基于选择性转移的进化多任务变化检测方法，以同时处理多个变化检测任务。对于每个变化检测任务，对数似然函数和中心点距离函数被视为需要同时优化的两个目标。在所提出的方法中，通过对参数的最大似然估计，设计了一种 RRMM 参数估计驱动的初始化方法，对数据进行随机分区。然后通过任务内和任务间遗传转移算子生成下一个种群。我们提出了一种基于知识转移的选择性局部搜索策略，通过应用 EM 算法进一步改进种群。在这一策略中，多个任务的不变类样本被用来估计参数，以获取从其他任务转移过来的知识。在三个真实遥感数据集上的实验证明，基于选择性转移的进化多任务变化检测方法能够加快收敛速度，并在精度方面取得优异的性能。
{"title":"Selective Transfer Based Evolutionary Multitasking Optimization for Change Detection","authors":"Hao Li;Tianshi Luo;Liwen Liu;Maoguo Gong;Wenyuan Qiao;Fei Xie;A. K. Qin","doi":"10.1109/TETCI.2024.3360331","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360331","url":null,"abstract":"Change detection in multitemporal remote sensing images aims to generate a difference image (DI) and then analyze it to identify the unchanged/changed areas. The current change detection techniques always investigate a single change detection task of two images from the image series one by one and may ignore the relevant information across the different tasks. Furthermore, theoretical results have demonstrated that the distribution of DI can be interpreted by a Rayleigh-Rice mixture model (RRMM). The parameters of RRMM are usually estimated by the expectation-maximization (EM) algorithm, which is easy to be trapped into local minima. In order to address these issues, a selective transfer based evolutionary multitasking change detection method is proposed to deal with multiple change detection tasks concurrently. For each change detection task, the log-likelihood function and centroid distance function are considered as two objectives to be optimized simultaneously. In the proposed method, a RRMM parameter estimation driven initialization method with random partition of the data is designed by maximum likelihood estimates of the parameters. Then the next population is generated by the intra-task and inter-task genetic transfer operators. A selective knowledge transfer based local search strategy is proposed to further improve the population by applying EM algorithm. In this strategy, the samples in the unchanged class of multiple tasks are utilized to estimate the parameters to acquire knowledge transferred from the other task. Experiments on three real remote sensing data sets demonstrate that the selective transfer based evolutionary multitasking change detection method is able to accelerate the convergence and achieve superior performance in terms of accuracy.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：