Pub Date : 2024-12-04DOI: 10.1109/LSP.2024.3511426
Yi Liu;Haowen Hou;Fei Ma;Shiguang Ni;Fei Richard Yu
In untrimmed video tasks, identifying temporal boundaries in videos is crucial for temporal video grounding. With the emergence of multimodal large language models (MLLMs), recent studies have focused on endowing these models with the capability of temporal perception in untrimmed videos. To address the challenge, in this paper, we introduce a multimodal large language model named MLLM-TA with precise temporal perception to obtain temporal attention. Unlike the traditional MLLMs, answering temporal questions through one or two words related to temporal information, we leverage the text description proficiency of MLLMs to acquire video temporal attention with description. Specifically, we design a dual temporal-aware generative branches aimed at the visual space of the entire video and the textual space of global descriptions, simultaneously generating mutually supervised consistent temporal attention, thereby enhancing the video temporal perception capabilities of MLLMs. Finally, we evaluate our approach on both video grounding task and highlight detection task on three popular benchmarks, including Charades-STA, ActivityNet Captions and QVHighlights. The extensive results show that our MLLM-TA significantly outperforms previous approaches both on zero-shot and supervised setting, achieving state-of-the-art performance.
{"title":"MLLM-TA: Leveraging Multimodal Large Language Models for Precise Temporal Video Grounding","authors":"Yi Liu;Haowen Hou;Fei Ma;Shiguang Ni;Fei Richard Yu","doi":"10.1109/LSP.2024.3511426","DOIUrl":"https://doi.org/10.1109/LSP.2024.3511426","url":null,"abstract":"In untrimmed video tasks, identifying temporal boundaries in videos is crucial for temporal video grounding. With the emergence of multimodal large language models (MLLMs), recent studies have focused on endowing these models with the capability of temporal perception in untrimmed videos. To address the challenge, in this paper, we introduce a multimodal large language model named MLLM-TA with precise temporal perception to obtain temporal attention. Unlike the traditional MLLMs, answering temporal questions through one or two words related to temporal information, we leverage the text description proficiency of MLLMs to acquire video temporal attention with description. Specifically, we design a dual temporal-aware generative branches aimed at the visual space of the entire video and the textual space of global descriptions, simultaneously generating mutually supervised consistent temporal attention, thereby enhancing the video temporal perception capabilities of MLLMs. Finally, we evaluate our approach on both video grounding task and highlight detection task on three popular benchmarks, including Charades-STA, ActivityNet Captions and QVHighlights. The extensive results show that our MLLM-TA significantly outperforms previous approaches both on zero-shot and supervised setting, achieving state-of-the-art performance.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"281-285"},"PeriodicalIF":3.2,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142890394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Reconfigurable intelligent surfaces (RIS) offer unprecedented flexibility for smart wireless channels. Recent research shows that RIS platforms enhance signal quality, coverage, and link capacity in integrated sensing and communication (ISAC) systems. This paper explores the use of fully-connected beyond diagonal RIS (BD-RIS) in ISAC. BD-RIS provides additional degrees of freedom by allowing non-zero off-diagonal elements in the scattering matrix, enhancing functionality and performance. We aim to maximize the weighted sum of the signal-to-noise ratio (SNR) at both the radar receiver and communication users using BD-RIS. Numerical results demonstrate the advantages of BD-RIS in ISAC, significantly improving SNR for both radar and communication users.
{"title":"Beyond Diagonal RIS: Key to Next-Generation Integrated Sensing and Communications?","authors":"Tara Esmaeilbeig;Kumar Vijay Mishra;Mojtaba Soltanalian","doi":"10.1109/LSP.2024.3511395","DOIUrl":"https://doi.org/10.1109/LSP.2024.3511395","url":null,"abstract":"Reconfigurable intelligent surfaces (RIS) offer unprecedented flexibility for smart wireless channels. Recent research shows that RIS platforms enhance signal quality, coverage, and link capacity in integrated sensing and communication (ISAC) systems. This paper explores the use of fully-connected beyond diagonal RIS (BD-RIS) in ISAC. BD-RIS provides additional degrees of freedom by allowing non-zero off-diagonal elements in the scattering matrix, enhancing functionality and performance. We aim to maximize the weighted sum of the signal-to-noise ratio (SNR) at both the radar receiver and communication users using BD-RIS. Numerical results demonstrate the advantages of BD-RIS in ISAC, significantly improving SNR for both radar and communication users.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"216-220"},"PeriodicalIF":3.2,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10777522","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The explosive growth of generative AI has saturated the internet with AI-generated images, raising security concerns and increasing the need for reliable detection methods. The primary requirement for such detection is generalizability, typically achieved by training on numerous fake images from various models. However, practical limitations, such as closed-source models and restricted access, often result in limited training samples. Therefore, training a general detector with few-shot samples is essential for modern detection mechanisms. To address this challenge, we propose FAMSeC, a general AI-generated image detection method based on LoRA-based F orgery A wareness M odule and Se mantic feature-guided C ontrastive learning strategy. To effectively learn from limited samples and prevent overfitting, we developed a forgery awareness module (FAM) based on LoRA, maintaining the generalization of pre-trained features. Additionally, to cooperate with FAM, we designed a semantic feature-guided contrastive learning strategy (SeC), making the FAM focus more on the differences between real/fake image than on the features of the samples themselves. Experiments show that FAMSeC outperforms state-of-the-art method, enhancing classification accuracy by 14.55% with just 0.56% of the training samples.
{"title":"FAMSeC: A Few-Shot-Sample-Based General AI-Generated Image Detection Method","authors":"Juncong Xu;Yang Yang;Han Fang;Honggu Liu;Weiming Zhang","doi":"10.1109/LSP.2024.3511421","DOIUrl":"https://doi.org/10.1109/LSP.2024.3511421","url":null,"abstract":"The explosive growth of generative AI has saturated the internet with AI-generated images, raising security concerns and increasing the need for reliable detection methods. The primary requirement for such detection is generalizability, typically achieved by training on numerous fake images from various models. However, practical limitations, such as closed-source models and restricted access, often result in limited training samples. Therefore, training a general detector with few-shot samples is essential for modern detection mechanisms. To address this challenge, we propose FAMSeC, a general AI-generated image detection method based on LoRA-based \u0000<bold>F</b>\u0000orgery \u0000<bold>A</b>\u0000wareness \u0000<bold>M</b>\u0000odule and \u0000<bold>Se</b>\u0000mantic feature-guided \u0000<bold>C</b>\u0000ontrastive learning strategy. To effectively learn from limited samples and prevent overfitting, we developed a forgery awareness module (FAM) based on LoRA, maintaining the generalization of pre-trained features. Additionally, to cooperate with FAM, we designed a semantic feature-guided contrastive learning strategy (SeC), making the FAM focus more on the differences between real/fake image than on the features of the samples themselves. Experiments show that FAMSeC outperforms state-of-the-art method, enhancing classification accuracy by 14.55% with just 0.56% of the training samples.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"226-230"},"PeriodicalIF":3.2,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-04DOI: 10.1109/LSP.2024.3511350
Sejun Hong;Eunwoo Kim
Neural radiance fields (NeRF) represent multi-view images as 3D scenes, achieving a photo-realistic novel view synthesis quality. However, capturing multi-view images in real-world scenarios is not well aligned and often results in blur or noise. Deblur-NeRF, which uses kernel deformation to improve sharpness, is effective but the quantity of training blur samples and imbalance significantly affect the overall results. In this study, we propose neural radiance fields deblurring with active learning (NeRF-DA), focusing on high-quality blurred images for 3D scene modeling. NeRF-DA uses pool-based active learning with uncertainty estimation to improve model efficiency with a high-quality training set. Subsequently, we deblur the data using the trained model and proceed with NeRF training by selecting the best-sharpened images for querying. Experiments on both camera motion blur and defocus blur demonstrate that NeRF-DA significantly enhances the quality of the existing Deblur-NeRF.
{"title":"NeRF-DA: Neural Radiance Fields Deblurring With Active Learning","authors":"Sejun Hong;Eunwoo Kim","doi":"10.1109/LSP.2024.3511350","DOIUrl":"https://doi.org/10.1109/LSP.2024.3511350","url":null,"abstract":"Neural radiance fields (NeRF) represent multi-view images as 3D scenes, achieving a photo-realistic novel view synthesis quality. However, capturing multi-view images in real-world scenarios is not well aligned and often results in blur or noise. Deblur-NeRF, which uses kernel deformation to improve sharpness, is effective but the quantity of training blur samples and imbalance significantly affect the overall results. In this study, we propose neural radiance fields deblurring with active learning (NeRF-DA), focusing on high-quality blurred images for 3D scene modeling. NeRF-DA uses pool-based active learning with uncertainty estimation to improve model efficiency with a high-quality training set. Subsequently, we deblur the data using the trained model and proceed with NeRF training by selecting the best-sharpened images for querying. Experiments on both camera motion blur and defocus blur demonstrate that NeRF-DA significantly enhances the quality of the existing Deblur-NeRF.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"261-265"},"PeriodicalIF":3.2,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142875117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Iterative decoding algorithms based on the alternating direction method of multipliers (ADMM) decoding of low density parity check (LDPC) codes has emerged as an alternating decoding method and bringed a boom of research on drawing upon mathematical optimization to LDPC decoding. Improving error-correcting performance is a key issue to enhance the superiority of ADMM decoding. In this letter, we investigate an efficient ADMM-based iterative decoder for binary LDPC codes. First, we build an mathematical programming equivalence of the maximum likelihood (ML) decoding problem by transforming parity-check constraints to multiple equivalent linear constraints and eliminating check-polytope projection (CPP). Then, an iterative algorithm based on ADMM technique is developed to solve this free-of-CPP (FCPP) equivalence and each ADMM update can be computed efficiently. Moreover, the proposed ADMM-FCPP decoding algorithm is analyzed to display a linear complexity to the length of the LDPC code at each iteration. Finally, simulation results demonstrate the superiority of the proposed decoder in error-correcting performance compared with the state-of-the-art ADMM-based decoders.
{"title":"Improved Free-of-CPP ADMM-Based Iterative Decoding Algorithm of Binary LDPC Codes","authors":"Jing Bai;Zedong An;Yuhao Chi;Guanghui Song;Chau Yuen","doi":"10.1109/LSP.2024.3511336","DOIUrl":"https://doi.org/10.1109/LSP.2024.3511336","url":null,"abstract":"Iterative decoding algorithms based on the alternating direction method of multipliers (ADMM) decoding of low density parity check (LDPC) codes has emerged as an alternating decoding method and bringed a boom of research on drawing upon mathematical optimization to LDPC decoding. Improving error-correcting performance is a key issue to enhance the superiority of ADMM decoding. In this letter, we investigate an efficient ADMM-based iterative decoder for binary LDPC codes. First, we build an mathematical programming equivalence of the maximum likelihood (ML) decoding problem by transforming parity-check constraints to multiple equivalent linear constraints and eliminating check-polytope projection (CPP). Then, an iterative algorithm based on ADMM technique is developed to solve this free-of-CPP (FCPP) equivalence and each ADMM update can be computed efficiently. Moreover, the proposed ADMM-FCPP decoding algorithm is analyzed to display a linear complexity to the length of the LDPC code at each iteration. Finally, simulation results demonstrate the superiority of the proposed decoder in error-correcting performance compared with the state-of-the-art ADMM-based decoders.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"211-215"},"PeriodicalIF":3.2,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844463","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Conventional sparse uniform arrays (SUAs) is composed of multiple identical and rigorously collinear uniform linear arrays. By adjusting the baseline length between the subarrays, the array aperture can be arbitrarily large, thus substantially improving the accuracy of the direction-of-arrival (DOA) estimation. However, in practical applications, it is challenging to meet the strict collinearity requirement due to geographical constraints. In this letter, to address this problem, we propose the non-collinear sparse uniform array (NCSUA) model to mitigate the influence of the non-ideal terrain and enhance the practicality of the SUA. A novel estimation algorithm is then proposed to resolve the angle ambiguity in NCSUA and effectively achieve high-accuracy DOA estimation. Compared with the conventional SUA, numerical simulation results demonstrate the superiority of NCSUA employing the new de-ambiguity algorithm in DOA estimation performance and practical applications.
{"title":"High-Accuracy DOA Estimation for Non-Collinear Sparse Uniform Array","authors":"Hongyong Wang;Xiaolong Chen;Weibo Deng;Caisheng Zhang;Yonghua Xue","doi":"10.1109/LSP.2024.3510462","DOIUrl":"https://doi.org/10.1109/LSP.2024.3510462","url":null,"abstract":"Conventional sparse uniform arrays (SUAs) is composed of multiple identical and rigorously collinear uniform linear arrays. By adjusting the baseline length between the subarrays, the array aperture can be arbitrarily large, thus substantially improving the accuracy of the direction-of-arrival (DOA) estimation. However, in practical applications, it is challenging to meet the strict collinearity requirement due to geographical constraints. In this letter, to address this problem, we propose the non-collinear sparse uniform array (NCSUA) model to mitigate the influence of the non-ideal terrain and enhance the practicality of the SUA. A novel estimation algorithm is then proposed to resolve the angle ambiguity in NCSUA and effectively achieve high-accuracy DOA estimation. Compared with the conventional SUA, numerical simulation results demonstrate the superiority of NCSUA employing the new de-ambiguity algorithm in DOA estimation performance and practical applications.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"206-210"},"PeriodicalIF":3.2,"publicationDate":"2024-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-02DOI: 10.1109/LSP.2024.3510457
Songbin Li;Hui Du;Jingang Wang
In scenarios where steganographic texts from various steganographic domains generated by different generative steganography algorithms are mixed, most existing linguistic steganalysis methods lack corresponding network structures designed to account for the differences in steganographic texts from different domains, leading to the potential for further improvement in their general detection performance. To address the above issue, we propose a general generative linguistic steganalysis method based on the basic idea of dynamically extracting lexical association features of different steganographic domains at the segment level. We utilize dynamic-static text feature matrix to construct a word importance semantic encoding module to mine steganography-sensitive word features of different steganographic domains. Based on the obtained features, we propose a word correlation multi-scale perception module to focus on the segment-level lexical association changes caused by secret information embedding in different domains. Experimental results show that this method can improve the detection accuracy of existing mainstream linguistic steganalysis methods in various mixed steganography scenarios.
{"title":"General Steganalysis of Generative Linguistic Steganography Based on Dynamic Segment-Level Lexical Association Extraction","authors":"Songbin Li;Hui Du;Jingang Wang","doi":"10.1109/LSP.2024.3510457","DOIUrl":"https://doi.org/10.1109/LSP.2024.3510457","url":null,"abstract":"In scenarios where steganographic texts from various steganographic domains generated by different generative steganography algorithms are mixed, most existing linguistic steganalysis methods lack corresponding network structures designed to account for the differences in steganographic texts from different domains, leading to the potential for further improvement in their general detection performance. To address the above issue, we propose a general generative linguistic steganalysis method based on the basic idea of dynamically extracting lexical association features of different steganographic domains at the segment level. We utilize dynamic-static text feature matrix to construct a word importance semantic encoding module to mine steganography-sensitive word features of different steganographic domains. Based on the obtained features, we propose a word correlation multi-scale perception module to focus on the segment-level lexical association changes caused by secret information embedding in different domains. Experimental results show that this method can improve the detection accuracy of existing mainstream linguistic steganalysis methods in various mixed steganography scenarios.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"191-195"},"PeriodicalIF":3.2,"publicationDate":"2024-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-28DOI: 10.1109/LSP.2024.3508538
Preeti Meena;Himanshu Kumar;Sandeep Yadav
Scene inference refers to the identification of the scene from a given set of scene representations such as images. A saliency graph of a scene contains scene-defining objects along with semantic information between them in the graph representation. Existing methods consider the entire scene with uniform weighting to all semantic information for scene inference, resulting in a suboptimal performance. This letter presents an optimal edge weight estimation using the trust theoretic framework to encode semantic information effectively in saliency graphs. We have utilized the notion of converged global absolute trust in saliency scores of salient objects to compute the weighting of semantic information. Experimental results highlight the efficacy of the proposed method.
{"title":"Scene Inference Using Saliency Graphs With Trust-Theoretic Semantic Information Encoding","authors":"Preeti Meena;Himanshu Kumar;Sandeep Yadav","doi":"10.1109/LSP.2024.3508538","DOIUrl":"https://doi.org/10.1109/LSP.2024.3508538","url":null,"abstract":"Scene inference refers to the identification of the scene from a given set of scene representations such as images. A saliency graph of a scene contains scene-defining objects along with semantic information between them in the graph representation. Existing methods consider the entire scene with uniform weighting to all semantic information for scene inference, resulting in a suboptimal performance. This letter presents an optimal edge weight estimation using the trust theoretic framework to encode semantic information effectively in saliency graphs. We have utilized the notion of converged global absolute trust in saliency scores of salient objects to compute the weighting of semantic information. Experimental results highlight the efficacy of the proposed method.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"256-260"},"PeriodicalIF":3.2,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142875013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a sound event detection (SED) method that handles sound event boundaries in a statistically principled manner. A typical approach to SED is to train a deep neural network (DNN) in a supervised manner such that the model predicts frame-wise event activities. Since the predicted activities often contain fine insertion and deletion errors due to their temporal fluctuations, post-processing has been applied to obtain more accurate onset and offset boundaries. Existing post-processing methods are, however, non-differentiable and prohibit end-to-end (E2E) training. In this paper, we propose an E2E detection method based on a probabilistic formulation of sound event sequences called a hidden semi-Markov model (HSMM). The HSMM is utilized to transform frame-wise features predicted by a DNN into posterior probabilities of sound events represented by their class labels and temporal boundaries. We jointly train the DNN and HSMM in a supervised E2E manner by maximizing the event-wise posterior probabilities of the HSMM. This objective is a differentiable function thanks to the forward-backward algorithm of the HSMM. Experimental results with real recordings show that our method outperforms baseline systems with standard post-processing methods.
{"title":"Onset-and-Offset-Aware Sound Event Detection via Differentiable Frame-to-Event Mapping","authors":"Tomoya Yoshinaga;Keitaro Tanaka;Yoshiaki Bando;Keisuke Imoto;Shigeo Morishima","doi":"10.1109/LSP.2024.3509336","DOIUrl":"https://doi.org/10.1109/LSP.2024.3509336","url":null,"abstract":"This paper presents a sound event detection (SED) method that handles sound event boundaries in a statistically principled manner. A typical approach to SED is to train a deep neural network (DNN) in a supervised manner such that the model predicts frame-wise event activities. Since the predicted activities often contain fine insertion and deletion errors due to their temporal fluctuations, post-processing has been applied to obtain more accurate onset and offset boundaries. Existing post-processing methods are, however, non-differentiable and prohibit end-to-end (E2E) training. In this paper, we propose an E2E detection method based on a probabilistic formulation of sound event sequences called a hidden semi-Markov model (HSMM). The HSMM is utilized to transform frame-wise features predicted by a DNN into posterior probabilities of sound events represented by their class labels and temporal boundaries. We jointly train the DNN and HSMM in a supervised E2E manner by maximizing the event-wise posterior probabilities of the HSMM. This objective is a differentiable function thanks to the forward-backward algorithm of the HSMM. Experimental results with real recordings show that our method outperforms baseline systems with standard post-processing methods.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"186-190"},"PeriodicalIF":3.2,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10771642","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-28DOI: 10.1109/LSP.2024.3509337
Quanqing Xu;Xiang He;Muhao Xu;Kaixuan Hu;Weiye Song
High-resolution retinal optical coherence tomography(OCT) images are crucial for the diagnosis of numerous retinal diseases, but images acquired by narrow bandwidth OCT devices suffer from axial resolution degradation and are difficult to support disease diagnosis. Deep learning-based methods can enhance the axial resolution of OCT images, but most methods focus on improving the model architecture, the potential of fully exploiting the fusion of spatial and frequency domain information for image reconstruction has not been fully explored. This paper proposes a Dual-branch Multidomain Feature Fusion Network (MDFNet). The core module of the model consists of a parallel Enhanced Multi-scale Spatial Feature module and an Auxiliary Frequcy Feature module to provide non-interfering dual-domain feature information to improve the reconstruction effect. MDFNet achieved the best performance in the tests of mouse retina and human retina datasets, outperforming the state-of-the-art (SOTA) algorithms by 0.11 dB and 0.18 dB respectively. In addition, the results of this method performed best in the retinal layer segmentation test.
{"title":"A Dual-Branch Multidomain Feature Fusion Network for Axial Super-Resolution in Optical Coherence Tomography","authors":"Quanqing Xu;Xiang He;Muhao Xu;Kaixuan Hu;Weiye Song","doi":"10.1109/LSP.2024.3509337","DOIUrl":"https://doi.org/10.1109/LSP.2024.3509337","url":null,"abstract":"High-resolution retinal optical coherence tomography(OCT) images are crucial for the diagnosis of numerous retinal diseases, but images acquired by narrow bandwidth OCT devices suffer from axial resolution degradation and are difficult to support disease diagnosis. Deep learning-based methods can enhance the axial resolution of OCT images, but most methods focus on improving the model architecture, the potential of fully exploiting the fusion of spatial and frequency domain information for image reconstruction has not been fully explored. This paper proposes a Dual-branch Multidomain Feature Fusion Network (MDFNet). The core module of the model consists of a parallel Enhanced Multi-scale Spatial Feature module and an Auxiliary Frequcy Feature module to provide non-interfering dual-domain feature information to improve the reconstruction effect. MDFNet achieved the best performance in the tests of mouse retina and human retina datasets, outperforming the state-of-the-art (SOTA) algorithms by 0.11 dB and 0.18 dB respectively. In addition, the results of this method performed best in the retinal layer segmentation test.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"461-465"},"PeriodicalIF":3.2,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142938250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：