Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128906
Weiyu Zhao , Yizhuo Jiang , Yan Gao , Jie Li , Xinbo Gao
The purpose of multi-object tracking lies in the estimation of both the bounding boxes of targets and their identities. Nonetheless, occlusion brought by the object interactions often cause identity switches and trajectory loss. Inspired by the human vision of three-dimensional tracking properties, we propose a tracking framework based on depth estimation called DETrack to address this issue. This framework features a Depth Information Module (DIM) under monocular conditions, which can produce depth features as an association cue for multi-object tracking. In addition, to actively retrieves information lost in trajectories, we have also put forward a ”refind” component, which echoes how human vision compensates for objects out of sight. Our framework can seamlessly integrate with most trackers, and introduce introducing an entirely new data dimension to the tracking task. We have tested DETrack using the MOT17 and DanceTrack benchmark datasets and compared it with alternative methods. The test results demonstrate that our technique works effectively with current MOT trackers, and it significantly enhances tracking results based on HOTA, IDF1, and MOTA metrics on both datasets.
{"title":"DETrack: Depth information is predictable for tracking","authors":"Weiyu Zhao , Yizhuo Jiang , Yan Gao , Jie Li , Xinbo Gao","doi":"10.1016/j.neucom.2024.128906","DOIUrl":"10.1016/j.neucom.2024.128906","url":null,"abstract":"<div><div>The purpose of multi-object tracking lies in the estimation of both the bounding boxes of targets and their identities. Nonetheless, occlusion brought by the object interactions often cause identity switches and trajectory loss. Inspired by the human vision of three-dimensional tracking properties, we propose a tracking framework based on depth estimation called DETrack to address this issue. This framework features a Depth Information Module (DIM) under monocular conditions, which can produce depth features as an association cue for multi-object tracking. In addition, to actively retrieves information lost in trajectories, we have also put forward a ”refind” component, which echoes how human vision compensates for objects out of sight. Our framework can seamlessly integrate with most trackers, and introduce introducing an entirely new data dimension to the tracking task. We have tested DETrack using the MOT17 and DanceTrack benchmark datasets and compared it with alternative methods. The test results demonstrate that our technique works effectively with current MOT trackers, and it significantly enhances tracking results based on HOTA, IDF1, and MOTA metrics on both datasets.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128906"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128881
Kaiqi Dong , Peijun Hu , Yu Tian , Yan Zhu , Xiang Li , Tianshu Zhou , Xueli Bai , Tingbo Liang , Jingsong Li
Accurate pancreatic tumor segmentation in CT images is crucial but challenging due to the complex anatomy and varied tumor appearance. Previous methods predominantly adopt two-stage segmentation approaches to identify and localize tumors and rely heavily on CNN-extracted texture features. In this study, we propose a tumor position-aware branch to learn pancreatic anatomical priors and integrate them into a standard 3D U-Net segmentation network. The tumor position-aware branch consists of three innovative components. Firstly, the proposed method utilizes discrete information bottleneck theory to extract compact and informative segmentation features with pancreatic anatomical priors. Secondly, we propose a coordinate position encoding transformer that encodes the spatial coordinates of each patch within the CT volume. This encoding provides the model with a global positional context, allowing it to effectively model the spatial relationships between anatomical structures. Thirdly, a probability margin regularization loss is proposed to further eliminate the interference of background patches on the learning of pancreatic anatomical positions. Our model is trained and validated our model on the public Medical Segmentation Decathlon (MSD) dataset and a private clinical dataset. Experimental results demonstrate that our approach achieves competitive performance compared to state-of-the-art (SOTA) methods in both pancreas and tumor segmentation, with Dice scores of 82.11% for the pancreas and 55.56% for the tumor on the MSD dataset. The proposed framework offers an effective solution to leverage anatomical priors and enhance representation learning for improved pancreatic tumor segmentation.
{"title":"Position-aware representation learning with anatomical priors for enhanced pancreas tumor segmentation","authors":"Kaiqi Dong , Peijun Hu , Yu Tian , Yan Zhu , Xiang Li , Tianshu Zhou , Xueli Bai , Tingbo Liang , Jingsong Li","doi":"10.1016/j.neucom.2024.128881","DOIUrl":"10.1016/j.neucom.2024.128881","url":null,"abstract":"<div><div>Accurate pancreatic tumor segmentation in CT images is crucial but challenging due to the complex anatomy and varied tumor appearance. Previous methods predominantly adopt two-stage segmentation approaches to identify and localize tumors and rely heavily on CNN-extracted texture features. In this study, we propose a tumor position-aware branch to learn pancreatic anatomical priors and integrate them into a standard 3D U-Net segmentation network. The tumor position-aware branch consists of three innovative components. Firstly, the proposed method utilizes discrete information bottleneck theory to extract compact and informative segmentation features with pancreatic anatomical priors. Secondly, we propose a coordinate position encoding transformer that encodes the spatial coordinates of each patch within the CT volume. This encoding provides the model with a global positional context, allowing it to effectively model the spatial relationships between anatomical structures. Thirdly, a probability margin regularization loss is proposed to further eliminate the interference of background patches on the learning of pancreatic anatomical positions. Our model is trained and validated our model on the public Medical Segmentation Decathlon (MSD) dataset and a private clinical dataset. Experimental results demonstrate that our approach achieves competitive performance compared to state-of-the-art (SOTA) methods in both pancreas and tumor segmentation, with Dice scores of 82.11% for the pancreas and 55.56% for the tumor on the MSD dataset. The proposed framework offers an effective solution to leverage anatomical priors and enhance representation learning for improved pancreatic tumor segmentation.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128881"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Referring image segmentation involves accurately segmenting objects based on natural language descriptions. This poses challenges due to the intricate and varied nature of language expressions, as well as the requirement to identify relevant image regions among multiple objects. Current models predominantly employ language-aware early fusion techniques, which may lead to misinterpretations of language expressions due to the lack of explicit visual guidance of the language encoder. Additionally, early fusion methods are unable to adequately leverage high-level contexts. To address these limitations, this paper introduces the Prompt-guided Bidirectional Deep Fusion Network (PBDF-Net) to enhance the fusion of language and vision modalities. In contrast to traditional unidirectional early fusion approaches, our approach employs a prompt-guided bidirectional encoder fusion (PBEF) module to promote mutual cross-modal fusion across multiple stages of the vision and language encoders. Furthermore, PBDF-Net incorporates a prompt-guided cross-modal interaction (PCI) module during the late fusion stage, facilitating a more profound integration of contextual information from both modalities, resulting in more accurate target segmentation. Comprehensive experiments conducted on the RefCOCO, RefCOCO+, G-Ref and ReferIt datasets substantiate the efficacy of our proposed method, demonstrating significant advancements in performance compared to existing approaches.
{"title":"Prompt-guided bidirectional deep fusion network for referring image segmentation","authors":"Junxian Wu , Yujia Zhang , Michael Kampffmeyer , Xiaoguang Zhao","doi":"10.1016/j.neucom.2024.128899","DOIUrl":"10.1016/j.neucom.2024.128899","url":null,"abstract":"<div><div>Referring image segmentation involves accurately segmenting objects based on natural language descriptions. This poses challenges due to the intricate and varied nature of language expressions, as well as the requirement to identify relevant image regions among multiple objects. Current models predominantly employ language-aware early fusion techniques, which may lead to misinterpretations of language expressions due to the lack of explicit visual guidance of the language encoder. Additionally, early fusion methods are unable to adequately leverage high-level contexts. To address these limitations, this paper introduces the Prompt-guided Bidirectional Deep Fusion Network (PBDF-Net) to enhance the fusion of language and vision modalities. In contrast to traditional unidirectional early fusion approaches, our approach employs a prompt-guided bidirectional encoder fusion (PBEF) module to promote mutual cross-modal fusion across multiple stages of the vision and language encoders. Furthermore, PBDF-Net incorporates a prompt-guided cross-modal interaction (PCI) module during the late fusion stage, facilitating a more profound integration of contextual information from both modalities, resulting in more accurate target segmentation. Comprehensive experiments conducted on the RefCOCO, RefCOCO+, G-Ref and ReferIt datasets substantiate the efficacy of our proposed method, demonstrating significant advancements in performance compared to existing approaches.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128899"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128873
Yongwei Zhang , Bo Zhao , Derong Liu , Marios M. Polycarpou , Shiguo Peng , Shunchao Zhang
This paper investigates the event-triggered robust hierarchical control (ETRHC) problem of uncertain multi-player nonlinear systems subject to actuator faults by using adaptive dynamic programming and integral sliding mode technique. Different from existing results where the control policies of all players are updated simultaneously, a hierarchical decision-making problem is considered as a Stackelberg game. The Stackelberg game consists of a single leader and multiple followers, the leader acts a control policy in advance by considering the responses of all the followers, and each follower responds optimally to the leader’s policy. The proposed control structure comprises of two components, namely integral sliding mode control and ETRHC. In the first step, the integral sliding mode control policy is developed to cope with actuator faults and matched uncertainties, and then, the fault-free multi-player nonlinear systems with mismatched uncertainties is obtained. In the second step, by designing an appropriate performance index function for each player, the ETRHC of the fault-free multi-player nonlinear system with mismatched uncertainties is converted to an event-triggered approximate optimal control of its nominal form, and the hierarchical decision-making problem is addressed. Subsequently, the ETRHC laws are derived by solving event-triggered Hamilton–Jacobi equations with the critic-only learning. Theoretical analysis demonstrates that the integral sliding mode-based ETRHC scheme guarantees the multi-player uncertain nonlinear systems with actuator faults to be asymptotically stable. Finally, the quadrotor attitude system is adopted to verify the effectiveness of the present scheme.
{"title":"Event-triggered robust hierarchical control for uncertain multiplayer Stackelberg games via adaptive dynamic programming","authors":"Yongwei Zhang , Bo Zhao , Derong Liu , Marios M. Polycarpou , Shiguo Peng , Shunchao Zhang","doi":"10.1016/j.neucom.2024.128873","DOIUrl":"10.1016/j.neucom.2024.128873","url":null,"abstract":"<div><div>This paper investigates the event-triggered robust hierarchical control (ETRHC) problem of uncertain multi-player nonlinear systems subject to actuator faults by using adaptive dynamic programming and integral sliding mode technique. Different from existing results where the control policies of all players are updated simultaneously, a hierarchical decision-making problem is considered as a Stackelberg game. The Stackelberg game consists of a single leader and multiple followers, the leader acts a control policy in advance by considering the responses of all the followers, and each follower responds optimally to the leader’s policy. The proposed control structure comprises of two components, namely integral sliding mode control and ETRHC. In the first step, the integral sliding mode control policy is developed to cope with actuator faults and matched uncertainties, and then, the fault-free multi-player nonlinear systems with mismatched uncertainties is obtained. In the second step, by designing an appropriate performance index function for each player, the ETRHC of the fault-free multi-player nonlinear system with mismatched uncertainties is converted to an event-triggered approximate optimal control of its nominal form, and the hierarchical decision-making problem is addressed. Subsequently, the ETRHC laws are derived by solving event-triggered Hamilton–Jacobi equations with the critic-only learning. Theoretical analysis demonstrates that the integral sliding mode-based ETRHC scheme guarantees the multi-player uncertain nonlinear systems with actuator faults to be asymptotically stable. Finally, the quadrotor attitude system is adopted to verify the effectiveness of the present scheme.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128873"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128902
Fumin Li , Mengjie Huang , Wenlong You , Longsheng Zhu , Hanjing Cheng , Rui Yang
In recent years, electroencephalogram (EEG) based on the brain–computer interface (BCI) systems have become increasingly advanced, with researcher using deep neural networks as tools to enhance performance. BCI systems heavily rely on EEG signals for effective human–computer interactions, and deep neural networks show excellent performance in processing and classifying these signals. Nevertheless, the vulnerability to backdoor attack is still a major problem. Backdoor attack is the injection of specially designed triggers into the model training process, which can lead to significant security issues. Therefore, in order to simulate the negative impact of backdoor attack and bridge the research gap in the field of BCI, this paper proposes a new backdoor attack method to call researcher attention to the security issues of BCI. In this paper, Spatialspectral-Backdoor is proposed to effectively attack the BCI system. The method is carefully designed to target the spectral active backdoor attack of the BCI system and includes a multi-channel preference method to select the electrode channels sensitive to the target task. Ultimately, the effectiveness of the comparison and ablation experiments is validated on the publicly available BCI competition datasets. The results show that the average attack success rate and clean sample accuracy of Spatialspectral-Backdoor in the BCI scenario are 97.12% and 85.16%, respectively, compared with other backdoor attack methods. Furthermore, by observing the infection ratio of backdoor triggers and visualization of the feature space, the proposed Spatialspectral-Backdoor outperforms other backdoor attack methods.
{"title":"Spatialspectral-Backdoor: Realizing backdoor attack for deep neural networks in brain–computer interface via EEG characteristics","authors":"Fumin Li , Mengjie Huang , Wenlong You , Longsheng Zhu , Hanjing Cheng , Rui Yang","doi":"10.1016/j.neucom.2024.128902","DOIUrl":"10.1016/j.neucom.2024.128902","url":null,"abstract":"<div><div>In recent years, electroencephalogram (EEG) based on the brain–computer interface (BCI) systems have become increasingly advanced, with researcher using deep neural networks as tools to enhance performance. BCI systems heavily rely on EEG signals for effective human–computer interactions, and deep neural networks show excellent performance in processing and classifying these signals. Nevertheless, the vulnerability to backdoor attack is still a major problem. Backdoor attack is the injection of specially designed triggers into the model training process, which can lead to significant security issues. Therefore, in order to simulate the negative impact of backdoor attack and bridge the research gap in the field of BCI, this paper proposes a new backdoor attack method to call researcher attention to the security issues of BCI. In this paper, Spatialspectral-Backdoor is proposed to effectively attack the BCI system. The method is carefully designed to target the spectral active backdoor attack of the BCI system and includes a multi-channel preference method to select the electrode channels sensitive to the target task. Ultimately, the effectiveness of the comparison and ablation experiments is validated on the publicly available BCI competition datasets. The results show that the average attack success rate and clean sample accuracy of Spatialspectral-Backdoor in the BCI scenario are 97.12% and 85.16%, respectively, compared with other backdoor attack methods. Furthermore, by observing the infection ratio of backdoor triggers and visualization of the feature space, the proposed Spatialspectral-Backdoor outperforms other backdoor attack methods.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128902"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743517","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128901
Zexuan Ji , Hao Qian , Xiao Ma
In the field of medical imaging, the automatic detection and segmentation of colon polyps is crucial for the early diagnosis of colorectal cancer. Currently, Transformer methods are commonly employed for colon polyp segmentation tasks, often utilizing dual attention mechanisms. However, these attention mechanisms typically utilize channel attention and spatial attention in a serial or parallel manner, which increases computational costs and model complexity. To address these issues, we propose a Shared Hybrid Attention Transformer (SHAT) framework, which shares queries and keys, thereby avoiding redundant computations and reducing computational complexity. Additionally, we introduce differential subtraction attention module to enhance feature fusion capability and significantly improve the delineation of polyp boundaries, effectively capture complex image details and edge information involved in the colon polyp images comparing with existing techniques. Our approach overcomes the limitations of existing colon polyp segmentation techniques. Experimental results on a large-scale annotated colon polyp image dataset demonstrate that our method excels in localizing and segmenting polyps of various sizes, shapes, and textures with high robustness. The source code for the SHAT framework is available at https://github.com/peanutHao/SHAT.
{"title":"Shared Hybrid Attention Transformer network for colon polyp segmentation","authors":"Zexuan Ji , Hao Qian , Xiao Ma","doi":"10.1016/j.neucom.2024.128901","DOIUrl":"10.1016/j.neucom.2024.128901","url":null,"abstract":"<div><div>In the field of medical imaging, the automatic detection and segmentation of colon polyps is crucial for the early diagnosis of colorectal cancer. Currently, Transformer methods are commonly employed for colon polyp segmentation tasks, often utilizing dual attention mechanisms. However, these attention mechanisms typically utilize channel attention and spatial attention in a serial or parallel manner, which increases computational costs and model complexity. To address these issues, we propose a Shared Hybrid Attention Transformer (SHAT) framework, which shares queries and keys, thereby avoiding redundant computations and reducing computational complexity. Additionally, we introduce differential subtraction attention module to enhance feature fusion capability and significantly improve the delineation of polyp boundaries, effectively capture complex image details and edge information involved in the colon polyp images comparing with existing techniques. Our approach overcomes the limitations of existing colon polyp segmentation techniques. Experimental results on a large-scale annotated colon polyp image dataset demonstrate that our method excels in localizing and segmenting polyps of various sizes, shapes, and textures with high robustness. The source code for the SHAT framework is available at <span><span>https://github.com/peanutHao/SHAT</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128901"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743050","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128892
Zhengzhong Zhu , Chujun Pu , Xuejie Zhang , Jin Wang , Xiaobing Zhou
Incomplete multi-view clustering (IMVC) is a critical task in real-world applications, where missing data in some views can severely limit the ability to leverage complementary information across views. This issue leads to incomplete sample representations, hindering model performance. Current contrastive learning methods for IMVC exacerbate the problem by directly constructing data pairs from incomplete samples, ignoring essential information and resulting in class collisions, where samples from different classes are incorrectly grouped together due to a lack of label guidance. These challenges are particularly detrimental in fields like recommendation systems and bioinformatics, where accurate clustering of high-dimensional and incomplete data is essential for decision-making. To address these issues, we propose Dual-dimensional Contrastive Learning (DCL), an online IMVC model that fills missing values through multi-view consistency transfer, enabling simultaneous clustering and representation learning via instance-level and cluster-level contrastive learning in both row and column spaces. DCL mitigates class collision issues by generating high-confidence pseudo-labels and using an optimal transport matrix, significantly improving clustering accuracy. Extensive experiments demonstrate that DCL achieves state-of-the-art results across five datasets. The code is available at https://github.com/2251821381/DCL.
{"title":"Dual-dimensional contrastive learning for incomplete multi-view clustering","authors":"Zhengzhong Zhu , Chujun Pu , Xuejie Zhang , Jin Wang , Xiaobing Zhou","doi":"10.1016/j.neucom.2024.128892","DOIUrl":"10.1016/j.neucom.2024.128892","url":null,"abstract":"<div><div>Incomplete multi-view clustering (IMVC) is a critical task in real-world applications, where missing data in some views can severely limit the ability to leverage complementary information across views. This issue leads to incomplete sample representations, hindering model performance. Current contrastive learning methods for IMVC exacerbate the problem by directly constructing data pairs from incomplete samples, ignoring essential information and resulting in class collisions, where samples from different classes are incorrectly grouped together due to a lack of label guidance. These challenges are particularly detrimental in fields like recommendation systems and bioinformatics, where accurate clustering of high-dimensional and incomplete data is essential for decision-making. To address these issues, we propose Dual-dimensional Contrastive Learning (DCL), an online IMVC model that fills missing values through multi-view consistency transfer, enabling simultaneous clustering and representation learning via instance-level and cluster-level contrastive learning in both row and column spaces. DCL mitigates class collision issues by generating high-confidence pseudo-labels and using an optimal transport matrix, significantly improving clustering accuracy. Extensive experiments demonstrate that DCL achieves state-of-the-art results across five datasets. The code is available at <span><span>https://github.com/2251821381/DCL</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128892"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142703005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128919
Xiujun Shu , Hanjun Li , Wei Wen , Ruizhi Qiao , Nannan Li , Weijian Ruan , Hanjing Su , Bo Wang , Shouzhi Chen , Jun Zhou
Occluded person re-IDentification (re-ID) is a challenging task in surveillance scenarios that remains unresolved. To address it, existing methods primarily rely on auxiliary models, e.g. pose estimation, to explore visible parts by detecting human keypoints. However, these approaches inevitably encounter two issues: domain gap and information asymmetry. The former arises from pre-training auxiliary models on different domains, while the latter indicates that the occluded query has asymmetric valid cues compared to the holistic visible gallery. In this paper, we propose a novel Precise Occlusion-aware and Feature-level Reconstruction (POFR) network for occluded re-ID. POFR addresses the occlusion issue from two viewpoints: perceiving the occlusions other than visible human bodies and reconstructing the occluded parts at the feature level. The first perspective is achieved through occlusion-driven contrastive learning (OCL). OCL incorporates an occlusion generator capable of generating object and person-specific occlusions. Unlike previous coarse occlusions, our generator leverages segmented pedestrians and obstacles to generate realistic occlusions which are then used for contrastive learning. The second perspective is implemented through an occlusion-guided feature reconstruction (OFR) module. OFR initially learns an occlusion predictor to estimate the occlusion mask, which is subsequently utilized to recover features corresponding to the occluded regions. Benefiting from the occlusion generator, the occlusion predictor can be effectively supervised with the precise occlusion masks, thereby mitigating the domain gap problem. Additionally, the recovered features alleviate information asymmetry when matching an occluded query and a holistic gallery. Extensive experiments conducted on occluded, partial, and holistic datasets demonstrate the superior performance of our POFR over state-of-the-art methods. The source code will be made publicly available upon paper acceptance.
{"title":"Precise occlusion-aware and feature-level reconstruction for occluded person re-identification","authors":"Xiujun Shu , Hanjun Li , Wei Wen , Ruizhi Qiao , Nannan Li , Weijian Ruan , Hanjing Su , Bo Wang , Shouzhi Chen , Jun Zhou","doi":"10.1016/j.neucom.2024.128919","DOIUrl":"10.1016/j.neucom.2024.128919","url":null,"abstract":"<div><div>Occluded person re-IDentification (re-ID) is a challenging task in surveillance scenarios that remains unresolved. To address it, existing methods primarily rely on auxiliary models, <em>e.g.</em> pose estimation, to explore visible parts by detecting human keypoints. However, these approaches inevitably encounter two issues: domain gap and information asymmetry. The former arises from pre-training auxiliary models on different domains, while the latter indicates that the occluded query has asymmetric valid cues compared to the holistic visible gallery. In this paper, we propose a novel <em>Precise Occlusion-aware and Feature-level Reconstruction</em> (POFR) network for occluded re-ID. POFR addresses the occlusion issue from two viewpoints: perceiving the occlusions other than visible human bodies and reconstructing the occluded parts at the feature level. The first perspective is achieved through occlusion-driven contrastive learning (OCL). OCL incorporates an occlusion generator capable of generating object and person-specific occlusions. Unlike previous coarse occlusions, our generator leverages segmented pedestrians and obstacles to generate realistic occlusions which are then used for contrastive learning. The second perspective is implemented through an occlusion-guided feature reconstruction (OFR) module. OFR initially learns an occlusion predictor to estimate the occlusion mask, which is subsequently utilized to recover features corresponding to the occluded regions. Benefiting from the occlusion generator, the occlusion predictor can be effectively supervised with the precise occlusion masks, thereby mitigating the domain gap problem. Additionally, the recovered features alleviate information asymmetry when matching an occluded query and a holistic gallery. Extensive experiments conducted on occluded, partial, and holistic datasets demonstrate the superior performance of our POFR over state-of-the-art methods. The source code will be made publicly available upon paper acceptance.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128919"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128875
Jorge C-Rella , Gerda Claeskens , Ricardo Cao , Juan M. Vilar
Instance-dependent cost-sensitive learning addresses classification problems where each observation has a different misclassification cost. In this paper, we propose cost-sensitive parametric models to minimize the expectation of losses. A loss function incorporating the misclassification costs is defined, which serves as the objective function for obtaining cost-sensitive parameter estimators. The consistency and asymptotic normality of these estimators are established under general conditions, theoretically demonstrating their good performance. Additionally, we derive bounds for the bias introduced when regularizing the optimization problem, which is generally necessary in practice. To conclude, the effectiveness of the proposed estimators is evaluated through an extensive novel simulation study and the analysis of five real data sets, further demonstrating their proficiency in practical settings.
{"title":"Instance-dependent cost-sensitive parametric learning","authors":"Jorge C-Rella , Gerda Claeskens , Ricardo Cao , Juan M. Vilar","doi":"10.1016/j.neucom.2024.128875","DOIUrl":"10.1016/j.neucom.2024.128875","url":null,"abstract":"<div><div>Instance-dependent cost-sensitive learning addresses classification problems where each observation has a different misclassification cost. In this paper, we propose cost-sensitive parametric models to minimize the expectation of losses. A loss function incorporating the misclassification costs is defined, which serves as the objective function for obtaining cost-sensitive parameter estimators. The consistency and asymptotic normality of these estimators are established under general conditions, theoretically demonstrating their good performance. Additionally, we derive bounds for the bias introduced when regularizing the optimization problem, which is generally necessary in practice. To conclude, the effectiveness of the proposed estimators is evaluated through an extensive novel simulation study and the analysis of five real data sets, further demonstrating their proficiency in practical settings.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128875"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142703003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-16DOI: 10.1016/j.neucom.2024.128893
Bingxuan Chen, Hao Zhang
Multistate associative memory models have shown a remarkable ability to remember non-binary data in recent years, including the complex-valued Hopfield neural networks (CHNNs) and their advanced counterpart of rotor Hopfield neural networks (RHNNs). However, the noise robustness of these models deteriorates significantly as the number of stored patterns and the resolution increase. To address this issue, inspired by the complex connections observed in biological neural systems, high-order connections are incorporated into CHNNs and RHNNs, resulting in the high-order complex-valued Hopfield neural networks (HCHNNs) and the high-order rotor Hopfield neural networks (HRHNNs). By abstracting virtual neurons, high-order connection-based update equations and projection rules are simultaneously modified as complex versions. The maximum storage capacity of the network is increased from to nearly , where and represent the number of neurons and the number of high-order connections. The associative memory capabilities of HRHNNs were validated on the CIFAR-10, MNIST, and CelebA datasets, demonstrating superior robustness to noise compared to RHNNs as the number of memory patterns increased.
{"title":"High-order rotor Hopfield neural networks for associative memory","authors":"Bingxuan Chen, Hao Zhang","doi":"10.1016/j.neucom.2024.128893","DOIUrl":"10.1016/j.neucom.2024.128893","url":null,"abstract":"<div><div>Multistate associative memory models have shown a remarkable ability to remember non-binary data in recent years, including the complex-valued Hopfield neural networks (CHNNs) and their advanced counterpart of rotor Hopfield neural networks (RHNNs). However, the noise robustness of these models deteriorates significantly as the number of stored patterns and the resolution increase. To address this issue, inspired by the complex connections observed in biological neural systems, high-order connections are incorporated into CHNNs and RHNNs, resulting in the high-order complex-valued Hopfield neural networks (HCHNNs) and the high-order rotor Hopfield neural networks (HRHNNs). By abstracting virtual neurons, high-order connection-based update equations and projection rules are simultaneously modified as complex versions. The maximum storage capacity of the network is increased from <span><math><mi>N</mi></math></span> to nearly <span><math><mrow><mo>(</mo><mi>N</mi><mo>+</mo><mi>M</mi><mo>)</mo></mrow></math></span>, where <span><math><mi>N</mi></math></span> and <span><math><mi>M</mi></math></span> represent the number of neurons and the number of high-order connections. The associative memory capabilities of HRHNNs were validated on the CIFAR-10, MNIST, and CelebA datasets, demonstrating superior robustness to noise compared to RHNNs as the number of memory patterns increased.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128893"},"PeriodicalIF":5.5,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号