The emergence of advanced Deepfake technologies has gradually raised concerns in society, prompting significant attention to Deepfake detection. However, in real-world scenarios, Deepfakes often involve multiple faces. Despite this, most existing detection methods still detect these faces individually, overlooking the informative correlation between them and the relationship between the global information of the image and the local information of the faces. In this paper, we address this limitation by proposing FILTER, a novel framework for multi-face forgery detection that explicitly captures underlying correlations. FILTER consists of two main modules: Multi-face Relationship Learning (MRL) and Global Feature Aggregation (GFA). Specifically, MRL learns the correlation of local facial features in multi-face images, and GFA constructs the relationship between image-level labels and individual facial features to enhance performance from a global perspective. In particular, a contrastive learning loss function is used to better discriminate between real and fake faces. Extensive experiments on two publicly available multi-face forgery datasets demonstrate the state-of-the-art performance of FILTER in multi-face forgery detection. For example, on Openforensics Test-Challenge dataset, FILTER outperforms the previous state-of-the-art methods with a higher AUC score (0.980) and higher detection accuracy (92.04%).
{"title":"Exploiting Facial Relationships and Feature Aggregation for Multi-Face Forgery Detection","authors":"Chenhao Lin;Fangbin Yi;Hang Wang;Jingyi Deng;Zhengyu Zhao;Qian Li;Chao Shen","doi":"10.1109/TIFS.2024.3461469","DOIUrl":"10.1109/TIFS.2024.3461469","url":null,"abstract":"The emergence of advanced Deepfake technologies has gradually raised concerns in society, prompting significant attention to Deepfake detection. However, in real-world scenarios, Deepfakes often involve multiple faces. Despite this, most existing detection methods still detect these faces individually, overlooking the informative correlation between them and the relationship between the global information of the image and the local information of the faces. In this paper, we address this limitation by proposing FILTER, a novel framework for multi-face forgery detection that explicitly captures underlying correlations. FILTER consists of two main modules: Multi-face Relationship Learning (MRL) and Global Feature Aggregation (GFA). Specifically, MRL learns the correlation of local facial features in multi-face images, and GFA constructs the relationship between image-level labels and individual facial features to enhance performance from a global perspective. In particular, a contrastive learning loss function is used to better discriminate between real and fake faces. Extensive experiments on two publicly available multi-face forgery datasets demonstrate the state-of-the-art performance of FILTER in multi-face forgery detection. For example, on Openforensics Test-Challenge dataset, FILTER outperforms the previous state-of-the-art methods with a higher AUC score (0.980) and higher detection accuracy (92.04%).","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"8832-8844"},"PeriodicalIF":6.3,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142313854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Single-shot face anti-spoofing (FAS) is a key technique for securing face recognition systems, relying solely on static images as input. However, single-shot FAS remains a challenging and under-explored problem due to two reasons: 1) On the data side, learning FAS from RGB images is largely context-dependent, and single-shot images without additional annotations contain limited semantic information. 2) On the model side, existing single-shot FAS models struggle to provide proper evidence for their decisions, and FAS methods based on depth estimation require expensive per-pixel annotations. To address these issues, we construct and release a large binocular NIR image dataset named BNI-FAS, which contains more than 300,000 real face and plane attack images, and propose an Interpretable FAS Transformer (IFAST) that requires only weak supervision to produce interpretable predictions. Our IFAST generates pixel-wise disparity maps using the proposed disparity estimation Transformer with Dynamic Matching Attention (DMA) blocks. Besides, we design a confidence map generator to work in tandem with a dual-teacher distillation module to obtain the final discriminant results. Comprehensive experiments show that our IFAST achieves state-of-the-art performance on BNI-FAS, verifying its effectiveness of single-shot FAS on binocular NIR images. The project page is available at https://ifast-bni.github.io/ .
{"title":"IFAST: Weakly Supervised Interpretable Face Anti-Spoofing From Single-Shot Binocular NIR Images","authors":"Jiancheng Huang;Donghao Zhou;Jianzhuang Liu;Linxiao Shi;Shifeng Chen","doi":"10.1109/TIFS.2024.3465930","DOIUrl":"10.1109/TIFS.2024.3465930","url":null,"abstract":"Single-shot face anti-spoofing (FAS) is a key technique for securing face recognition systems, relying solely on static images as input. However, single-shot FAS remains a challenging and under-explored problem due to two reasons: 1) On the data side, learning FAS from RGB images is largely context-dependent, and single-shot images without additional annotations contain limited semantic information. 2) On the model side, existing single-shot FAS models struggle to provide proper evidence for their decisions, and FAS methods based on depth estimation require expensive per-pixel annotations. To address these issues, we construct and release a large binocular NIR image dataset named BNI-FAS, which contains more than 300,000 real face and plane attack images, and propose an Interpretable FAS Transformer (IFAST) that requires only weak supervision to produce interpretable predictions. Our IFAST generates pixel-wise disparity maps using the proposed disparity estimation Transformer with Dynamic Matching Attention (DMA) blocks. Besides, we design a confidence map generator to work in tandem with a dual-teacher distillation module to obtain the final discriminant results. Comprehensive experiments show that our IFAST achieves state-of-the-art performance on BNI-FAS, verifying its effectiveness of single-shot FAS on binocular NIR images. The project page is available at \u0000<uri>https://ifast-bni.github.io/</uri>\u0000.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"9270-9284"},"PeriodicalIF":6.3,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142313836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TIFS.2024.3465847
Mingyue Xie;Zheng Chang;Hongwei Li;Geyong Min
Unmanned aerial vehicles (UAVs) have emerged as pivotal roles within internet of vehicles (IoV), serving as mobile base stations. However, while expanding coverage and improving mobility, the deployment of UAVs also poses a threat to the integrity and privacy of sensitive data due to open wireless communication channels in IoV. Therefore, preventing unauthorized access and data tampering is critically important between UAVs and vehicles. For the authenticity and legitimacy of the UAV certificate, existing authentication approaches may lead to significant challenges in key management overhead or dependence on a trusted third party. In this paper, a blockchain-based authentication scheme for UAV-assisted IoV system (BASUV) is proposed. This solution enables dependable UAV registration and authentication services, and permits the dynamic addition and removal. Specifically, blockchain is introduced to achieve the decentralized management and distributed trust of the UAV certificate ledger. Furthermore, to prevent information tampering and identity deception, we design CMPES, a novel combined scheme based on multiple public key generators (PKGs) for encryption and signature. Identical key pair in encryption and signature can reduce key generation and management overhead. The security and experimental analysis demonstrates the effectiveness and efficiency of the proposed scheme.
{"title":"BASUV: A Blockchain-Enabled UAV Authentication Scheme for Internet of Vehicles","authors":"Mingyue Xie;Zheng Chang;Hongwei Li;Geyong Min","doi":"10.1109/TIFS.2024.3465847","DOIUrl":"10.1109/TIFS.2024.3465847","url":null,"abstract":"Unmanned aerial vehicles (UAVs) have emerged as pivotal roles within internet of vehicles (IoV), serving as mobile base stations. However, while expanding coverage and improving mobility, the deployment of UAVs also poses a threat to the integrity and privacy of sensitive data due to open wireless communication channels in IoV. Therefore, preventing unauthorized access and data tampering is critically important between UAVs and vehicles. For the authenticity and legitimacy of the UAV certificate, existing authentication approaches may lead to significant challenges in key management overhead or dependence on a trusted third party. In this paper, a blockchain-based authentication scheme for UAV-assisted IoV system (BASUV) is proposed. This solution enables dependable UAV registration and authentication services, and permits the dynamic addition and removal. Specifically, blockchain is introduced to achieve the decentralized management and distributed trust of the UAV certificate ledger. Furthermore, to prevent information tampering and identity deception, we design CMPES, a novel combined scheme based on multiple public key generators (PKGs) for encryption and signature. Identical key pair in encryption and signature can reduce key generation and management overhead. The security and experimental analysis demonstrates the effectiveness and efficiency of the proposed scheme.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"9055-9069"},"PeriodicalIF":6.3,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142313834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The widespread collection and analysis of private speech signals have become increasingly prevalent, raising significant privacy concerns. To protect speech signals from unauthorized analysis, adversarial attack methods for deceiving speaker recognition models have been proposed. While a few of these methods are specifically designed for real-time protection of speech signals, they introduce significant delays that can severely impact speech communication when applied to streaming speech data. In this paper, we present a novel approach that aims to offer real-time protection for speech signals without delays. By utilizing observed data only, we generate initial adversarial seed perturbations and refine them to obtain the necessary adversarial perturbations predicted for adjacent unobserved signals. This refinement process is conducted via a proposed model called PAPG. On the basis of perturbation prediction, we develop a streaming audio processing framework that generates perturbations in synchronization with the playback of the original signal, effectively eliminating delays. The experimental results demonstrate that under the proposed attack, the average Top-1 accuracy of various advanced speaker recognition methods is reduced by 89%, and the average equal error rate (EER) increases to 36%. Remarkably, these results are achieved without delays while maintaining superior perceptual quality.
{"title":"Adversarial Perturbation Prediction for Real-Time Protection of Speech Privacy","authors":"Zhaoyang Zhang;Shen Wang;Guopu Zhu;Dechen Zhan;Jiwu Huang","doi":"10.1109/TIFS.2024.3463538","DOIUrl":"10.1109/TIFS.2024.3463538","url":null,"abstract":"The widespread collection and analysis of private speech signals have become increasingly prevalent, raising significant privacy concerns. To protect speech signals from unauthorized analysis, adversarial attack methods for deceiving speaker recognition models have been proposed. While a few of these methods are specifically designed for real-time protection of speech signals, they introduce significant delays that can severely impact speech communication when applied to streaming speech data. In this paper, we present a novel approach that aims to offer real-time protection for speech signals without delays. By utilizing observed data only, we generate initial adversarial seed perturbations and refine them to obtain the necessary adversarial perturbations predicted for adjacent unobserved signals. This refinement process is conducted via a proposed model called PAPG. On the basis of perturbation prediction, we develop a streaming audio processing framework that generates perturbations in synchronization with the playback of the original signal, effectively eliminating delays. The experimental results demonstrate that under the proposed attack, the average Top-1 accuracy of various advanced speaker recognition methods is reduced by 89%, and the average equal error rate (EER) increases to 36%. Remarkably, these results are achieved without delays while maintaining superior perceptual quality.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"8701-8716"},"PeriodicalIF":6.3,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142313835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TIFS.2024.3465928
Yanrong Liang;Jianfeng Ma;Yinbin Miao;Yuan Su;Robert H. Deng
Privacy-preserving range query, which allows the server to implement secure and efficient range query on encrypted data, has been widely studied in recent years. Existing privacy-preserving range query schemes can realize effective range query, but usually suffer from the low efficiency and security. In order to solve the above issues, we propose an Efficient and Privacy-preserving encode-based Range Query over encrypted cloud data (namely basic EPRQ), which encodes the data and range by using Range Encode (REncoder), and then encrypts the codes via Additional Symmetric-Key Hidden Vector Encryption (ASHVE) technology. The basic EPRQ can achieve effective range query while ensuring privacy protection. Then, we split the codes to reduce the storage cost. We further propose an improved scheme, EPRQ+, which constructs a binary tree-based index to achieve faster-than-linear retrieval. Finally, our formal security analysis proves that our schemes are secure against Indistinguishability under Chosen-Plaintext Attack (IND-CPA), and extensive experiments demonstrate that our schemes are feasible in practice, where EPRQ+ scheme improves the storage efficiency by about 4 times and the query efficiency by about 8 times compared to the basic EPRQ.
{"title":"Efficient and Privacy-Preserving Encode-Based Range Query Over Encrypted Cloud Data","authors":"Yanrong Liang;Jianfeng Ma;Yinbin Miao;Yuan Su;Robert H. Deng","doi":"10.1109/TIFS.2024.3465928","DOIUrl":"10.1109/TIFS.2024.3465928","url":null,"abstract":"Privacy-preserving range query, which allows the server to implement secure and efficient range query on encrypted data, has been widely studied in recent years. Existing privacy-preserving range query schemes can realize effective range query, but usually suffer from the low efficiency and security. In order to solve the above issues, we propose an Efficient and Privacy-preserving encode-based Range Query over encrypted cloud data (namely basic EPRQ), which encodes the data and range by using Range Encode (REncoder), and then encrypts the codes via Additional Symmetric-Key Hidden Vector Encryption (ASHVE) technology. The basic EPRQ can achieve effective range query while ensuring privacy protection. Then, we split the codes to reduce the storage cost. We further propose an improved scheme, EPRQ+, which constructs a binary tree-based index to achieve faster-than-linear retrieval. Finally, our formal security analysis proves that our schemes are secure against Indistinguishability under Chosen-Plaintext Attack (IND-CPA), and extensive experiments demonstrate that our schemes are feasible in practice, where EPRQ+ scheme improves the storage efficiency by about 4 times and the query efficiency by about 8 times compared to the basic EPRQ.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"9085-9099"},"PeriodicalIF":6.3,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142313833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Virtual reality (VR), offering 3D visuals and stereophonic sounds, significantly enhances users’ immersive experiences and has become a milestone in the era of the metaverse. However, due to the limited battery capacity of VR devices, it is common for users to rely on charging cables, which serve the dual purpose of power supply and audio output, to recharge their VR devices while in use. In this study, we propose an inconspicuous and stealthy side channel attack, coined as LineTalker, which can unveil visual-related and audio-related activities from VR devices during the charging process. The insight behind LineTalker is rooted in the observation that visual-related activities (e.g., 3D image rendering) are power-intensive and result in fluctuations in the current strength of the cable’s power supply line, which can be leveraged as side channel information. Similarly, audio-related activities (e.g., playing music) leave traces on the cable’s audio output line. Rather than providing a user with a compromised charging cable (i.e., embedding a current sensor) to measure the current strength, to make the attack less conspicuous, LineTalker employs the Hall effect to indirectly access side channel information. This is achieved by capturing magnetic signals using a Hall sensor placed near the target cable in a contactless manner. Experimental results demonstrate that LineTalker achieves an overall accuracy of 94.60% and 64.38% in inferring user activities in VR devices with intrusive and non-intrusive attack manners, respectively.
{"title":"Dangers Behind Charging VR Devices: Hidden Side Channel Attacks via Charging Cables","authors":"Jiachun Li;Yan Meng;Yuxia Zhan;Le Zhang;Haojin Zhu","doi":"10.1109/TIFS.2024.3465026","DOIUrl":"10.1109/TIFS.2024.3465026","url":null,"abstract":"Virtual reality (VR), offering 3D visuals and stereophonic sounds, significantly enhances users’ immersive experiences and has become a milestone in the era of the metaverse. However, due to the limited battery capacity of VR devices, it is common for users to rely on charging cables, which serve the dual purpose of power supply and audio output, to recharge their VR devices while in use. In this study, we propose an inconspicuous and stealthy side channel attack, coined as LineTalker, which can unveil visual-related and audio-related activities from VR devices during the charging process. The insight behind LineTalker is rooted in the observation that visual-related activities (e.g., 3D image rendering) are power-intensive and result in fluctuations in the current strength of the cable’s power supply line, which can be leveraged as side channel information. Similarly, audio-related activities (e.g., playing music) leave traces on the cable’s audio output line. Rather than providing a user with a compromised charging cable (i.e., embedding a current sensor) to measure the current strength, to make the attack less conspicuous, LineTalker employs the Hall effect to indirectly access side channel information. This is achieved by capturing magnetic signals using a Hall sensor placed near the target cable in a contactless manner. Experimental results demonstrate that LineTalker achieves an overall accuracy of 94.60% and 64.38% in inferring user activities in VR devices with intrusive and non-intrusive attack manners, respectively.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"8892-8907"},"PeriodicalIF":6.3,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent research has shown that adversarial samples are highly transferable and can be used to attack other unknown black-box Deep Neural Networks (DNNs). To improve the transferability of adversarial samples, several feature-based adversarial attack methods have been proposed to disrupt neuron activation in the middle layers. However, current state-of-the-art feature-based attack methods typically require additional computation costs for estimating the importance of neurons. To address this challenge, we propose a Singular Value Decomposition (SVD)-based feature-level attack method. Our approach is inspired by the discovery that eigenvectors associated with the larger singular values decomposed from the middle layer features exhibit superior generalization and attention properties. Specifically, we conduct the attack by retaining the dominant decomposed feature that corresponds to the largest singular value (i.e., Rank-1 decomposed feature) for computing the output logits before the final softmax. These logits are later integrated with the original logits to optimize adversarial examples. Our extensive experimental results verify the effectiveness of our proposed method, which can be easily integrated into various baselines to significantly enhance the transferability of adversarial samples for disturbing normally trained CNNs and advanced defense strategies. The source code is available at Link.
{"title":"Boosting Adversarial Transferability via Logits Mixup With Dominant Decomposed Feature","authors":"Juanjuan Weng;Zhiming Luo;Shaozi Li;Dazhen Lin;Zhun Zhong","doi":"10.1109/TIFS.2024.3465212","DOIUrl":"10.1109/TIFS.2024.3465212","url":null,"abstract":"Recent research has shown that adversarial samples are highly transferable and can be used to attack other unknown black-box Deep Neural Networks (DNNs). To improve the transferability of adversarial samples, several feature-based adversarial attack methods have been proposed to disrupt neuron activation in the middle layers. However, current state-of-the-art feature-based attack methods typically require additional computation costs for estimating the importance of neurons. To address this challenge, we propose a Singular Value Decomposition (SVD)-based feature-level attack method. Our approach is inspired by the discovery that eigenvectors associated with the larger singular values decomposed from the middle layer features exhibit superior generalization and attention properties. Specifically, we conduct the attack by retaining the dominant decomposed feature that corresponds to the largest singular value (i.e., Rank-1 decomposed feature) for computing the output logits before the final softmax. These logits are later integrated with the original logits to optimize adversarial examples. Our extensive experimental results verify the effectiveness of our proposed method, which can be easily integrated into various baselines to significantly enhance the transferability of adversarial samples for disturbing normally trained CNNs and advanced defense strategies. The source code is available at Link.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"8939-8951"},"PeriodicalIF":6.3,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TIFS.2024.3465037
De Cheng;Haichun Tai;Nannan Wang;Chaowei Fang;Xinbo Gao
Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels, but these pseudo labels are inevitably noisy, which deteriorates model performance. In this paper, we propose a Neighbour Consistency guided Pseudo Label Refinement (NCPLR) framework, which can be regarded as a transductive form of label propagation under the assumption that the prediction of each example should be similar to its nearest neighbours’. Specifically, the refined label for each training instance can be obtained from the original clustering result and a weighted ensemble of its neighbours’ predictions, with weights determined according to their similarities in the feature space. Furthermore, we also explore building a unified global-local NCPLR mechanism through a global-local label interaction module to achieve mutual label refinement. Such a strategy promotes efficient complementary learning while mitigating some unreliable information, finally improving the quality of the refined pseudo labels for each global-local region. Extensive experimental results demonstrate the effectiveness of the proposed method, showing superior performance to state-of-the-art methods by a large margin. Our source code is released in https://github.com/haichuntai/NCPLR-ReID .
{"title":"Neighbor Consistency and Global-Local Interaction: A Novel Pseudo-Label Refinement Approach for Unsupervised Person Re-Identification","authors":"De Cheng;Haichun Tai;Nannan Wang;Chaowei Fang;Xinbo Gao","doi":"10.1109/TIFS.2024.3465037","DOIUrl":"10.1109/TIFS.2024.3465037","url":null,"abstract":"Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels, but these pseudo labels are inevitably noisy, which deteriorates model performance. In this paper, we propose a Neighbour Consistency guided Pseudo Label Refinement (NCPLR) framework, which can be regarded as a transductive form of label propagation under the assumption that the prediction of each example should be similar to its nearest neighbours’. Specifically, the refined label for each training instance can be obtained from the original clustering result and a weighted ensemble of its neighbours’ predictions, with weights determined according to their similarities in the feature space. Furthermore, we also explore building a unified global-local NCPLR mechanism through a global-local label interaction module to achieve mutual label refinement. Such a strategy promotes efficient complementary learning while mitigating some unreliable information, finally improving the quality of the refined pseudo labels for each global-local region. Extensive experimental results demonstrate the effectiveness of the proposed method, showing superior performance to state-of-the-art methods by a large margin. Our source code is released in \u0000<uri>https://github.com/haichuntai/NCPLR-ReID</uri>\u0000.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"9070-9084"},"PeriodicalIF":6.3,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The abuse of drones has raised critical concerns about public security and personal privacy, bringing an urgent requirement for drone recognition. Existing radio frequency (RF)-based recognition methods follow the assumption of the closed set, resulting in the unknown signals being misclassified as known classes. To address this problem, we propose a Signal Semantic-based open Set Recognition (S3R) method in this paper. First, the short-time Fourier transform is introduced to construct the signal spectra, decoupling the drone signals with other interference signals. Then, we design a texture extractor and a position extractor to extract the texture features and position features from the spectra, respectively. The extracted features are further fused and structurally optimized to construct distinguishable signal semantics. Based on the structural characteristics of signal semantics, an outlier analysis-based semantic classifier is proposed, which searches the outliers of each known class in the closed set as the bounding thresholds to detect unknown instances. Finally, the detected unknown instances are further classified into their exact classes by implementing clustering in a new semantic space, where semantics are augmented by introducing basic features from the intermediate layers of the texture extractor. Besides, a real-world spectrogram dataset of commonly-used drones is released, which includes 24 classes and covers 7 brands. Extensive experiments demonstrate that the proposed S3R method outperforms the state-of-the-art methods in terms of accuracy and generalizability for both the closed set and the open set.
{"title":"Open Set Learning for RF-Based Drone Recognition via Signal Semantics","authors":"Ningning Yu;Jiajun Wu;Chengwei Zhou;Zhiguo Shi;Jiming Chen","doi":"10.1109/TIFS.2024.3463535","DOIUrl":"10.1109/TIFS.2024.3463535","url":null,"abstract":"The abuse of drones has raised critical concerns about public security and personal privacy, bringing an urgent requirement for drone recognition. Existing radio frequency (RF)-based recognition methods follow the assumption of the closed set, resulting in the unknown signals being misclassified as known classes. To address this problem, we propose a Signal Semantic-based open Set Recognition (S3R) method in this paper. First, the short-time Fourier transform is introduced to construct the signal spectra, decoupling the drone signals with other interference signals. Then, we design a texture extractor and a position extractor to extract the texture features and position features from the spectra, respectively. The extracted features are further fused and structurally optimized to construct distinguishable signal semantics. Based on the structural characteristics of signal semantics, an outlier analysis-based semantic classifier is proposed, which searches the outliers of each known class in the closed set as the bounding thresholds to detect unknown instances. Finally, the detected unknown instances are further classified into their exact classes by implementing clustering in a new semantic space, where semantics are augmented by introducing basic features from the intermediate layers of the texture extractor. Besides, a real-world spectrogram dataset of commonly-used drones is released, which includes 24 classes and covers 7 brands. Extensive experiments demonstrate that the proposed S3R method outperforms the state-of-the-art methods in terms of accuracy and generalizability for both the closed set and the open set.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"9894-9909"},"PeriodicalIF":6.3,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TIFS.2024.3463971
Yubo Zheng;Peng Xu;Miao Wang;Wanying Xu;Wei Wang;Tianyang Chen;Hai Jin
Dynamic searchable symmetric encryption (DSSE), as one of the promising cryptographic tools in cloud-based services, faces two crying needs at the age of multi-device. One is a lightweight client, and the other is robustness. A lightweight client facilitates seamless synchronization among multiple devices allowing users to feel as if they are operating on a single device, even on resource-constrained devices. Robustness ensures a reliable system that can tolerate misoperations. DSSE requires both of them to achieve a leap in practicability. However, to our best knowledge, lightweight client and robustness have not been effectively combined thus far. Most existing DSSE schemes maintain a substantial amount of state information on the client for sub-linear search efficiency, but they fail to guarantee security even correctness, after executing the client’s misoperations (e.g., duplicate addition or deletion operation and deleting non-existent targets). The seminal work on robustness, ROSE (TIFS’22), leverages a heavy primitive to preserve security and correctness during post-processing and requires a heavy client storage burden. To guarantee robustness and constant client storage simultaneously, we devise a novel method to preserve robustness timely in the process of misoperations. Specifically, we introduce an alarm mechanism to promptly eliminate the effects of misoperations. Based on the misoperation alarm mechanism and the vORAM+HIRB oblivious map (S&P’16), we propose a new DSSE scheme Themis . In addition to satisfying robustness and constant client storage, it has competitive search and update performance compared to prior representative DSSE schemes. Moreover, it is superior to existing robust schemes in search.
{"title":"Themis: Robust and Light-Client Dynamic Searchable Symmetric Encryption","authors":"Yubo Zheng;Peng Xu;Miao Wang;Wanying Xu;Wei Wang;Tianyang Chen;Hai Jin","doi":"10.1109/TIFS.2024.3463971","DOIUrl":"10.1109/TIFS.2024.3463971","url":null,"abstract":"Dynamic searchable symmetric encryption (DSSE), as one of the promising cryptographic tools in cloud-based services, faces two crying needs at the age of multi-device. One is a lightweight client, and the other is robustness. A lightweight client facilitates seamless synchronization among multiple devices allowing users to feel as if they are operating on a single device, even on resource-constrained devices. Robustness ensures a reliable system that can tolerate misoperations. DSSE requires both of them to achieve a leap in practicability. However, to our best knowledge, lightweight client and robustness have not been effectively combined thus far. Most existing DSSE schemes maintain a substantial amount of state information on the client for sub-linear search efficiency, but they fail to guarantee security even correctness, after executing the client’s misoperations (e.g., duplicate addition or deletion operation and deleting non-existent targets). The seminal work on robustness, ROSE (TIFS’22), leverages a heavy primitive to preserve security and correctness during post-processing and requires a heavy client storage burden. To guarantee robustness and constant client storage simultaneously, we devise a novel method to preserve robustness timely in the process of misoperations. Specifically, we introduce an alarm mechanism to promptly eliminate the effects of misoperations. Based on the misoperation alarm mechanism and the \u0000<monospace>vORAM+HIRB</monospace>\u0000 oblivious map (S&P’16), we propose a new DSSE scheme \u0000<monospace>Themis</monospace>\u0000. In addition to satisfying robustness and constant client storage, it has competitive search and update performance compared to prior representative DSSE schemes. Moreover, it is superior to existing robust schemes in search.","PeriodicalId":13492,"journal":{"name":"IEEE Transactions on Information Forensics and Security","volume":"19 ","pages":"8802-8816"},"PeriodicalIF":6.3,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142275543","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：