Journal of Visual Communication and Image Representation最新文献

With the rising demands for remote desktops and online meetings, screen content videos have drawn significant attention. Different from natural videos, screen content videos often exhibit scene switches where the content abruptly changes from one frame to the next. These scene switches result in obvious distortions in compressed videos. Besides, frame freezing, where the content remains unchanged for a certain duration, is also very common in screen content videos. Existing alignment-based models struggle to effectively enhance scene switch frames and lack efficiency when dealing with frame freezing situations. Therefore, we propose a novel alignment-free method that effectively handles both scene switches and frame freezing. In our approach, we develop a spatial and temporal feature extraction module that compresses and extracts spatio-temporal information from three groups of frame inputs. This enables efficient handling of scene switches. In addition, an edge aware block is proposed for extracting edge information, which guides the model to focus on restoring the high-frequency components in frame freezing situations. The fusion module is then designed to adaptively fuse the features from three groups, considering different positions of video frames, to enhance frames during scene switch and frame freezing scenarios. Experimental results demonstrate the significant advancements achieved by the proposed edge aware with spatio-temporal information fusion network (EAST) in enhancing the quality of compressed videos, surpassing the current state-of-the-art methods.

Recently, forensics has encountered a new challenge with video surveillance object forgery. This type of forgery combines the characteristics of popular video copy-move and splicing forgeries, failing most existing video forgery detection schemes. In response to this new forgery challenge, this paper proposes a Video Surveillance Object Forgery Detection (VSOFD) method including three parts components: (i) The proposed method presents a special-combined extraction technique that incorporates Temporal-Spatial-Frequent (TSF) perspectives for TSF feature extraction. Furthermore, TSF features can effectively represent video information and benefit from feature dimension reduction, improving computational efficiency. (ii) The proposed method introduces a universal, extensible attention-based Convolutional Neural Network (CNN) baseline for feature processing. This CNN processing architecture is compatible with various series and parallel feed-forward CNN structures, considering these structures as processing backbones. Therefore, the proposed CNN architecture benefits from various state-of-the-art structures, leading to addressing each independent TSF feature. (iii) The method adopts an encoder-attention-decoder RNN framework for feature classification. By incorporating temporal characteristics, the framework can further identify the correlations between the adjacent frames to classify the forgery frames better. Finally, experimental results show that the proposed network can achieve the best F₁ = 94.69 % score, increasing at least 5–12 % from the existing State-Of-The-Art (SOTA) VSOFD schemes and other video forensics.

There are many challenging scenarios in the task of salient object detection in optical remote sensing images (RSIs), such as various scales and irregular shapes of salient objects, cluttered backgrounds, etc. Therefore, it is difficult to directly apply saliency models targeting natural scene images to optical RSIs. Besides, existing models often do not give sufficient exploration for the potential relationship of different salient objects or different parts of the salient object. In this paper, we propose a graph interaction network (i.e. GINet) with semantic-guided spatial refinement to conduct salient object detection in optical RSIs. The key advantages of GINet lie in two points. Firstly, the graph interactive reasoning (GIR) module conducts information exchange of different-level features via the graph interaction operation, and enhances features along spatial and channel dimensions via the graph reasoning operation. Secondly, we designed the global content-aware refinement (GCR) module, which incorporates the foreground and background feature-based local information and the semantic feature-based global information simultaneously. Experiments results on two public optical RSIs datasets clearly show the effectiveness and superiority of the proposed GINet when compared with the state-of-the-art models.

Salient Object Detection (SOD) approaches usually aggregate high-level semantics with object details layer by layer through a pyramid fusion structure. However, the progressive feature fusion mechanism may lead to gradually dilution of valuable semantics and prediction accuracy. In this work, we propose a Cross-stage Feature Fusion Network (CFFNet) for salient object detection. CFFNet consists of a Cross-stage Semantic Fusion Module (CSF), a Feature Filtering and Fusion Module (FFM), and a progressive decoder to tackle the above problems. Specifically, to alleviate the semantics dilution problem, CSF concatenates different stage backbone features and extracts multi-scale global semantics using transformer blocks. Global semantics are then distributed to corresponding backbone stages for cross-stage semantic fusion. The FFM module implements efficient self-attention-based feature fusion. Different from regular self-attention which has quadratic computational complexity. Finally, a progressive decoder is adopted to refine saliency maps. Experimental results demonstrate that CFFNet outperforms state-of-the-arts on six SOD datasets.

Depth maps can provide supplementary information for salient object detection (SOD) and perform better in handling complex scenes. Most existing RGB-D methods only utilize deep cues at the same level, and few methods focus on the information linkage between cross-level features. In this study, we propose a Progressive Cross-level Fusion Network (PCF-Net). It ensures the cross-flow of cross-level features by gradually exploring deeper features, which promotes the interaction and fusion of information between different-level features. First, we designed a Cross-Level Guide Cross-Modal Fusion Module (CGCF) that utilizes the spatial information of upper-level features to suppress modal feature noise and to guide lower-level features for cross-modal feature fusion. Next, the proposed Semantic Enhancement Module (SEM) and Local Enhancement Module (LEM) are used to further introduce deeper features, enhance the high-level semantic information and low-level structural information of cross-modal features, and use self-modality attention refinement to improve the enhancement effect. Finally, the multi-scale aggregation decoder mines enhanced feature information in multi-scale spaces and effectively integrates cross-scale features. In this study, we conducted numerous experiments to demonstrate that the proposed PCF-Net outperforms 16 of the most advanced methods on six popular RGB-D SOD datasets.

Due to different imaging principles of visible-infrared cameras, there are modal differences between similar person images. For visible-infrared person re-identification (VI-ReID), existing works focus on extracting and aligning cross-modal global descriptors in the shared feature space, while ignoring local variations and graph-structural correlations in cross-modal image pairs. In order to bridge the modal differences with graph-structure between key-parts, a Meta-Graph Isomerization Aggregation Module (MIAM) is proposed, which includes Meta-Graph Node Isomerization Module (MNI) and Dual Aggregation Module (DA). In order to completely describe discriminative in-graph local features, MNI establishes meta-secondary cyclic isomorphism relations of in-graph local features by using a multi-branch embedding generation mechanism. Then, the local features not only contain the limited information of the fixed regions, but also benefit from the neighboring regions. Meanwhile, the secondary node generation process considers similar and different nodes of pedestrian graph structure to reduce the interference of identity differences. In addition, the Dual Aggregation (DA) module combines spatial self-attention and channel self-attention to establish the interdependence of modal heterogeneous graph structure pairs and achieve inter-modal feature aggregation. To match heterogeneous graph structures, a Node Center Joint Mining Loss (NCJM Loss) is proposed to constrain the distance between the node centers of heterogeneous graphs. The experiments performed on the SYSU-MM01, RegDB and LLCM public datasets demonstrate that the proposed method performs excellently in VI-ReID.

DeepFake face forgery has a serious negative impact on both society and individuals. Therefore, research on DeepFake detection technologies is necessary. At present, DeepFake detection technology based on deep learning has achieved acceptable results on high-quality datasets; however, its detection performance on low-quality datasets and cross-datasets remains poor. To address this problem, this paper presents a multi-scale interactive dual-stream network (MSIDSnet). The network is divided into spatial- and frequency-domain streams and uses a multi-scale fusion module to capture both the facial features of images that have been manipulated in the spatial domain under different circumstances and the fine-grained high-frequency noise information of forged images. The network fully integrates the features of the spatial- and frequency-domain streams through an interactive dual-stream module and uses vision transformer (ViT) to further learn the global information of the forged facial features for classification. Experimental results confirm that the accuracy of this method reached 99.30 % on the high-quality dataset Celeb-DF-v2, and 95.51 % on the low-quality dataset FaceForensics++. Moreover, the results of the cross-dataset experiments were superior to those of the other comparison methods.

This paper proposed an efficient and novel end-to-end text detection and recognition framework called DiZNet. DiZNet is built upon a core representation using text detail maps and employs the classical lightweight ResNet18 as the backbone for the text detection and recognition algorithm model. The redesigned Text Attention Head (TAH) takes multiple shallow backbone features as input, effectively extracting pixel-level information of text in images and global text positional features. The extracted text features are integrated into the stackable Feature Pyramid Enhancement Fusion Module (FPEFM). Supervised with text detail map labels, which include boundary information and texture of important text, the model predicts text detail maps and fuses them into the text detection and recognition heads. Through end-to-end testing on publicly available natural scene text benchmark datasets, our approach demonstrates robust generalization capabilities and real-time detection speeds. Leveraging the advantages of text detail map representation, DiZNet achieves a good balance between precision and efficiency on challenging datasets. For example, DiZNet achieves 91.2% Precision and 85.9% F-measure at a speed of 38.4 FPS on Total-Text and 83.8% F-measure at a speed of 30.0 FPS on ICDAR2015, it attains 83.8% F-measure at 30.0 FPS. The code is publicly available at: https://github.com/DiZ-gogogo/DiZNet

Accurate localization and delineation of vertebrae are crucial for diagnosing and treating spinal disorders. To achieve this, we propose an efficient X-ray full-spine vertebra instance segmentation method based on an enhanced U-Net architecture. Several key improvements have been made: the ConvNeXt encoder is employed to effectively capture complex features, and IFE feature extraction is introduced in the skip connections to focus on texture-rich and edge-clear clues. The CBAM attention mechanism is used in the bottleneck to integrate coarse and fine-grained semantic information. The decoder employs a residual structure combined with skip connections to achieve multi-scale contextual information and feature fusion. Our method has been validated through experiments on anterior-posterior and lateral spinal segmentation, demonstrating robust feature extraction and precise semantic segmentation capabilities. It effectively handles various spinal disorders, including scoliosis, vertebral wedging, lumbar spondylolisthesis and spondylolysis. This segmentation foundation enables rapid calibration of vertebral parameters and the computation of relevant metrics, providing valuable references and guidance for advancements in medical imaging.

This manuscript introduces an innovative bilevel optimization approach designed to improve the deblurring process by incorporating a nonlocal $p$-Laplacien model with variable weights. The study includes a theoretical analysis to examine the model’s solution, and an effective algorithm is devised for computing the pristine image, incorporating the learning of parameters associated with weights and nonlocal regularization terms. By carefully selecting these parameters, the suggested nonlocal deblurring model demonstrates superior effectiveness and performance when compared to other existing models.