Signal Processing-Image Communication最新文献

Existing methods for infrared and visible image fusion (IVIF) often overlook the analysis of common and distinct features among source images. Consequently, this study develops A self-supervised infrared and visible image fusion based on co-attention network, incorporating auxiliary networks and backbone networks in its design. The primary concept is to transform both common and distinct features into common features and reconstructed features, subsequently deriving the distinct features through their subtraction. To enhance the similarity of common features, we designed the fusion block based on co-attention (FBC) module specifically for this purpose, capturing common features through co-attention. Moreover, fine-tuning the auxiliary network enhances the image reconstruction effectiveness of the backbone network. It is noteworthy that the auxiliary network is exclusively employed during training to guide the self-supervised completion of IVIF by the backbone network. Additionally, we introduce a novel estimate for weighted fidelity loss to guide the fused image in preserving more brightness from the source image. Experiments conducted on diverse benchmark datasets demonstrate the superior performance of our S2CANet over state-of-the-art IVIF methods.

The distinguishable deep features are essential for the 3D point cloud recognition as they influence the search for the optimal classifier. Most existing point cloud classification methods mainly focus on local information aggregation while ignoring the feature distribution of the whole dataset that indicates more informative and intrinsic semantic relationships of labeled data, if better exploited, which could learn more distinguishing inter-class features. Our work attempts to construct a more distinguishable feature space through performing feature distribution refinement inspired by contrastive learning and sample mining strategies, without modifying the model architecture. To explore the full potential of feature distribution refinement, two modules are involved to boost exceptionally distributed samples distinguishability in an adaptive manner: (i) Confusion-Prone Classes Mining (CPCM) module is aimed at hard-to-distinct classes, which alleviates the massive category-level confusion by generating class-level soft labels; (ii) Entropy-Aware Attention (EAA) mechanism is proposed to remove influence of the trivial cases which could substantially weaken model performance. Our method achieves competitive results on multiple applications of point cloud. In particular, our method gets 85.8% accuracy on ScanObjectNN, and substantial performance gains up to 2.7% in DCGNN, 3.1% in PointNet++, and 2.4% in GBNet. Our code is available at https://github.com/YangFengSEU/CEDR.

In recent years, deep neural networks pretrained on large-scale datasets have been used to address data deficiency and achieve better performance through prior knowledge. Contrastive language–image pretraining (CLIP), a vision-language model pretrained on an extensive dataset, achieves better performance in image recognition. In this study, we harness the power of multimodality in image clustering tasks, shifting from a single modality to a multimodal framework using the describability property of image encoder of the CLIP model. The importance of this shift lies in the ability of multimodality to provide richer feature representations. By generating text centroids corresponding to image features, we effectively create a common descriptive language for each cluster. It generates text centroids assigned by the image features and improves the clustering performance. The text centroids use the results generated by using the standard clustering algorithm as a pseudo-label and learn a common description of each cluster. Finally, only text centroids were added when the image features on the same space were assigned to the text centroids, but the clustering performance improved significantly compared to the standard clustering algorithm, especially on complex datasets. When the proposed method is applied, the normalized mutual information score rises by 32% on the Stanford40 dataset and 64% on ImageNet-Dog compared to the $k$-means clustering algorithm.

Indoor 3D object detection is an essential task in single image scene understanding, impacting spatial cognition fundamentally in visual reasoning. Existing works on 3D object detection from a single image either pursue this goal through independent predictions of each object or implicitly reason over all possible objects, failing to harness relational geometric information between objects. To address this problem, we propose a sparse graph-based pipeline named Explicit3D based on object geometry and semantics features. Taking the efficiency into consideration, we further define a relatedness score and design a novel dynamic pruning method via group sampling for sparse scene graph generation and updating. Furthermore, our Explicit3D introduces homogeneous matrices and defines new relative loss and corner loss to model the spatial difference between target pairs explicitly. Instead of using ground-truth labels as direct supervision, our relative and corner loss are derived from homogeneous transforms, which renders the model to learn the geometric consistency between objects. The experimental results on the SUN RGB-D dataset demonstrate that our Explicit3D achieves better performance balance than the-state-of-the-art.

Cropping an image is a common image editing technique that aims to find viewpoints with suitable image composition. It is also a frequently used post-processing technique to reduce the evidence of tampering in an image. Detecting cropped images poses a significant challenge in the field of digital image forensics, as the distortions introduced by image cropping are often imperceptible to the human eye. Although deep neural networks achieve state-of-the-art performance, due to their ability to encode large-scale data and handle billions of model parameters. However, due to their high computational complexity and substantial storage requirements, it is difficult to deploy these large deep learning models on resource-constrained devices such as mobile phones and embedded systems. To address this issue, we propose a lightweight deep learning framework for cropping detection in spatial domain, based on knowledge distillation. Initially, we constructed four datasets containing a total of 60,000 images cropped using various tools. We then used Efficient-Net-B0, pre-trained on ImageNet with significant surgical adjustments, as the teacher model, which makes it more robust and faster to converge in this downstream task. The model was trained on 20,000 cropped and uncropped images from our own dataset, and we then applied its knowledge to a more compact model called the student model. Finally, we selected the best-performing lightweight model as the final prediction model, with a testing accuracy of 98.44% on the test dataset, which outperforms other methods. Extensive experiments demonstrate that our proposed model, distilled from Efficient-Net-B0, achieves state-of-the-art performance in terms of detection accuracy, training parameters, and FLOPs, outperforming existing methods in detecting cropped images.

For immediate access and online backup, cloud storage has become a mainstream way for digital data storage and distribution. To assure images accessed or downloaded from clouds are reliable is critical to storage service providers. In this study, a new distortion-free color image authentication method based on secret sharing, data compression and image interpolation with the tampering recovery capability is proposed. The proposed method generates elaborate authentication signals which have double functions of tampering localization and image repairing. The authentication signals are subsequently converted into many shares with the use of (k, n)-threshold method so as to increase the multiplicity of authentication signals for reinforcing the capability of tampering recovery. These shares are then randomly concealed in the alpha channel of the to-be-protected image that has been transformed into the PNG format containing RGBA channels. In authentication, the authentication signals computed from the alpha channel are not only used for indicating if an image block has been tampered with or not, but used as a signal to find the corresponding color in a predefined palette to recover the tampered image block. Compared with several state-of-the-art methods, the proposed method attains positive properties including losslessness, tampering localization and tampering recovery. Experimental results and discussions on security consideration and comparison with other related methods are provided to demonstrate the outperformance of the proposed method.

Object cosegmentation aims to obtain common objects from multiple images or videos, which performs by employing handcraft features to evaluate region similarity or learning higher semantic information via deep learning. However, the former based on handcraft features is sensitive to illumination, appearance changes and clutter background to the domain gap. The latter based on deep learning needs the groundtruth of object segmentation to train the co-attention model to spotlight the common object regions in different domain. This paper proposes an adversarial domain adaption-based video object cosegmentation method without any pixel-wise supervision. Intuitively, high-level semantic similarity are beneficial for common object recognition. However, there are inconsistency distributions of different video sources, i.e., domain gap. We propose an adversarial learning method to align feature distributions of different videos, which aims to maintain the feature similarity of common objects to overcome the dataset bias. Hence, a feature encoder via Siamese network is constructed to fool a discriminative network to obtain domain adapted feature mapping. To further assist the feature embedding of common objects, we define a latent task for label generation to train a classifying network, which could make full use of high-level semantic information. Experimental results on several video cosegmentation datasets suggest that domain adaption based on adversarial learning could significantly improve the common semantic feature exaction.

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

The task of binarization of historical document images has been in the forefront of image processing research, during the digital transition of libraries. The process of storing and transcribing valuable historical printed or handwritten material can salvage world cultural heritage and make it available online without physical attendance. The task of binarization can be viewed as a pre-processing step that attempts to separate the printed/handwritten characters in the image from possible noise and stains, which will assist in the Optical Character Recognition (OCR) process. Many approaches have been proposed before, including deep learning based approaches. In this article, we propose a U-Net style deep learning architecture that incorporates many other developments of deep learning, including residual connections, multi-resolution connections, visual attention blocks and dilated convolution blocks for upsampling. The novelties in the proposed DMVAnet lie in the use of these elements in combination in a novel U-Net style architecture and the application of DMVAnet in image binarization for the first time. In addition, the proposed DMVAnet is a very computationally lightweight network that performs very close or even better than the state-of-the-art approaches with a fraction of the network size and parameters. Finally, it can be used on platforms with restricted processing power and system resources, such as mobile devices and through scaling can result in inference times that allow for real-time applications.

Anomaly detection in multimedia datasets is a widely studied area. Yet, the concept drift challenge in data has been ignored or poorly handled by the majority of the anomaly detection frameworks. The state-of-the-art approaches assume that the data distribution at training and deployment time will be the same. However, due to various real-life environmental factors, the data may encounter drift in its distribution or can drift from one class to another in the late future. Thus, a one-time trained model might not perform adequately. In this paper, we systematically investigate the effect of concept drift on various detection models and propose a modified Adaptive Gaussian Mixture Model (AGMM) based framework for anomaly detection in multimedia data. In contrast to the baseline AGMM, the proposed extension of AGMM remembers the past for a longer period in order to handle the drift better. Extensive experimental analysis shows that the proposed model better handles the drift in data as compared with the baseline AGMM. Further, to facilitate research and comparison with the proposed framework, we contribute three multimedia datasets constituting faces as samples. The face samples of individuals correspond to the age difference of more than ten years to incorporate a longer temporal context.