Image and Vision Computing最新文献

Unpaired image dehazing models based on Cycle-Consistent Adversarial Networks (CycleGAN) typically consist of two cycle branches: dehazing-rehazing branch and hazing-dehazing branch. In these two branches, there is an asymmetry of information in the mutual transformation process between haze images and haze-free images. Previous models tended to focus more on the transformation process from haze images to haze-free images within the dehazing-rehazing branch, overlooking the provision of effective information for the formation of haze images in the hazing-dehazing branch. This oversight results in the production of haze patterns that are both monotonous and simplistic, ultimately impeding the overall performance and generalization capabilities of dehazing networks. In light of this, this paper proposes a novel model called HEDehazeNet (Dehazing Net based on Haze Generation Enhancement), which provides crucial information for the generation process of haze images through a dedicated haze generation enhancement module. This module is capable of producing three distinct modes of transmission maps - random transmission map, simulated transmission map, and mixed transmission maps combining both. Employing these transmission maps to generate hazing images with varying density and patterns provides the dehazing network with a more diverse and dynamically complex set of training samples, thereby enhancing its capacity to handle intricate scenes. Additionally, we made minor modifications to the U-Net, replacing residual blocks with multi-scale parallel convolutional blocks and channel self-attention, to further enhance the network's performance. Experiments were conducted on both synthetic and real-world datasets, substantiating the superiority of HEDehazeNet over the current state-of-the-art unpaired dehazing models.

Deep learning models are highly vulnerable to adversarial examples, leading to significant attention on techniques for detecting them. However, current methods primarily rely on detecting image features for identifying adversarial examples, often failing to address the diverse types and intensities of such examples. We propose a novel adversarial example detection method based on perturbation estimation and denoising to overcome this limitation. We develop an autoencoder to predict the latent adversarial perturbations of samples and select appropriately sized noise based on these predictions to cover the perturbations. Subsequently, we employ a non-blind denoising autoencoder to remove noise and residual perturbations effectively. This approach allows us to eliminate adversarial perturbations while preserving the original information, thus altering the prediction results of adversarial examples without affecting predictions on benign samples. Inconsistencies in predictions before and after processing by the model identify adversarial examples. Our experiments on datasets such as MNIST, CIFAR-10, and ImageNet demonstrate that our method surpasses other advanced detection methods in accuracy.

Human motion detection in complex scenarios poses challenges due to occlusions. This paper presents an integrated approach for accurate human motion detections by combining Adapted Canny Edge detection as a preprocessing step, backbone-modified Mask R-CNN for precise segmentation, Hybrid RDA-WOA-based RNN as a classification, and a Multiple-hypothesis model for effective occlusion handling. Adapted Canny Edge detection is utilized as an initial preprocessing step to highlight significant edges in the input image. The resulting edge map enhances object boundaries and highlights structural features, simplifying subsequent processing steps. The improved image is then passed through backbone-modified Mask R-CNN for the pixel-level segmentation of humans. Backbone-modified Mask R-CNN along with IoU, Euclidean Distance, and Z-Score recognizes moving objects in complex scenes exactly. After recognizing moving objects, the optimized Hybrid RDA-WOA-based RNN classifies humans. To handle the self-occlusion, Multiple Hypothesis Tracking (MHT) is used. Real-world situations frequently include occlusions where humans can be partially or completely hidden by objects. The proposed approach integrates a Multiple-hypothesis model into the detection pipeline to address this challenge. Moreover, the proposed human motion detection approach includes an optimized Hybrid RDA-WOA-based RNN trained with 2D representations of 3D skeletal motion. The proposed work was evaluated using the IXMAS, KTH, Weizmann, NTU RGB + D, and UCF101 Datasets. It achieved an accuracy of 98% on the IXMAS, KTH, Weizmann, and UCF101 Datasets and 97.1% on the NTU RGB + D Dataset. The simulation results unveil the superiority of the proposed methodology over the existing approaches.

Typically, Video Object Segmentation (VOS) always has the semi-supervised setting in the testing phase. The VOS aims to track and segment one or several target objects in the following frames in the sequence, only given the ground-truth segmentation mask in the initial frame. A fundamental issue in VOS is how to best utilize the temporal information to improve the accuracy. To address the aforementioned issue, we provide an end-to-end framework that simultaneously extracts long-term and short-term historical sequential information to current frame as temporal memories. The integrated temporal architecture consists of a short-term and a long-term memory modules. Specifically, the short-term memory module leverages a high-order graph-based learning framework to simulate the fine-grained spatial–temporal interactions between local regions across neighboring frames in a video, thereby maintaining the spatio-temporal visual consistency on local regions. Meanwhile, to relieve the occlusion and drift issues, the long-term memory module employs a Simplified Gated Recurrent Unit (S-GRU) to model the long evolutions in a video. Furthermore, we design a novel direction-aware attention module to complementarily augment the object representation for more robust segmentation. Our experiments on three mainstream VOS benchmarks, containing DAVIS 2017, DAVIS 2016, and Youtube-VOS, demonstrate that our proposed solution provides a fair tradeoff performance between both speed and accuracy.

Due to the high intrinsic similarity between camouflaged objects and the background, camouflaged objects often exhibit blurred boundaries, making it challenging to distinguish the boundaries of objects. Existing methods still focus on the overall regional accuracy but not on the boundary quality and are not competent to identify camouflaged objects from the background in complex scenarios. Thus, we propose a novel reverse cross-refinement network called RCR-Net. Specifically, we design a diverse feature enhancement module that simulates the correspondingly expanded receptive fields of the human visual system by using convolutional kernels with different dilation rates in parallel. Also, the boundary attention module is used to reduce the noise of the bottom features. Moreover, a multi-scale feature aggregation module is proposed to transmit the diverse features from pixel-level camouflaged edges to the entire camouflaged object region in a coarse-to-fine manner, which consists of reverse guidance, group guidance, and position guidance. Reverse guidance mines complementary regions and details by erasing already estimated object regions. Group guidance and position guidance integrate different features through simple and effective splitting and connecting operations. Extensive experiments show that RCR-Net outperforms the existing 18 state-of-the-art methods on four widely-used COD datasets. Especially, compared with the existing top-1 model HitNet, RCR-Net significantly improves the performance by ∼ 16.4% (Mean Absolute Error) on the CAMO dataset, showing that RCR-Net could accurately detect camouflaged objects.

In the field of autonomous driving, perception tasks based on Bird's-Eye-View (BEV) have attracted considerable research attention due to their numerous benefits. Despite recent advancements in performance, efficiency remains a challenge for real-world implementation. In this study, we propose an efficient and effective framework that constructs a spatio-temporal BEV feature from multi-camera inputs and leverages it for 3D object detection. Specifically, the success of our network is primarily attributed to the design of the lifting strategy and a tailored BEV encoder. The lifting strategy is tasked with the conversion of 2D features into 3D representations. In the absence of depth information in the images, we innovatively introduce a prior mask for the BEV feature, which can assess the significance of the feature along the camera ray at a low cost. Moreover, we design a lightweight BEV encoder, which significantly boosts the capacity of this physical-interpretation representation. In the encoder, we investigate the spatial relationships of the BEV feature and retain rich residual information from upstream. To further enhance performance, we establish a 2D object detection auxiliary head to delve into insights offered by 2D object detection and leverage the 4D information to explore the cues within the sequence. Benefiting from all these designs, our network can capture abundant semantic information from 3D scenes and strikes a balanced trade-off between efficiency and performance.

Model adaptation aims to transfer knowledge in pre-trained source models to a new unlabeled dataset. Despite impressive progress, prior methods always need to access the source model and develop data-reconstruction approaches to align the data distributions between target samples and the generated instances, which may raise privacy concerns from source individuals. To alleviate the above problem, we propose a new method in the setting of Black-box model adaptation for semantic segmentation, in which only the pseudo-labels from multiple source domain is required during the adaptation process. Specifically, the proposed method structurally distills the knowledge with multiple classifiers to obtain a customized target model, and then the predictions of target data are refined to fit the target domain with co-regularization. We conduct extensive experiments on several standard datasets, and our method can achieve promising results.

Remote sensing (RS) image dehazing holds significant importance in enhancing the quality and information extraction capability of RS imagery. The enhancement in image dehazing quality has progressively advanced alongside the evolution of convolutional neural network (CNN). Due to the fixed receptive field of CNN, there is insufficient utilization of contextual information on haze features in multi-scale RS images. Additionally, the network fails to adequately extract both local and global information of haze features. In addressing the above problems, in this paper, we propose an RS image dehazing network based on multi-scale large kernel convolution and hybrid attention (MKHANet). The network is mainly composed of multi-scale large kernel convolution (MSLKC) module, hybrid attention (HA) module and feature fusion attention (FFA) module. The MSLKC module fully fuses the multi-scale information of features while enhancing the effective receptive field of the network by parallel multiple large kernel convolutions. To alleviate the problem of uneven distribution of haze and effectively extract the global and local information of haze features, the HA module is introduced by focusing on the importance of haze pixels at the channel level. The FFA module aims to boost the interaction of feature information between the network's deep and shallow layers. The subjective and objective experimental results on on multiple RS hazy image datasets illustrates that MKHANet surpasses existing state-of-the-art (SOTA) approaches. The source code is available at https://github.com/tohang98/MKHA_Net.

The identification of anomalies in videos is a particularly complex visual challenge, given the wide variety of potential real-world events. To address this issue, our paper introduces a unique approach for detecting divergent behavior in surveillance videos, utilizing triplet-loss for video anomaly detection. Our method involves selecting a triplet set of video segments from normal (n) and abnormal (a) data points for deep feature learning. We begin by creating a database of triplet sets of two types: a-a-n and n-n-a. By computing a triplet loss, we model the proximity between n-n chunks and the distance between ‘a’ chunks from the n-n ones. Additionally, we train the deep network to model the closeness of a-a chunks and the divergent behavior of ‘n’ from the a-a chunks.

The model acquired in the initial stage can be viewed as a prior, which is subsequently employed for modeling normality. As a result, our method can leverage the advantages of both straightforward classification and normality modeling-based techniques. We also present a data selection mechanism for the efficient generation of triplet sets. Furthermore, we introduce a novel video anomaly dataset, AnoVIL, designed for human-centric anomaly detection. Our proposed method is assessed using the UCF-Crime dataset encompassing all 13 categories, the IIT-H accident dataset, and AnoVIL. The experimental findings demonstrate that our method surpasses the current state-of-the-art approaches. We conduct further evaluations of the performance, considering various configurations such as cross-dataset evaluation, loss functions, siamese structure, and embedding size. Additionally, an ablation study is carried out across different settings to provide insights into our proposed method.

Multi-scale object detection is a preeminent challenge in computer vision and image processing. Several deep learning models that are designed to detect various objects miss out on the detection capabilities for small objects, reducing their detection accuracies. Intending to focus on different scales, from extremely small to large-sized objects, this work proposes a Spatially Dilated Multi-Scale Network (SDMNet) architecture for UAV-based ground object detection. It proposes a Multi-scale Enhanced Effective Channel Attention mechanism to preserve the object details in the images. Additionally, the proposed model incorporates dilated convolution, sub-pixel convolution, and additional prediction heads to enhance object detection performance specifically for aerial imaging. It has been evaluated on two popular aerial image datasets, VisDrone 2019 and UAVDT, containing publicly available annotated images of ground objects captured from UAV. Different performance metrics, such as precision, recall, mAP, and detection rate, benchmark the proposed architecture with the existing object detection approaches. The experimental results demonstrate the effectiveness of the proposed model for multi-scale object detection with an average precision score of 54.2% and 98.4% for VisDrone and UAVDT datasets, respectively.