Pattern Recognition最新文献

Text localization across multiple domains is crucial for applications like autonomous driving and tracking marathon runners. This work introduces DIPCYT, a novel model that utilizes Domain Independent Partial Convolution and a Yolov5-based Transformer for text localization in scene images from various domains, including natural scenes, underwater, and drone images. Each domain presents unique challenges: underwater images suffer from poor quality and degradation, drone images suffer from tiny text and loss of shapes, and scene images suffer from arbitrarily oriented, shaped text. Additionally, license plates in drone images may not provide rich semantic information compared to other text types due to loss of contextual information between characters. To tackle these challenges, DIPCYT employs new partial convolution layers within Yolov5 and integrates Transformer detection heads with a novel Fourier Positional Convolutional Block Attention Module (FPCBAM). This approach leverages common text properties across domains, such as contextual (global) and spatial (local) relationships. Experimental results demonstrate that DIPCYT outperforms existing methods, achieving F-scores of 0.90, 0.90, 0.77, 0.85, 0.85, and 0.88 on Total-Text, ICDAR 2015, ICDAR 2019 MLT, CTW1500, Drone, and Underwater datasets, respectively.

Video Anomaly Detection (VAD) aims to identify events in videos that deviate from typical patterns. Given the scarcity of anomalous samples, previous research has primarily focused on learning regular patterns from datasets exclusively containing normal behaviors, and treating deviations from these patterns as anomalies. However, most of these methods are constrained by coarse-grained modeling approaches that renders them incapable of learning highly-discriminative features, which are necessary to effectively distinguish between the subtle differences between normal and abnormal behaviors. To better capture these features, we propose an innovative method. Initially, pseudo-anomalous samples for appearance and motion are generated through geometric transformations (2D rotations) and the scrambling of video sequences. Subsequently, a dual-branch network featuring spatio-temporal decoupling is proposed, in which the spatial and temporal branches each handle a specific proxy task. These tasks are designed to distinguish between normal and pseudo-anomalous samples, involving operations such as predicting patch-based 2D rotation angles and classifying video frame triplets as total-anomaly, left-anomaly, right-anomaly, and non-anomaly. Our approach employs an end-to-end training methodology, without relying on pre-trained models (except for the object detector). Evaluations on the UCSD Ped2, Avenue, and ShanghaiTech datasets show that our method achieved AUC scores of 99.1%, 91.9%, and 81.1%, respectively, demonstrating its effectiveness. The code is publicly accessible at the following link: https://spatio-temporal-tasks.

Fashion-image editing is a challenging computer-vision task where the goal is to incorporate selected apparel into a given input image. Most existing techniques, known as Virtual Try-On methods, deal with this task by first selecting an example image of the desired apparel and then transferring the clothing onto the target person. Conversely, in this paper, we consider editing fashion images with text descriptions. Such an approach has several advantages over example-based virtual try-on techniques: (i) it does not require an image of the target fashion item, and (ii) it allows the expression of a wide variety of visual concepts through the use of natural language. Existing image-editing methods that work with language inputs are heavily constrained by their requirement for training sets with rich attribute annotations or they are only able to handle simple text descriptions. We address these constraints by proposing a novel text-conditioned editing model called FICE (Fashion Image CLIP Editing) that is capable of handling a wide variety of diverse text descriptions to guide the editing procedure. Specifically, with FICE, we extend the common GAN-inversion process by including semantic, pose-related, and image-level constraints when generating images. We leverage the capabilities of the CLIP model to enforce the text-provided semantics, due to its impressive image–text association capabilities. We furthermore propose a latent-code regularization technique that provides the means to better control the fidelity of the synthesized images. We validate the FICE through rigorous experiments on a combination of VITON images and Fashion-Gen text descriptions and in comparison with several state-of-the-art, text-conditioned, image-editing approaches. Experimental results demonstrate that the FICE generates very realistic fashion images and leads to better editing than existing, competing approaches. The source code is publicly available from: https://github.com/MartinPernus/FICE.

In the field of graph neural networks (GNNs) for representation learning, a noteworthy highlight is the potential of embedding fusion architectures for augmented graphs. However, prevalent GNN embedding fusion architectures mainly focus on handling graph combinations from a global perspective, often ignoring their collaboration with the information of local graph combinations. This inherent limitation constrains the ability of the constructed models to handle multiple input graphs, particularly when dealing with noisy input graphs collected from error-prone sources or those resulting from deficiencies in graph augmentation methods. In this paper, we propose an effective and robust embedding fusion architecture from a local-to-global perspective termed collaborative graph neural networks for augmented graphs (LoGo-GNN). Essentially, LoGo-GNN leverages a pairwise graph combination scheme to generate local perspective inputs. Together with the global graph combination, this serves as the basis to generate a local-to-global perspective. Specifically, LoGo-GNN employs a perturbation augmentation strategy to generate multiple augmentation graphs, thereby facilitating collaboration and embedding fusion from a local-to-global perspective through the use of graph combinations. In addition, LoGo-GNN incorporates a novel loss function for learning complementary information between different perspectives. We also conduct theoretical analysis to assess its expressive power under ideal conditions, demonstrating the effectiveness of LoGo-GNN. Our experiments, focusing on node classification and clustering tasks, highlight the superior performance of LoGo-GNN compared to state-of-the-art methods. Additionally, robustness analysis further confirms its effectiveness in addressing uncertainty challenges.

Asymmetric appearance between positive pair effectively reduces the risk of representation degradation in contrastive learning. However, there are still a mass of appearance similarities between positive pair constructed by the existing methods, thus inhibiting the further representation improvement. To address the above issue, we propose a novel asymmetric patch sampling strategy, which significantly reduces the appearance similarities but retains the image semantics. Specifically, dual patch sampling strategies are respectively applied to the given image. First, sparse patch sampling is conducted to obtain the first view, which reduces spatial redundancy of image and allows a more asymmetric view. Second, a selective patch sampling is proposed to construct another view with large appearance discrepancy relative to the first one. Due to the inappreciable appearance similarities between positive pair, the trained model is encouraged to capture the similarities on semantics, instead of low-level ones.

Experimental results demonstrate that our method significantly outperforms the existing self-supervised learning methods on ImageNet-1K and CIFAR datasets, e.g., 2.5% finetuning accuracy improvement on CIFAR100. Furthermore, our method achieves state-of-the-art performance on downstream tasks, object detection and instance segmentation on COCO. Additionally, compared to other self-supervised methods, our method is more efficient on both memory and computation during pretraining. The source code and the trained weights are available at https://github.com/visresearch/aps.

Recently, infrared (IR) small target detection problem has attracted increasing attention. Tensor component analysis-based techniques have been widely utilized, while they are faced with challenges such as tensor structures, background and target estimation, and real-time performance. In this paper, we propose a 5-D spatial–temporal factor-based completion model (5D-STFC) for IR small target detection. Specifically, a 5-D whitened spatial–temporal patch-tensor is constructed. Then, we devise a spatial–temporal factor-based low-rank background estimation norm and a Moreau envelope-derived sparsity estimation norm based on joint spatial–temporal knowledge. Furthermore, we establish a comprehensive completion model for component analysis. To efficiently solve this model, we design a multi-block alternating direction method of multipliers (multi-block ADMM)-based optimization scheme. Extensive experiments conducted on five real IR sequences demonstrate the superiority of 5D-STFC over nine state-of-the-art competitive methods. It can be concluded that 5D-STFC is excellent and practical in target detectability, background suppressibility, overall performance, and real-time performance.

Recently, scene Chinese recognition has attracted increasing attention. While mainstream scene text recognition methods exhibit outstanding performance in English recognition, they are considerably limited in Chinese recognition, due to inter-class similarity, intra-class variability, and complex combination of components in scene Chinese text. In this paper, we design Adaptive Position Encoding(APE) to enhance the model’s ability to perceive spatial information. Based on APE, we have innovatively designed Local Attention Module (LAM) and Global Attention Module (GAM). Specifically, LAM captures local features to identify common characteristics among characters of the same category, addressing the issue of intra-class variability. Meanwhile, LAM captures global features to identify the subordination relationships of Chinese character components. By integrating LAM and GAM, combining both local and global features, it is possible to find differences in the details among features that are fundamentally similar, thus solving the problem of inter-class similarity. Further, we contrive the transformer encoder–decoder structure to identify the vast variety of Chinese characters. Based on the Local/Global Attention Module and transformer encoder–decoder framework, we devise the novel sequence-to-sequence Local and Global Attention Network(LGANet), where both the backbone and the encoder/decoder are composed of attention mechanisms. Subsequent experiments on the Chinese scene dataset show that the recognition accuracy of our proposed LGANet is 77.3% and the normalized editing distance is 88.6%, both of which achieve the SOTA results in Fig. 1.

High dynamic range (HDR) video represents a wider range of brightness, detail and colour than standard dynamic range (SDR) video. However, SDR-based VQA (Video Quality Assessment) models struggle to capture HDR distortions. In addition, some of the existing methods designed for HDR video focus on emphasising the distortion of local areas of the video frame, ignoring the distortion of the video frame as a whole. Therefore, we propose a no reference VQA model based on luminance decomposition and recombination that provides excellent performance for HDR videos, called HDR-DRVQA. Specifically, HDR-DRVQA utilises a luminance decomposition strategy to decompose video frames into different regions for explicit extraction of perceptual features in different regions of the high dynamic range. We then further propose a residual aggregation module for recombining multi-region features to extract static spatial distortion representations and dynamic motion perception (captured by feature differences). Taking advantage of the Transformer network in remote dependency modelling, this information is fed into the Transformer network for interactive learning of motion perception and adaptively constructs a stream of spatial distortion information from shallow to deep layers during temporal aggregation. We validate that our model significantly outperforms SDR VQA and existing HDR VQA methods on the publicly available HDR databases.

Image deraining refers to removing the visible rain streaks to restore the rain-free scenes. Existing methods rely on manually crafted networks to model the distribution of rain streaks. However, complex scenes disrupt the uniformity of rain streak characteristics assumed in ideal conditions, resulting in rain streaks of varying directions, intensities, and brightness intersecting within the same scene, challenging the deep learning based deraining performance. To address the chaotic rain streak removal, we handle the rain streaks with similar distribution characteristics in the same layer and employ a dynamic recursive mechanism to extract and unveil them progressively. Specifically, we employ neural architecture search to determine the models of different rain streaks. To avoid the loss of texture details associated with overly deep structures, we integrate multi-scale modeling and cross-scale recruitment within the dynamic structure. Considering the application of real-world scenes, we incorporate contrastive training to improve the generalization. Experimental results indicate superior performance in rain streak depiction compared to existing methods. Practical evaluation confirms its effectiveness in object detection and semantic segmentation tasks. Code is available at https://github.com/Jzy2017/DRNet.

In end-to-end image fusion models, the loss function significantly impacts performance. However, most loss functions treat salient and background regions in source images equally, failing to distinguish complementary areas in multimodal images. This limits the model’s ability to effectively integrate information from these regions. Therefore, we propose difference correlation-driven fusion mechanism of infrared and visible images, which called DCFusion. Specifically, the model utilizes a dual-branch interactive network that dynamically fuses cross-modal multi-scale complementary information through element-wise multiplication, effectively integrating region-specific information. We introduce a two-stage method for generating salient target masks that adaptively focus on high-contrast regions in infrared images by analyzing pixel contrasts in local areas. Furthermore, we utilize the salient target masks to create heterogeneous images and design the $L_{SCD}$ loss function to minimize the information gap between the heterogeneous images and the fused image, thereby enhancing the model’s interpretability. Experiments on the RoadScene and TNO datasets show that DCFusion surpasses with existing representativity fusion approaches, achieving state-of-the-art performance in both subjective visual and objective evaluations. Our code will be publicly available at https://github.com/MinLila/DCFusion.