IET Computer Vision最新文献

Camouflaged object detection (COD) aims to segment objects embedded in the environment from the background. Most existing methods are easily affected by background interference in cluttered environments and cannot accurately locate camouflage areas, resulting in over-segmentation or incomplete segmentation structures. To effectively improve the performance of COD, we propose a spatial perceptual activation network (SPANet). SPANet extracts the spatial positional relationship between each object in the scene by activating spatial perception and uses it as global information to guide segmentation. It mainly consists of three modules: perceptual activation module (PAM), feature inference module (FIM), and interaction recovery module (IRM). Specifically, the authors design a PAM to model the positional relationship between the camouflaged object and the surrounding environment to obtain semantic correlation information. Then, a FIM that can effectively combine correlation information to suppress background interference and re-encode to generate multi-scale features is proposed. In addition, to further fuse multi-scale features, an IRM to mine the complementary information and differences between features at different scales is designed. Extensive experimental results on four widely used benchmark datasets (i.e. CAMO, CHAMELEON, COD10K, and NC4K) show that the authors’ method outperforms 13 state-of-the-art methods.

Using a deep learning method to classify a large amount of labelled remote sensing scene data produces good performance. However, it is challenging for deep learning based methods to generalise to classification tasks with limited data. Few-shot learning allows neural networks to classify unseen categories when confronted with a handful of labelled data. Currently, episodic tasks based on meta-learning can effectively complete few-shot classification, and training an encoder that can conduct representation learning has become an important component of few-shot learning. An end-to-end few-shot remote sensing scene classification model based on ProtoNet and self-supervised learning is proposed. The authors design the Pre-prototype for a more discrete feature space and better integration with self-supervised learning, and also propose the ProtoMixer for higher quality prototypes with a global receptive field. The authors’ method outperforms the existing state-of-the-art self-supervised based methods on three widely used benchmark datasets: UC-Merced, NWPU-RESISC45, and AID. Compare with previous state-of-the-art performance. For the one-shot setting, this method improves by 1.21%, 2.36%, and 0.84% in AID, UC-Merced, and NWPU-RESISC45, respectively. For the five-shot setting, this method surpasses by 0.85%, 2.79%, and 0.74% in the AID, UC-Merced, and NWPU-RESISC45, respectively.

Pedestrian re-identification (re-ID) is an important research direction in computer vision, with extensive applications in pattern recognition and monitoring systems. Due to uneven data distribution, and the need to solve clustering standards and similarity evaluation problems, the performance of unsupervised methods is limited. To address these issues, an improved unsupervised re-ID method, called Enhanced Feature Representation and Robust Clustering (EFRRC), which combines EFRRC is proposed. First, a relation network that considers the relations between each part of the pedestrian's body and other parts is introduced, thereby obtaining more discriminative feature representations. The network makes the feature at the single-part level also contain partial information of other body parts, making it more discriminative. A global contrastive pooling (GCP) module is introduced to obtain the global features of the image. Second, a dispersion-based clustering method, which can effectively evaluate the quality of clustering and discover potential patterns in the data is designed. This approach considers a wider context of sample-level pairwise relationships for robust cluster affinity assessment. It effectively addresses challenges posed by imbalanced data distributions in complex situations. The above structures are connected through a clustering contrastive learning framework, which not only improves the discriminative power of features and the accuracy of clustering, but also solves the problem of inconsistent clustering updates. Experimental results on three public datasets demonstrate the superiority of our method over existing unsupervised re-ID methods.

Diabetic retinopathy (DR) is a severe ophthalmic condition that can lead to blindness if not diagnosed and provided timely treatment. Hence, the development of efficient automated DR grading systems is crucial for early screening and treatment. Although progress has been made in DR detection using deep learning techniques, these methods still face challenges in handling the complexity of DR lesion characteristics and the nuances in grading criteria. Moreover, the performance of these algorithms is hampered by the scarcity of large-scale, high-quality annotated data. An innovative semi-supervised fundus image DR grading framework is proposed, employing a saliency estimation map to bolster the model's perception of fundus structures, thereby improving the differentiation between lesions and healthy regions. By integrating semi-supervised and contrastive learning, the model's ability to recognise inter-class and intra-class variations in DR grading is enhanced, allowing for precise discrimination of various lesion features. Experiments conducted on publicly available DR grading datasets, such as EyePACS and Messidor, have validated the effectiveness of our proposed method. Specifically, our approach outperforms the state of the art on the kappa metric by 0.8% on the full EyePACS dataset and by 3.2% on a 10% subset of EyePACS, demonstrating its superiority over previous methodologies. The authors’ code is publicly available at https://github.com/500ZhangJC/SCL-SEM-framework-for-DR-Grading.

The authors study the problem of reconstructing detailed 3D human surfaces in various poses and clothing from images. The parametric human body allows accurate 3D clothed human reconstruction. However, the offset of large and loose clothing from the inferred parametric body mesh confines the generalisation of the existing parametric body-based methods. A distinctive method that simultaneously generalises well to unseen poses and unseen clothing is proposed. The authors first discover the unbalanced nature of existing implicit function-based methods. To address this issue, the authors propose to synthesise the balanced training samples with a new dependency coefficient in training. The dependency coefficient can tell the network whether the prior from the parametric body model is reliable. The authors then design a novel positional embedding-based attenuation strategy to incorporate the dependency coefficient into the implicit function (IF) network. Comprehensive experiments are conducted on the CAPE dataset to study the effectiveness of the authors’ approach. The proposed method significantly surpasses state-of-the-art approaches and generalises well on unseen poses and clothing. As an illustrative example, the proposed method improves the Chamfer Distance Error and Normal Error by 38.2% and 57.6%.

With the development of automatic driving and path planning technology, predicting the moving trajectory of pedestrians in dynamic scenes has become one of key and urgent technical problems. However, most of the existing techniques regard all pedestrians in the scene as equally important influence on the predicted pedestrian's trajectory, and the existing methods which use sequence-based time-series generative models to obtain the predicted trajectories, do not allow for parallel computation, it will introduce a significant computational overhead. A new social trajectory prediction network, Social-ATPGNN which integrates both temporal information and spatial one based on ATPGNN is proposed. In space domain, the pedestrians in the predicted scene are formed into an undirected and non fully connected graph, which solves the problem of homogenisation of pedestrian relationships, then, the spatial interaction between pedestrians is encoded to improve the accuracy of modelling pedestrian social consciousness. After acquiring high-level spatial data, the method uses Temporal Convolutional Network which could perform parallel calculations to capture the correlation of time series of pedestrian trajectories. Through a large number of experiments, the proposed model shows the superiority over the latest models on various pedestrian trajectory datasets.

Image captioning aims to automatically generate a natural language description of a given image, and most state-of-the-art models have adopted an encoder–decoder transformer framework. Such transformer structures, however, show two main limitations in the task of image captioning. Firstly, the traditional transformer obtains high-level fusion features to decode while ignoring other-level features, resulting in losses of image content. Secondly, the transformer is weak in modelling the natural order characteristics of language. To address theseissues, the authors propose a HIerarchical and Sequential Transformer (HIST) structure, which forces each layer of the encoder and decoder to focus on features of different granularities, and strengthen the sequentially semantic information. Specifically, to capture the details of different levels of features in the image, the authors combine the visual features of multiple regions and divide them into multiple levels differently. In addition, to enhance the sequential information, the sequential enhancement module in each decoder layer block extracts different levels of features for sequentially semantic extraction and expression. Extensive experiments on the public datasets MS-COCO and Flickr30k have demonstrated the effectiveness of our proposed method, and show that the authors’ method outperforms most of previous state of the arts.

As the proliferation of video content continues, and many video archives lack suitable metadata, therefore, video retrieval, particularly through example-based search, has become increasingly crucial. Existing metadata often fails to meet the needs of specific types of searches, especially when videos contain elements from different modalities, such as visual and audio. Consequently, developing video retrieval methods that can handle multi-modal content is essential. An innovative Multi-modal Video Search by Examples (MVSE) framework is introduced, employing state-of-the-art techniques in its various components. In designing MVSE, the authors focused on accuracy, efficiency, interactivity, and extensibility, with key components including advanced data processing and a user-friendly interface aimed at enhancing search effectiveness and user experience. Furthermore, the framework was comprehensively evaluated, assessing individual components, data quality issues, and overall retrieval performance using high-quality and low-quality BBC archive videos. The evaluation reveals that: (1) multi-modal search yields better results than single-modal search; (2) the quality of video, both visual and audio, has an impact on the query precision. Compared with image query results, audio quality has a greater impact on the query precision (3) a two-stage search process (i.e. searching by Hamming distance based on hashing, followed by searching by Cosine similarity based on embedding); is effective but increases time overhead; (4) large-scale video retrieval is not only feasible but also expected to emerge shortly.

Human actions are predominantly presented in 2D format in video surveillance scenarios, which hinders the accurate determination of action details not apparent in 2D data. Depth estimation can aid human action recognition tasks, enhancing accuracy with neural networks. However, reliance on images for depth estimation requires extensive computational resources and cannot utilise the connectivity between human body structures. Besides, the depth information may not accurately reflect actual depth ranges, necessitating improved reliability. Therefore, a 2D human skeleton action recognition method with spatial constraints (2D-SCHAR) is introduced. 2D-SCHAR employs graph convolution networks to process graph-structured human action skeleton data comprising three parts: depth estimation, spatial transformation, and action recognition. The initial two components, which infer 3D information from 2D human skeleton actions and generate spatial transformation parameters to correct abnormal deviations in action data, support the latter in the model to enhance the accuracy of action recognition. The model is designed in an end-to-end, multitasking manner, allowing parameter sharing among these three components to boost performance. The experimental results validate the model's effectiveness and superiority in human skeleton action recognition.

Siamese networks excel at comparing two images, serving as an effective class verification technique for a single-per-class reference image. However, when multiple reference images are present, Siamese verification necessitates multiple comparisons and aggregation, often unpractical at inference. The Centre-Loss approach, proposed in this research, solves a class verification task more efficiently, using a single forward-pass during inference, than sample-to-sample approaches. Optimising a Centre-Loss function learns class centres and minimises intra-class distances in latent space. The authors compared verification accuracy using Centre-Loss against aggregated Siamese when other hyperparameters (such as neural network backbone and distance type) are the same. Experiments were performed to contrast the ubiquitous Euclidean against other distance types to discover the optimum Centre-Loss layer, its size, and Centre-Loss weight. In optimal architecture, the Centre-Loss layer is connected to the penultimate layer, calculates Euclidean distance, and its size depends on distance type. The Centre-Loss method was validated on the Self-Checkout products and Fruits 360 image datasets. Centre-Loss comparable accuracy and lesser complexity make it a preferred approach over sample-to-sample for the class verification task, when the number of reference image per class is high and inference speed is a factor, such as in self-checkouts.