Signal Processing-Image Communication最新文献

Object cosegmentation aims to obtain common objects from multiple images or videos, which performs by employing handcraft features to evaluate region similarity or learning higher semantic information via deep learning. However, the former based on handcraft features is sensitive to illumination, appearance changes and clutter background to the domain gap. The latter based on deep learning needs the groundtruth of object segmentation to train the co-attention model to spotlight the common object regions in different domain. This paper proposes an adversarial domain adaption-based video object cosegmentation method without any pixel-wise supervision. Intuitively, high-level semantic similarity are beneficial for common object recognition. However, there are inconsistency distributions of different video sources, i.e., domain gap. We propose an adversarial learning method to align feature distributions of different videos, which aims to maintain the feature similarity of common objects to overcome the dataset bias. Hence, a feature encoder via Siamese network is constructed to fool a discriminative network to obtain domain adapted feature mapping. To further assist the feature embedding of common objects, we define a latent task for label generation to train a classifying network, which could make full use of high-level semantic information. Experimental results on several video cosegmentation datasets suggest that domain adaption based on adversarial learning could significantly improve the common semantic feature exaction.

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

The task of binarization of historical document images has been in the forefront of image processing research, during the digital transition of libraries. The process of storing and transcribing valuable historical printed or handwritten material can salvage world cultural heritage and make it available online without physical attendance. The task of binarization can be viewed as a pre-processing step that attempts to separate the printed/handwritten characters in the image from possible noise and stains, which will assist in the Optical Character Recognition (OCR) process. Many approaches have been proposed before, including deep learning based approaches. In this article, we propose a U-Net style deep learning architecture that incorporates many other developments of deep learning, including residual connections, multi-resolution connections, visual attention blocks and dilated convolution blocks for upsampling. The novelties in the proposed DMVAnet lie in the use of these elements in combination in a novel U-Net style architecture and the application of DMVAnet in image binarization for the first time. In addition, the proposed DMVAnet is a very computationally lightweight network that performs very close or even better than the state-of-the-art approaches with a fraction of the network size and parameters. Finally, it can be used on platforms with restricted processing power and system resources, such as mobile devices and through scaling can result in inference times that allow for real-time applications.

Anomaly detection in multimedia datasets is a widely studied area. Yet, the concept drift challenge in data has been ignored or poorly handled by the majority of the anomaly detection frameworks. The state-of-the-art approaches assume that the data distribution at training and deployment time will be the same. However, due to various real-life environmental factors, the data may encounter drift in its distribution or can drift from one class to another in the late future. Thus, a one-time trained model might not perform adequately. In this paper, we systematically investigate the effect of concept drift on various detection models and propose a modified Adaptive Gaussian Mixture Model (AGMM) based framework for anomaly detection in multimedia data. In contrast to the baseline AGMM, the proposed extension of AGMM remembers the past for a longer period in order to handle the drift better. Extensive experimental analysis shows that the proposed model better handles the drift in data as compared with the baseline AGMM. Further, to facilitate research and comparison with the proposed framework, we contribute three multimedia datasets constituting faces as samples. The face samples of individuals correspond to the age difference of more than ten years to incorporate a longer temporal context.

Video quality assessment (VQA) remains an important and challenging problem that affects many applications at the widest scales. Recent advances in mobile devices and cloud computing techniques have made it possible to capture, process, and share high resolution, high frame rate (HFR) videos across the Internet nearly instantaneously. Being able to monitor and control the quality of these streamed videos can enable the delivery of more enjoyable content and perceptually optimized rate control. Accordingly, there is a pressing need to develop VQA models that can be deployed at enormous scales. While some recent effects have been applied to full-reference (FR) analysis of variable frame rate and HFR video quality, the development of no-reference (NR) VQA algorithms targeting frame rate variations has been little studied. Here, we propose a first-of-a-kind blind VQA model for evaluating HFR videos, which we dub the Framerate-Aware Video Evaluator w/o Reference (FAVER). FAVER uses extended models of spatial natural scene statistics that encompass space–time wavelet-decomposed video signals, and leverages the advantages of the deep neural network to provide motion perception, to conduct efficient frame rate sensitive quality prediction. Our extensive experiments on several HFR video quality datasets show that FAVER outperforms other blind VQA algorithms at a reasonable computational cost. To facilitate reproducible research and public evaluation, an implementation of FAVER is being made freely available online: https://github.com/uniqzheng/HFR-BVQA.

Using computer vision techniques such as stereo vision systems for sea state measurement or for offshore structures monitoring can improve the measurement fidelity and accuracy with no significant additional cost. In this paper, two experiments (in-lab/open-sea) are conducted to study the performance of stereo vision system to measure the water wave surface elevation and rigid body heaving motion. For the in-lab experiment, regular water waves are generated in a wave tank for different frequencies and wave heights, where the water surface is scanned by the stereo vision camera installed on the top of the tank. Surface elevation inferred by the stereo vision is verified by an installed stationary side camera that records the water surface through the tank transparent side window, water surface elevation measured by the side camera recordings is extracted using edge detection algorithm. During the in-lab experiment a heaving buoy is installed to test the performance of Visual Simultaneous Localization and Mapping (VSLAM) algorithm to monitor the buoy heave motion. The VSLAM algorithm fuses a buoy onboard stereo vision recordings with an embedded Inertial Measurement Unit (IMU) to estimate the 6-DOF of a rigid body. The Buoy motion VSLAM measurements are verified by a KLT tracking algorithm implemented on the video recordings of the stationary side camera. The open-sea experiment is implemented in Lake Somerville, Texas. The stereo vision system is installed to measure the water surface elevation and directional spectrum of the wind generated irregular waves. The open-sea wave measurements by the stereo vision are verified by a Sofar commercial wave buoys deployed in the testing location.

Synthesizing images with fine details from text descriptions is a challenge. The existing single-stage generative adversarial networks (GANs) fuse sentence features into the image generation process through affine transformation, which alleviate the problems of missing details and large computation from stacked networks. However, existing single-stage networks ignore the word features in the text description, resulting in a lack of detail in the generated image. To address this issue, we proposed a text aggregation module (TAM) to fuse sentence features and word features in a text by a simple spatial attention mechanism. Then we built a text connection fusion (TCF) block consisting mainly of gated recurrent unit (GRU) and up-sampled block. It can connect text features used in the up-sampled blocks to improve text utilization. Besides, to further improve the semantic consistency between text and the generated images, we introduce the deep attentional multimodal similarity model (DAMSM) loss, which monitors the similarity between text and improves semantic consistency. Experimental results prove that our method is superior to the state-of-the-art models on the CUB and COCO datasets, regarding both image fidelity and semantic consistency with the text.

Continuous wave indirect Time-of-Flight cameras obtain depth images by emitting a modulated continuous light wave and measuring the delay of the received signal. In this paper we generalize the estimation of the effect of the shot noise when obtaining the phase delay with an arbitrary number of points in the Discrete Fourier Transform, extending and generalizing the analysis done in previous works for the case of four points. For that particular case, we compare our analysis with the state of art. Moreover, we extend the error model using a second order approximation in the error propagation analysis, which provides more accurate estimations according to the Montecarlo simulation experiments. The analysis, based on both analytical and numerical methods, shows that the phase error is, in general, related to the exposure time and weakly to the number of points in the Discrete Fourier Transform. It also depends on the background illumination level, on the amplitude of the received signal, and, when using a three point DFT, on the distance to the objects.

Localization of facial landmarks plays an important role in the measurement of facial metrics applicable for beauty analysis and facial plastic surgery. The first step in detecting facial landmarks is to estimate the face bounding box. Clinical images of patients' faces usually show intensity non-uniformity. These conditions cause common face detection algorithms do not perform well in face detection under varying illumination. To solve this problem, a modified fuzzy c-means (MFCM) algorithm is used under varying illumination modeling. The cascade regression method (CRM) has an appropriate performance in face alignment. This algorithm has two main drawbacks. (1) In the training phase, increasing the real data without considering normal data can lead to over-fitting. To solve this problem, a weighted CRM (WCRM) is presented. (2) In the test phase, using a mean shape causes the initial shape to be either near to or far from the face shape. To overcome this problem, a Procrustes-based analysis is presented. One of the most important steps in facial landmark localization is feature extraction. In this study, to increase detection accuracy of the cephalometric landmarks, local phase quantization (LPQ) is used for feature extraction in all three channels of RGB color space. Finally, the proposed algorithm is used to measure facial anthropometric metrics. Experimental results show that the proposed algorithm has a better performance in facial landmark localization than other compared algorithms.

Although high resolution (HR) depth images are required in many applications such as virtual reality and autonomous navigation, their resolution and quality generated by consumer depth cameras fall short of the requirements. Existing depth upsampling methods focus on extracting multiscale features of HR color image to guide low resolution (LR) depth upsampling, thus causing blurry and inaccurate edges in depth. In this paper, we propose a depth super-resolution (SR) network guided by blurry depth and clear intensity edges, called DSRNet. DSRNet differentiates effective edges from a number of HR edges with the guidance of blurry depth and clear intensity edges. First, we perform global residual estimation based on an encoder–decoder architecture to extract edge structure from HR color image for depth SR. Then, we distinguish effective edges from HR edges in the decoder side with the guidance of LR depth upsampling. To maintain edges for depth SR, we use intensity edge guidance that extracts clear intensity edges from HR image. Finally, we use residual loss to generate accurate high frequency (HF) residual and reconstruct HR depth maps. Experimental results show that DSRNet successfully reconstructs depth edges in SR results as well as outperforms the state-of-the-art methods in terms of visual quality and quantitative measurements.¹