Signal Processing-Image Communication最新文献

Interframe EZBC/JP2K has been shown to be an effective fine-grain scalable video coding system. However, its Lagrange multiplier values for motion estimation of multiple temporal levels are not specified, and must be specified by the user in the config file in order to run the program. In this paper, we investigate how to select these Lagrange parameters for optimized performance. By designing an iterative mechanism, we make it possible for the encoder to adaptively select Lagrange multipliers based on the feedback of Y-PSNR closed GOP performance. Experimental results regarding both classic test video clips and their concatenations are obtained and discussed. We also present a new analytical model for optimized Lagrange multiplier selection in terms of target Y-PSNR.

Although numerous methods have been proposed to remove blocking artifacts in JPEG-compressed images, one important issue not well addressed so far is the construction of a unified model that requires no prior knowledge of the JPEG encoding parameters to operate effectively on different compression-level images (grayscale/color) while occupying relatively small storage space to save and run. To address this issue, in this paper, we present a unified JPEG compression artifact reduction model called DSPW-Net, which employs (1) the deep steerable pyramid wavelet transform network for Y-channel restoration, and (2) the classic U-Net architecture for CbCr-channel restoration. To enable our model to work effectively on images with a wide range of compression levels, the quality factor (QF) related features extracted by the convolutional layers in the QF-estimation network are incorporated in the two restoration branches. Meanwhile, recursive blocks with shared parameters are utilized to drastically reduce model parameters and shared-source residual learning is employed to avoid the gradient vanishing/explosion problem in training. Extensive quantitative and qualitative results tested on various benchmark datasets demonstrate the effectiveness of our model as compared with other state-of-the-art deblocking methods.

Augmented reality applications are beginning to change the way sports are broadcast, providing richer experiences and valuable insights to fans. The first step of augmented reality systems is camera calibration, possibly based on detecting the line markings of the playing field. Most existing proposals for line detection rely on edge detection and Hough transform, but radial distortion and extraneous edges cause inaccurate or spurious detections of line markings. We propose a novel strategy to automatically and accurately segment and classify line markings. First, line points are segmented thanks to a stochastic watershed transform that is robust to radial distortions, since it makes no assumptions about line straightness, and is unaffected by the presence of players or the ball. The line points are then linked to primitive structures (straight lines and ellipses) thanks to a very efficient procedure that makes no assumptions about the number of primitives that appear in each image. The strategy has been tested on a new and public database composed by 60 annotated images from matches in five stadiums. The results obtained have proven that the proposed strategy is more robust and accurate than existing approaches, achieving successful line mark detection even under challenging conditions.

The explosive growth of video data constitutes a series of new challenges in computer vision, and the function of video summarization (VS) is becoming more and more prominent. Recent works have shown the effectiveness of sparse dictionary selection (SDS) based VS, which selects a representative frame set to sufficiently reconstruct a given video. Existing SDS based VS methods use conventional handcrafted features or single-scale deep features, which could diminish their summarization performance due to the underutilization of frame feature representation. Deep learning techniques based on convolutional neural networks (CNNs) exhibit powerful capabilities among various vision tasks, as the CNN provides excellent feature representation. Therefore, in this paper, a multi-scale deep feature fusion based sparse dictionary selection (MSDFF-SDS) is proposed for VS. Specifically, multi-scale features include the directly extracted features from the last fully connected layer and the global average pooling (GAP) processed features from intermediate layers, then VS is formulated as a problem of minimizing the reconstruction error using the multi-scale deep feature fusion. In our formulation, the contribution of each scale of features can be adjusted by a balance parameter, and the row-sparsity consistency of the simultaneous reconstruction coefficient is used to select as few keyframes as possible. The resulting MSDFF-SDS model is solved by using an efficient greedy pursuit algorithm. Experimental results on two benchmark datasets demonstrate that the proposed MSDFF-SDS improves the F-score of keyframe based summarization more than 3% compared with the existing SDS methods, and performs better than most deep-learning methods for skimming based summarization.

Just noticeable distortion (JND), reflecting the perceptual redundancy directly, has been widely used in image and video compression. However, the human visual system (HVS) is extremely complex and the visual signal processing has not been fully understood, which result in existing JND models are not accurate enough and the bitrate saving of JND-based perceptual compression schemes is limited. This paper presents a novel pixel-based JND model for videos and a JND-based perceptual quantization scheme for HEVC codecs. In particular, positive and negative perception effects of the inter-frame difference and the motion information are analyzed and measured with an information-theoretic approach. Then, a surprise-based JND model is developed for perceptual video coding (PVC). In our PVC scheme, the frame-level perceptual quantization parameter (QP) is derived on the premise that the coding distortion is infinitely close to the estimated JND threshold. On the basis of the frame-level perceptual QP, we determine the perceptual QP for each coding unit through a perceptual adjustment function to achieve better perceptual quality. Experimental results indicate that the proposed JND model outperforms existing models significantly, the proposed perceptual quantization scheme improves video compression efficiency with better perceptual quality and lower coding complexity.

Image steganography aims to achieve covert communication between two partners utilizing stego images generated by hiding secret images within cover images. Existing deep image steganography methods have been rapidly developed in this area. Such methods, however, usually generate the stego images and reveal the secret images using one-process networks, lacking sufficient refinement in these methods. Thus, the security and quality of stego and revealed secret images still have much room for promotion, especially for large-capacity image steganography. This paper proposes Joint Adjustment Image Steganography Networks (JAIS-Nets), containing a series of coarse-to-fine iterative adjustment processes, for image steganography. Our JAIS-Nets first proposes Cross-Process Contrastive Refinement (CPCR) adjustment method, using the cross-process contrastive information from cover-stego and secret-revealed secret image pairs, to iteratively refine the generated stego and revealed secret images, respectively. In addition, our JAIS-Nets further proposes Cross-Process Multi-Scale (CPMS) adjustment method, using the cross-process multi-scale information from different scales cover-stego and secret-revealed secret image pairs, to directly adjust and enhance the intermediate representations of the proposed JAIS-Nets. Integrating the proposed CPCR with CPMS methods, the proposed JAIS-Nets can jointly adjust the quality of the stego and revealed secret images at both the learning process and image scale levels. Extensive experiments demonstrate that our JAIS-Nets can achieve state-of-the-art performances on the security and quality of the stego and revealed secret images on both the regular and large capacity image steganography.

This paper presents a self-embedding reversible color-to-grayscale conversion (RCGC) algorithm that makes good use of deep learning, vector quantization, and halftoning techniques to achieve its goals. By decoupling the luminance information of a pixel from its chrominance information, it explicitly controls the luminance error of both the conversion outputs and their corresponding reconstructed color images. It can also alleviate the burden of the deep learning network used to restore the embedded chrominance information during the reconstruction of the color image. Luminance-guided chrominance quantization and checkerboard-based halftoning are introduced in the paper to encode the chrominance information to be embedded while reference-guided inverse halftoning is proposed to restore the color image. Simulation results verify that its performance is remarkably superior to conventional state-of-art RCGC algorithms in various measures. In the aspect of authentication, embedding the watermark and chrominance information is realized with context-based pixel-wise encryption and a key-based watermark bit positioning mechanism, which makes us possible to locate tampered regions and prevent unauthorized use of the chrominance information.

Next-generation multimedia networks are expected to provide systems and applications with top Quality of Experience (QoE) to users. To this end, robust quality evaluation metrics are critical. Unfortunately, most current research focuses only on modeling and evaluating mainly distortions across the pipeline of multimedia networks. While distortions are important, it is also as important to consider the effects of enhancement and other manipulations of multimedia content, especially images and videos. In contrast to most existing works dedicated to evaluating image/video quality in its traditional context, very few research efforts have been devoted to Image Quality Enhancement Assessment (IQEA) and more specifically, Contrast Enhancement Evaluation (CEE). Our contribution fills this gap by proposing a pairwise ranking scheme for estimating and evaluating the perceptual quality of image contrast change (contrast enhancement and/or contrast-distorted images) process. We propose a novel Deep Learning-based Blind Quality pairwise Ranking scheme for Contrast-Changed (Deep-BQRCC) images. This method provides an automatic pairwise ranking of a set of contrast-changed images. The proposed framework is based on using a pair of Convolutional Neural Networks (CNN) together with a saliency-based attention model and a color-difference visual map. Extensive experiments were conducted to validate the effectiveness of the proposed workflow through an ablation analysis. Different combinations of CNN models and pooling strategies were analyzed. The proposed Deep-BQRCC approach was evaluated over three dedicated publicly available datasets. The experimental results showed an increase in performance within a range of $3\text{–}10$% compared to state-of-the-art IQEA measures.

Background information is an important aspect of pre-processing for advanced applications in computer vision. The literature has made rapid progress in background initialization. However, background initialization still suffers from high-dynamic complex scenes, such as illumination change, background motion, or camera jitter. Therefore, this study presents a novel Co-occurrence Spatial–Temporal (CoST) model for background initialization in high-dynamic complex scenes. CoST achieves a spatial–temporal model through a co-occurrence pixel-block structure. The proposed approach extracts the spatial–temporal information of pixels to self-adaptively generate the background without the influence of high-dynamic complex scenes. The efficiency of CoST is verified through experimental results compared with state-of-the-art algorithms. The source code of CoST is available online at: https://github.com/HelloMrDeng/CoST.git.

Image Retargeting (IR) technology is proposed to flexibly display images on various display devices while protecting the important content of the images undistorted. IR methods mainly use Salient Object Detection (SOD) to obtain important content, however, most existing SOD methods treat multiple salient objects with the same saliency degrees, which makes IR methods assign the same retargeting ratios for different objects and leads to producing information-loss retargeted results. Multi-operator IR demonstrates better generalization than single operator by using multiple operators to find the optimal sequence of operators. Meanwhile, the tremendous processing time limits its practical use. To address these problems, we propose a multi-operator IR method based on Salient Object Ranking (SOR) and Similarity Evaluation Metric (SORSEM-IR), which includes two stages: importance map generation and multi-operator IR. In the first stage, a SOR module with Context-aware Semantic Refinement (SORCSR) is proposed, which extracts the salient instances and infers their saliency ranks with a context-aware semantic refinement module, then the SOR map, face map, and gradient map are fused as the importance map. In the second stage, to speed up multiple operations, a similarity evaluation metric is proposed to measure the similarity between the original image and the seam-removal image by Seam Carving (SC) operation, and switch SC to uniform scaling to meet the aspect ratio when distortion caused by SC arrives at a certain extent. Experimental results show that the SORCSR network achieves state-of-the-art performance on the ASSR dataset subjectively and objectively, and the SORSEM-IR guided by SORCSR can not only protect the salient objects with minimum deformation but also meet human aesthetic perception.