Computers & Graphics-Uk最新文献

Predicting the path followed by the viewer’s eyes when observing an image (a scanpath) is a challenging problem, particularly due to the inter- and intra-observer variability and the spatio-temporal dependencies of the visual attention process. Most existing approaches have focused on progressively optimizing the prediction of a gaze point given the previous ones. In this work we propose instead a probabilistic approach, which we call tSPM-Net. We build our method to account for observers’ variability by resorting to Bayesian deep learning and a probabilistic approach. Besides, we optimize our model to jointly consider both spatial and temporal dimensions of scanpaths using a novel spatio-temporal loss function based on a combination of Kullback–Leibler divergence and dynamic time warping. Our tSPM-Net yields results that outperform those of current state-of-the-art approaches, and are closer to the human baseline, suggesting that our model is able to generate scanpaths whose behavior closely resembles those of the real ones.

In this paper, we present an algebraic framework that can be used to construct a large class of 3D shapes and structures that can potentially provide unusual material properties. We formalized this framework as a 3D generalization of planar nonwoven textile structures that are used to mimic the woven structures. Our extension is based on the fact that it is straightforward to extend planar nonwoven textile structures into volumetric nonwoven textile structures, which we also call nonwoven volumetric fabrics. This property is essential because such an extension is impossible with planar woven structures. In other words, using this approach, it can be possible to easily produce volumetric structures that mimic the fabric behavior as if they were planar nonwoven textile structures, which is impossible to produce. These volumetric structures also correspond to regular & semiregular frame structures and are capable of representing previously unknown infinite regular polyhedra and flexible wood structures.

Reconstructing High Dynamic Range (HDR) video from alternating exposure Low Dynamic Range (LDR) sequence is an exceptionally challenging task. It not only demands the reliable reconstruction of missing information caused by occlusion or motion without introducing artifacts but also balances the exposure differences between frames to ensure a visually pleasing reconstructed HDR video. Unfortunately, existing methods are typically complex and struggle with unavoidable artifacts and noise, especially when dealing with low-exposed scenes. To tackle this formidable challenge, we propose a two-stage HDR video reconstruction method that employs a global to local alignment strategy. Firstly, we utilize iterative optical flow estimation and hybrid weighting to achieve global alignment, ensuring well-reconstructed in majority of areas. Secondly, the recursive refinement network further addresses locally misaligned areas, reconstructing HDR frames from bottom to top and recursively refining them to yield faithful reconstruction results. Extensive experimental results demonstrate that our method generates the HDR video with fine details and superior visually, surpassing the state-of-the-art method across diverse scenes.

In order to facilitate the robust and precise 3D vessel shape extraction and quantification from in-vivo Magnetic Resonance Imaging (MRI), this paper presents a novel multi-scale Knowledge Transfer Vision Transformer (i.e., KT-ViT) for 3D vessel shape segmentation. First, it uniquely integrates convolutional embeddings with transformer in a U-net architecture, which simultaneously responds to local receptive fields with convolution layers and global contexts with transformer encoders in a multi-scale fashion. Therefore, it intrinsically enriches local vessel feature and simultaneously promotes global connectivity and continuity for a more accurate and reliable vessel shape segmentation. Furthermore, to enable using relatively low-resolution (LR) images to segment fine scale vessel shapes, a novel knowledge transfer network is designed to explore the inter-dependencies of data and automatically transfer the knowledge gained from high-resolution (HR) data to the low-resolution handling network at multiple levels, including the multi-scale feature levels and the decision level, through an integration of multi-level loss functions. The modeling capability of fine-scale vessel shape data distribution, possessed by the HR image transformer network, can be transferred to the LR image transformer to enhance its knowledge for fine vessel shape segmentation. Extensive experimental results on public image datasets have demonstrated that our method outperforms all other state-of-the-art deep learning methods.

Immersive environments with head mounted displays (HMD) and hand-held controllers, either in Virtual or Augmented Reality (VR/AR), offer new possibilities for the creation of artistic 3D content. Some of them are exploited by mid-air drawing applications: the user’s hand trajectory generates a set of stylized curves or ribbons in space, giving the impression of painting or drawing in 3D. We propose a method to extend this approach to the sketching of surfaces with a VR controller. The idea is to favor shape exploration, offering a tool, where the user creates a surface just by painting ribbons. These ribbons are not constrained to form patch boundaries for example or to completely cover the shape. They can be very sparse, disordered, overlap or not, intersect or not. The shape is computed simultaneously, starting with the first piece of ribbon drawn by the user and continuing to evolve in real-time as long as the user continues sketching. Our method involves minimizing an energy function based on the projections of the ribbon strokes on a proxy surface by taking the controller’s orientations into account. The current implementation considers elevation surfaces. In addition to many examples, we evaluate the time performance of the dynamic shape modeling with respect to an increasing number of input ribbon strokes. Finally, we present images of an artistic creation that combines stylized curve drawings in VR with our surface sketching tool created by a professional artist.

We propose a novel method to design differentiable microstructures. Central to our algorithm is a new representation of the mapping from the parameters to microstructures, formulated as the anisotropic thermal diffusion. A metric field governs the anisotropic diffusion. The metric associated with each point is represented as a 2 × 2 symmetric positive definite matrix that becomes the design variable. To alleviate the difficulties caused by symmetric positive definite constraints, we perform the singular value decomposition of the metric matrix so that the design variable includes a rotation angle and a diagonal matrix. Then, the positive definiteness is converted to requiring the two diagonal entries of the diagonal matrix to be positive, which is easier to deal with. The effectiveness of our algorithm is demonstrated through evaluations and comparisons over various examples.

Text-conditioned image synthesis methods such as DALLE-2, IMAGEN, and Stable Diffusion are gaining strong attention from deep learning and art communities recently. Meanwhile, Image-to-Image (Img2Img) synthesis applications that emerged from the pioneering Neural Style Transfer (NST) approach have swiftly transitioned towards the feed-forward Automatic Style Transfer (AST) methods, due to numerous constraints inherent in the former method, including inconsistent synthesis outcomes and sluggish optimization-based synthesis process. However, NST holds significant potential yet remains relatively underexplored within this research domain. In this paper, we revisited the original NST method and uncovered its potential to attain image quality comparable to the AST synthesis methods across a diverse range of artistic styles. We propose a two-stage Feature-guided Style Transfer (FeaST) which consists (a) pre-stylization step called Sketching to address the poor initialization issue, and (b) Finetuning to guide the synthesis process based on high-frequency (HF) and low-frequency (LF) guidance channels. By addressing the issues of inconsistent synthesis and slow convergence inherent in the original method, FeaST unlocks the full capabilities of NST and significantly enhances its efficiency.

Omnidirectional images, also known as spherical images, offer a significant advantage for the environmental sensing of mobile robots due to their wide field of view. However, previous studies of constructing convolutional neural networks on spherical images have been limited by non-uniform pixel sampling, leading to suboptimal performance in semantic segmentation. To address this issue, a novel pixel segmentation approach is proposed to achieve near-uniform pixel distribution across the entire spherical surface. The corresponding convolution operation for the resulting image is designed as well, which extends the capabilities of spherical CNNs from semantic segmentation to more complex tasks such as instance segmentation. The method is evaluated on the Stanford 2D3DS dataset and shows superior performance compared to conventional spherical CNNs. Furthermore, the method also achieves impressive instance segmentation results on our experimental LiDAR data, demonstrating the general feasibility of our approach for common CNN tasks. The related code and dataset are released in the following link: https://github.com/YoungRainy/UVS-U-Net.

Point cloud registration methods based on Gaussian Mixture Models (GMMs) exhibit high robustness. However, GMM cannot precisely depict point clouds, because the Gaussian distribution is spatially symmetric and local surfaces of point clouds are typically non-symmetric. In this paper, we propose a novel method for rigid point cloud registration, termed coherent point drift with Skewed Distribution (Skewed CPD). Our method employs an asymmetric distribution constructed from the local surface normals and curvature radii. Compared to the Gaussian distribution, this skewed distribution provides a more accurate spatial description of points on local surfaces. Additionally, we integrate an adaptive multiplier to the covariance, which reallocates the weight of the covariance for different components in the probabilistic mixture model. We employ the EM algorithm to address this maximum likelihood estimation (MLE) issue and leverage GPU acceleration. In the M-step, we adopt an unconstrained optimization technique rooted in a Lie group and Lie algebra to attain the optimal transformation. Experimental results indicate that our method outperforms state-of-the-art methods in both accuracy and robustness. Remarkably, even without loop closure detection, the cumulative error of our approach remains minimal.

Facial retargeting is a widely used technique in the game and film industries that replicates the expressions of a source facial model onto a target model. Existing methods for facial retargeting rely on either hand-crafted uniform triangle meshes or sparse points obtained from motion capture(mocap). In this paper, we propose an end-to-end facial retargeting algorithm that copies facial expressions from unordered dense point clouds onto the target model. First, a corresponding building method based on bi-harmonic function is introduced to ensure that the template model and a cluster of point clouds share the same triangle topology. Second, a deformation transferring method is presented to transfer the calculated deformation onto the target model. Several experiments are conducted on the SIAT-3DFE dataset to demonstrate the accuracy and efficiency of our method.