Graphical Models最新文献

The restoration of digital images holds practical significance due to the fact that degradation of digital image data on the internet is common. State-of-the-art image restoration methods usually employ end-to-end trained networks. However, we argue that a network trained with diverse image pairs is not optimal for restoring line drawings which have extensive plain backgrounds. We propose a line-drawing restoration framework which takes a restoration neural network as backbone and processes an input degraded line drawing in two steps. First, a proposed mask-predicting network predicts a line mask which indicates the possible location of foreground and background in the potential original line drawing. Next, we feed the degraded input line drawing together with the predicted line mask into the backbone restoration network. The traditional $L_1$ loss for the backbone restoration network is substituted with a masked Mean Square Error (MSE) loss. We test our framework on two classical image restoration tasks: JPEG restoration and super-resolution, and experiments demonstrate that our framework can achieve better quantitative and visual results in most cases.

Free-form bending belongs to the kinematics-based forming processes and allows the manufacturing of arbitrary 3D-bent components. To obtain the desired part, the tool kinematics is adjusted by comparing the target and obtained bending line. While the target geometry consists of parametric CAD data, the obtained geometry is a surface mesh, making the bending line extraction a challenging task. In this paper the reconstruction of the bending line for free-form bent components is presented. The strategy relies on the extraction of the centroids, for which a ray casting algorithm is developed and compared to an existing Voronoi-based method. Subsequently the obtained points are used to fit a NURBS parametric model of the curve. The algorithm parameters are investigated with a sensitivity analysis, and its performance is evaluated with a defined error metric. Finally, the strategy is validated comparing its results with a Voronoi-based algorithm, and investigating different cross-sections and geometries.

With the support of Virtual Reality (VR) and Augmented Reality (AR) technologies, the 3D virtual eyeglasses try-on application is well on its way to becoming a new trending solution that offers a “try on” option to select the perfect pair of eyeglasses at the comfort of your own home. Reconstructing eyeglasses frames from a single image with traditional depth and image-based methods is extremely difficult due to their unique characteristics such as lack of sufficient texture features, thin elements, and severe self-occlusions. In this paper, we propose the first mesh deformation-based reconstruction framework for recovering high-precision 3D full-frame eyeglasses models from a single RGB image, leveraging prior and domain-specific knowledge. Specifically, based on the construction of a synthetic eyeglasses frame dataset, we first define a class-specific eyeglasses frame template with pre-defined keypoints. Then, given an input eyeglasses frame image with thin structure and few texture features, we design a keypoint detector and refiner to detect predefined keypoints in a coarse-to-fine manner to estimate the camera pose accurately. After that, using differentiable rendering, we propose a novel optimization approach for producing correct geometry by progressively performing free-form deformation (FFD) on the template mesh. We define a series of loss functions to enforce consistency between the rendered result and the corresponding RGB input, utilizing constraints from inherent structure, silhouettes, keypoints, per-pixel shading information, and so on. Experimental results on both the synthetic dataset and real images demonstrate the effectiveness of the proposed algorithm.

Instructional image editing has received a significant surge of attention recently. In this work, we are interested in the challenging problem of instructional image editing within the particular fashion realm, a domain with significant potential demand in both commercial and personal contexts. This specific domain presents heightened challenges owing to the stringent quality requirements. It necessitates not only the creation of vivid details in alignment with instructions, but also the preservation of precise attributes unrelated to the text guidance. Naive extensions of existing image editing methods produce noticeable artifacts. In order to achieve high-fidelity fashion editing, we propose a novel framework, leveraging the generative prior of a pre-trained human generator and performing edit in the latent space. In addition, we introduce a novel CLIP-based loss to better align the generated target with the instruction. Extensive experiments demonstrate that our approach outperforms prior works including GAN-based editing as well as diffusion-based editing by a large margin, showing impressive visual quality.

This paper addresses the problem of animating a person in static images, the core task of which is to infer future poses for the person. Existing approaches predict future poses in the 2D space, suffering from entanglement of pose action and shape. We propose a method that generates actions in the 3D space and then transfers them to the 2D person. We first lift the 2D pose of the person to a 3D skeleton, then propose a 3D action synthesis network predicting future skeletons, and finally devise a self-supervised action transfer network that transfers the actions of 3D skeletons to the 2D person. Actions generated in the 3D space look plausible and vivid. More importantly, self-supervised action transfer allows our method to be trained only on a 3D MoCap dataset while being able to process images in different domains. Experiments on three image datasets validate the effectiveness of our method.

In recent years, deep learning based methods for single image 3D planar recovery have made significant progress, but most of the research has focused on overall plane segmentation performance rather than the accuracy of small scale plane segmentation. In order to solve the problem of feature loss in the feature extraction process of small target object features, a three dimensional planar recovery method based on bilateral transformer was proposed. The two sided network branches capture rich small object target features through different scale sampling, and are used for detecting planar and non-planar regions respectively. In addition, the loss of variational information is used to share the parameters of the bilateral network, which achieves the output consistency of the bilateral network and alleviates the problem of feature loss of small target objects. The method is verified on Scannet and Nyu V2 datasets, and a variety of evaluation indexes are superior to the current popular algorithms, proving the effectiveness of the method in three dimensional planar recovery.

A multi-level pattern is proposed for the unconstrained two-dimensional cutting problems of rectangular items, and an exact generation algorithm is constructed. The arrangement of rectangular items with the same type in multiple rows and columns is referred to as a 0-level pattern. An n-level pattern is the horizontal or vertical combination of an n-1 level pattern with a pattern whose level will not exceed n-1. The generation algorithm of multi-level pattern is constructed on the base of dynamic programming, and the multi-level patterns with various sizes are generated with increased level order. The normal size is chosen to reduce unnecessary computation in the algorithm. Three sets of benchmark instances and one set of random production instance from the literatures are used for comparison. Comparing to the exact algorithm in the literature, the results in this paper are equivalent, but the computation time is shorter. Comparing to heuristic algorithms in literatures, the results in this paper are better and the computation time is still good enough for practical applications.

The Bonded Discrete Element Method (BDEM) has raised interests in the graphics community in recent years because of its good performance in fracture simulations. However, current explicit BDEM usually needs to work under very small time steps to avoid numerical instability. We propose a new BDEM, namely Rod-BDEM (RBDEM), which uses Cosserat energy and yields integrable forces and torques. We further derive a novel Cosserat rod discretization method to effectively represent the three-dimensional topological connections between discrete elements. Then, a complete implicit BDEM system integrating the appropriate fracture model and contact model is constructed using the implicit Euler integration scheme. Our method allows high Young’s modulus and larger time steps in elastic deformation, breaking, cracking, and impacting, achieving up to 8 times speed up of the total simulation.

We introduce deformable interaction analogy (DINA) as a means to generate close interactions between two 3D objects. Given a single demo interaction between an anchor object (e.g. a hand) and a source object (e.g. a mug grasped by the hand), our goal is to generate many analogous 3D interactions between the same anchor object and various new target objects (e.g. a toy airplane), where the anchor object is allowed to be rigid or deformable. To this end, we optimize the pose or shape of the anchor object to adapt it to a new target object to mimic the demo. To facilitate the optimization, we advocate using interaction interface (ITF), defined by a set of points sampled on the anchor object, as a descriptive and robust interaction representation that is amenable to non-rigid deformation. We model similarity between interactions using ITF, while for interaction analogy, we transform the ITF, either rigidly or non-rigidly, to guide the feature matching to the reposing and deformation of the anchor object. Qualitative and quantitative experiments show that our ITF-guided deformable interaction analogy works surprisingly well even with simple distance features compared to variants of state-of-the-art methods that utilize more sophisticated interaction representations and feature learning from large datasets.