Computers & Graphics-Uk最新文献

Recently, graph-based and Transformer-based deep learning have demonstrated excellent performances on various point cloud tasks. Most of the existing graph-based methods rely on static graph, which take a fixed input to establish graph relations. Moreover, many graph-based methods apply maximizing and averaging to aggregate neighboring features, so that only a single neighboring point affects the feature of centroid or different neighboring points own the same influence on the centroid’s feature, which ignoring the correlation and difference between points. Most Transformer-based approaches extract point cloud features based on global attention and lack the feature learning on local neighbors. To solve the above issues of graph-based and Transformer-based models, we propose a new feature extraction block named Graph Transformer and construct a 3D point cloud learning network called GTNet to learn features of point clouds on local and global patterns. Graph Transformer integrates the advantages of graph-based and Transformer-based methods, and consists of Local Transformer that use intra-domain cross-attention and Global Transformer that use global self-attention. Finally, we use GTNet for shape classification, part segmentation and semantic segmentation tasks in this paper. The experimental results show that our model achieves good learning and prediction ability on most tasks. The source code and pre-trained model of GTNet will be released on https://github.com/NWUzhouwei/GTNet.

Skin flaps are common procedures used by surgeons to cover an excised area during the reconstruction of a defect. It is often a challenging task for a surgeon to come up with the most optimal design for a patient. In this paper, we set up a simulation system based on the finite element method for one of the most common flap types — the rhomboid flap. Instead of using the standard 2D planar patch, we constructed a 3D patch with multiple layers. This allowed us to investigate the impact of different undermining areas and depths. We compared the suture forces for each case and identified vertices with the largest suture force. The shape of the final suture line is also visualized for each case, which is an important clue when deciding on the most optimal skin flap orientation according to medical textbooks. We found that under the optimal undermining setup, the maximum suture force is around 0.7 N for top of the undermined layer and 1.0 N for bottom of the undermined layer. When measuring difference in final suture line shape, the maximum normalized Hausdorff distance is 0.099, which suggests that different undermining region can have significant impact on the shape of the suture line, especially in the tail region. After analyzing the suture force plots, we provided recommendations on the most optimal undermining region for rhomboid flaps.

Computer-aided design (CAD) tools are utilized in the manufacturing industry for modeling everything from cups to spacecraft. These programs are complex to use and typically require years of training and experience to master. Structured and well-constrained 2D sketches and 3D constructions are crucial components of CAD modeling. A well-executed CAD model can be seamlessly integrated into the manufacturing process, thereby enhancing production efficiency. Deep generative models of 3D shapes and 3D object reconstruction models have garnered significant research interest. However, most of these models produce discrete forms of 3D objects that are not editable. Moreover, the few models based on CAD operations often have substantial input restrictions. In this work, we fine-tuned pre-trained models to create OpenECAD models (0.55B, 0.89B, 2.4B and 3.1B), leveraging the visual, logical, coding, and general capabilities of visual language models. OpenECAD models can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands, ensuring that the designs are editable. These outputs can be directly used with existing CAD tools’ APIs to generate project files. To train our network, we created a series of OpenECAD datasets. These datasets are derived from existing public CAD datasets, adjusted and augmented to meet the specific requirements of vision language model (VLM) training. Additionally, we have introduced an approach that utilizes dependency relationships to define and generate sketches, further enriching the content and functionality of the datasets.

In recent years, remarkable ability has been demonstrated by the Transformer model in capturing remote dependencies and improving point cloud segmentation performance. However, localized regions separated from conventional sampling architectures have resulted in the destruction of structural information of instances and a lack of exploration of potential geometric relationships between localized regions. To address this issue, a Local Spatial Latent Geometric Relation Learning Network (LSGRNet) is proposed in this paper, with the geometric properties of point clouds serving as a reference. Specifically, spatial transformation and gradient computation are performed on the local point cloud to uncover potential geometric relationships within the local neighborhood. Furthermore, a local relationship aggregator based on semantic and geometric relationships is constructed to enable the interaction of spatial geometric structure and information within the local neighborhood. Simultaneously, boundary interaction feature learning module is employed to learn the boundary information of the point cloud, aiming to better describe the local structure. The experimental results indicate that excellent segmentation performance is exhibited by the proposed LSGRNet in benchmark tests on the indoor datasets S3DIS and ScanNetV2, as well as the outdoor datasets SemanticKITTI and Semantic3D.

Convolutional Neural Networks (CNNs) have proven highly effective for demosaicking, transforming raw Color Filter Array (CFA) sensor samples into standard RGB images. Directly applying convolution to the CFA tensor can lead to misinterpretation of the color context, so existing demosaicking networks typically embed the CFA tensor into the Euclidean space before convolution. The most prevalent embedding options are Reordering and Pre-interpolation. However, it remains unclear which option is more advantageous for demosaicking. Moreover, no existing demosaicking network is suitable for conducting a fair comparison. As a result, in practice, the selection of these two embedding options is often based on intuition and heuristic approaches. This paper addresses the non-comparability between the two options and investigates whether pre-interpolation contributes additional knowledge to the demosaicking network. Based on rigorous mathematical derivation, we design pairs of end-to-end fully convolutional evaluation networks, ensuring that the performance difference between each pair of networks can be solely attributed to their differing CFA embedding strategies. Under strictly fair comparison conditions, we measure the performance contrast between the two embedding options across various scenarios. Our comprehensive evaluation reveals that the prior knowledge introduced by pre-interpolation benefits lightweight models. Additionally, pre-interpolation enhances the robustness to imaging artifacts for larger models. Our findings offer practical guidelines for designing imaging software or Image Signal Processors (ISPs) for RGB cameras.

Category-level pose estimation offers the generalization ability to novel objects unseen during training, which has attracted increasing attention in recent years. Despite the advantage, annotating real-world data with pose label is intricate and laborious. Although using synthetic data with free annotations can greatly reduce training costs, the Synthetic-to-Real (Sim2Real) domain gap could result in a sharp performance decline on real-world test. In this paper, we propose Dual-COPE, a novel prior-based category-level object pose estimation method with dual Sim2Real domain adaptation to avoid expensive real pose annotations. First, we propose an estimation network featured with conjoined prior deformation and transformer-based matching to realize high-precision pose prediction. Upon that, an efficient dual Sim2Real domain adaptation module is further designed to reduce the feature distribution discrepancy between synthetic and real-world data both semantically and geometrically, thus maintaining superior performance on real-world test. Moreover, the adaptation module is loosely coupled with estimation network, allowing for easy integration with other methods without any additional inference overhead. Comprehensive experiments show that Dual-COPE outperforms existing unsupervised methods and achieves state-of-the-art precision under supervised settings.

Text-to-3D generation is a challenging but significant task and has gained widespread attention. Its capability to rapidly generate 3D digital assets holds huge potential application value in fields such as film, video games, and virtual reality. However, current methods often face several drawbacks, including long generation times, difficulties with the multi-face Janus problem, and issues like chaotic topology and redundant structures during mesh extraction. Additionally, the lack of control over the generated results limits their utility in downstream applications. To address these problems, we propose a novel text-to-3D framework capable of generating meshes with high fidelity and controllability. Our approach can efficiently produce meshes and textures that match the text description and the desired level of detail (LOD) by specifying input text and LOD preferences. This framework consists of two stages. In the coarse stage, 3D Gaussians are employed to accelerate generation speed, and weighted positive and negative prompts from various observation perspectives are used to address the multi-face Janus problem in the generated results. In the refinement stage, mesh vertices and faces are iteratively refined to enhance surface quality and output meshes and textures that meet specified LOD requirements. Compared to the state-of-the-art text-to-3D methods, extensive experiments demonstrate that the proposed method performs better in solving the multi-face Janus problem, enabling the rapid generation of 3D meshes with enhanced prompt adherence. Furthermore, the proposed framework can generate meshes with enhanced topology, offering controllable vertices and faces with textures featuring UV adaptation to achieve multi-level-of-detail(LODs) outputs. Specifically, the proposed method can preserve the output’s relevance to input texts during simplification, making it better suited for mesh editing and rendering efficiency. User studies also indicate that our framework receives higher evaluations compared to other methods.

3D face animation has been a critical component of character animation in a wide range of media since the early 90’s. The conventional process for animating a 3D face is usually keyframe-based, which is labor-intensive. Therefore, the film and game industries have started using live-action actors’ performances to animate the faces of 3D characters, the process is also known as performance-driven facial animation. At the core of performance-driven facial animation is facial motion retargeting, which transfers the source facial motions to a target 3D face. However, facial motion retargeting still has many limitations that influence its capability to further assist the facial animation process. Existing motion retargeting frameworks cannot accurately transfer the source motion’s semantic information (i.e., meaning and intensity of the motion), especially when applying the motion to non-human-like or stylized target characters. The retargeting quality relies on the parameterization of the target face, which is time-consuming to build and usually not generalizable across proportionally different faces. In this survey paper, we review the literature relating to 3D facial motion retargeting methods and the relevant topics within this area. We provide a systematic understanding of the essential modules of the retargeting pipeline, a taxonomy of the available approaches under these modules, and a thorough analysis of their advantages and limitations with research directions that could potentially contribute to this area. We also contributed a 3D character categorization matrix, which has been used in this survey and might be useful for future research to evaluate the character compatibility of their retargeting or face parameterization methods.