Computer Graphics Forum最新文献

With the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real-time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.

In deep functional maps, the regularizer computing the functional map is especially crucial for ensuring the global consistency of the computed pointwise map. As the regularizers integrated into deep learning should be differentiable, it is not trivial to incorporate informative axiomatic structural constraints into the deep functional map, such as the orientation-preserving term. Although commonly used regularizers include the Laplacian-commutativity term and the resolvent Laplacian commutativity term, these are limited to single-scale analysis for capturing geometric information. To this end, we propose a novel and theoretically well-justified regularizer commuting the functional map with the multiscale spectral manifold wavelet operator. This regularizer enhances the isometric constraints of the functional map and is conducive to providing it with better structural properties with multiscale analysis. Furthermore, we design an unsupervised deep functional map with the regularizer in a fully differentiable way. The quantitative and qualitative comparisons with several existing techniques on the (near-)isometric and non-isometric datasets show our method's superior accuracy and generalization capabilities. Additionally, we illustrate that our regularizer can be easily inserted into other functional map methods and improve their accuracy.

It is still challenging with existing methods to distinguish structures from texture details, and so preventing texture filtering. Considering that the textures on both sides of a structural edge always differ much from each other in appearances, we determine whether a pixel is on a structure edge by exploiting the appearance contrast between patches around the pixel, and further propose an efficient implementation method. We demonstrate that our proposed method is more effective than existing methods to distinguish structures from texture details, and our required patches for texture measurement can be smaller than the used patches in existing methods by at least half. Thus, we can improve texture filtering on both quality and efficiency, as shown by the experimental results, e.g., we can handle the textured images with a resolution of 800 × 600 pixels in real-time. (The code is available at https://github.com/hefengxiyulu/MLPC)

We present a new method that allows efficient ray tracing of virtually artefact-free animated displaced micro-meshes (DMMs) [MMT23] and preserves their low memory footprint and low BVH build and update cost. DMMs allow for compact representation of micro-triangle geometry through hierarchical encoding of displacements. Displacements are computed with respect to a coarse base mesh and are used to displace new vertices introduced during 1 : 4 subdivision of the base mesh. Applying non-rigid transformation to the base mesh can result in silhouette and normal artefacts (see Figure 1) during animation. We propose an approach which prevents these artefacts by interpolating transformation matrices before applying them to the DMM representation. Our interpolation-based algorithm does not change DMM data structures and it allows for efficient bounding of animated micro-triangle geometry which is essential for fast tessellation-free ray tracing of animated DMMs.

Anisotropic specular appearances are ubiquitous in the environment: brushed stainless steel pans, kettles, elevator walls, fur, or scratched plastics. Real-time rendering of these materials with image-based lighting is challenging due to the complex shape of the bidirectional reflectance distribution function (BRDF). We propose an anisotropic specular image-based lighting method that can serve as a drop-in replacement for the standard bent normal technique [Rev11]. Our method yields more realistic results with a 50% increase in computation time of the previous technique, using the same high dynamic range (HDR) preintegrated environment image. We use several environment samples positioned along the major axis of the specular microfacet BRDF. We derive an analytic formula to determine the two closest and two farthest points from the reflected direction on an approximation of the BRDF confidence region boundary. The two farthest points define the BRDF major axis, while the two closest points are used to approximate the BRDF width. The environment level of detail is derived from the BRDF width and the distance between the samples. We extensively compare our method with the bent normal technique and the ground truth using the GGX specular BRDF.

The synthesis of porous, lattice, or microstructure geometries has captured the attention of many researchers in recent years. Implicit forms, such as triply periodic minimal surfaces (TPMS) has captured a significant attention, recently, as tiles in lattices, partially because implicit forms have the potential for synthesizing with ease more complex topologies of tiles, compared to parametric forms. In this work, we show how variable offsets of implicit forms could be used in lattice design as well as lattice analysis, while graded wall and edge thicknesses could be fully controlled in the lattice and even vary within a single tile. As a result, (geometrically) heterogeneous lattices could be created and adapted to follow analysis results while maintaining continuity between adjacent tiles. We demonstrate this ability on several 3D models, including TPMS.

Three-dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applications including pre / post-surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time-consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these limitations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh reconstruction accuracy when compared to segmentation-based methods. In this study, we address these limitations by combining the implicit function representation with a multi-level deep learning architecture. We introduce a novel multi-level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi-level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z-axis resolutions. We show that our method minimised the staircase artefacts while achieving comparable results in surface accuracy when compared to the state-of-the-art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.

Spherical harmonics are a favorable technique for 3D representation, employing a frequency-based approach through the spherical harmonic transform (SHT). Typically, SHT is performed using equiangular sampling grids. However, these grids are non-uniform on spherical surfaces and exhibit local anisotropy, a common limitation in existing spherical harmonic decomposition methods. This paper proposes a 3D representation method using Fibonacci Spherical Harmonics (FSH3D). We introduce a spherical Fibonacci grid (SFG), which is more uniform than equiangular grids for SHT in the frequency domain. Our method employs analytical weights for SHT on SFG, effectively assigning sampling errors to spherical harmonic degrees higher than the recovered band-limited function. This provides a novel solution for spherical harmonic transformation on non-equiangular grids. The key advantages of our FSH3D method include: 1) With the same number of sampling points, SFG captures more features without bias compared to equiangular grids; 2) The root mean square error of 32-degree spherical harmonic coefficients is reduced by approximately 34.6% for SFG compared to equiangular grids; and 3) FSH3D offers more stable frequency domain representations, especially for rotating functions. FSH3D enhances the stability of frequency domain representations under rotational transformations. Its application in 3D shape reconstruction and 3D shape classification results in more accurate and robust representations. Our code is publicly available at https://github.com/Miraclelzk/Fibonacci-Spherical-Harmonics.

Due to the widespread applications of 2-dimensional spherical designs, there has been an increasing requirement of modeling on the 𝕊² manifold in recent years. Due to the non-Euclidean nature of the sphere, it has some challenges to find a method to represent 2D regions on 𝕊² manifold. In this paper, a skeleton-based representation method of regions on 𝕊², disk B-spline(DBSC) on 𝕊² is proposed. Firstly, we give the definition and basic algorithms of DBSC on 𝕊². Then we provide the calculation method of DBSC on 𝕊², which includes calculating the boundary points, internal points and their corresponding derivatives. Based on that, we give some modeling methods of DBSC on 𝕊², including approximation, deformation. In the end, some stunning application examples of DBSC on 𝕊² are shown. This work lays a theoretical foundation for further applications of DBSC on 𝕊².

Low-light image/video enhancement is a challenging task when images or video are captured under harsh lighting conditions. Existing methods mostly formulate this task as an image-to-image conversion task via supervised or unsupervised learning. However, such conversion methods require an extremely large amount of data for training, whether paired or unpaired. In addition, these methods are restricted to specific training data, making it difficult for the trained model to enhance other types of images or video. In this paper, we explore a novel, fast and flexible, zero-shot, low-light image or video enhancement framework. Without relying on prior training or relationships among neighboring frames, we are committed to estimating the illumination of the input image/frame by a well-designed network. The proposed zero-shot, low-light image/video enhancement architecture includes illumination estimation and residual correction modules. The network architecture is very concise and does not require any paired or unpaired data during training, which allows low-light enhancement to be performed with several simple iterations. Despite its simplicity, we show that the method is fast and generalizes well to diverse lighting conditions. Many experiments on various images and videos qualitatively and quantitatively demonstrate the advantages of our method over state-of-the-art methods.