Implicit Neural Representations have gained prominence as a powerful framework for capturing complex data modalities, encompassing a wide range from 3D shapes to images and audio. Within the realm of 3D shape representation, Neural Signed Distance Functions (SDF) have demonstrated remarkable potential in faithfully encoding intricate shape geometry. However, learning SDFs from sparse 3D point clouds in the absence of ground truth supervision remains a very challenging task. While recent methods rely on smoothness priors to regularize the learning, our method introduces a regularization term that leverages adversarial samples around the shape to improve the learned SDFs. Through extensive experiments and evaluations, we illustrate the efficacy of our proposed method, highlighting its capacity to improve SDF learning with respect to baselines and the state-of-the-art using synthetic and real data.
{"title":"Few-Shot Unsupervised Implicit Neural Shape Representation Learning with Spatial Adversaries","authors":"Amine Ouasfi, Adnane Boukhayma","doi":"arxiv-2408.15114","DOIUrl":"https://doi.org/arxiv-2408.15114","url":null,"abstract":"Implicit Neural Representations have gained prominence as a powerful\u0000framework for capturing complex data modalities, encompassing a wide range from\u00003D shapes to images and audio. Within the realm of 3D shape representation,\u0000Neural Signed Distance Functions (SDF) have demonstrated remarkable potential\u0000in faithfully encoding intricate shape geometry. However, learning SDFs from\u0000sparse 3D point clouds in the absence of ground truth supervision remains a\u0000very challenging task. While recent methods rely on smoothness priors to\u0000regularize the learning, our method introduces a regularization term that\u0000leverages adversarial samples around the shape to improve the learned SDFs.\u0000Through extensive experiments and evaluations, we illustrate the efficacy of\u0000our proposed method, highlighting its capacity to improve SDF learning with\u0000respect to baselines and the state-of-the-art using synthetic and real data.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hyunwoo Kim, Itai Lang, Noam Aigerman, Thibault Groueix, Vladimir G. Kim, Rana Hanocka
We propose MeshUp, a technique that deforms a 3D mesh towards multiple target concepts, and intuitively controls the region where each concept is expressed. Conveniently, the concepts can be defined as either text queries, e.g., "a dog" and "a turtle," or inspirational images, and the local regions can be selected as any number of vertices on the mesh. We can effectively control the influence of the concepts and mix them together using a novel score distillation approach, referred to as the Blended Score Distillation (BSD). BSD operates on each attention layer of the denoising U-Net of a diffusion model as it extracts and injects the per-objective activations into a unified denoising pipeline from which the deformation gradients are calculated. To localize the expression of these activations, we create a probabilistic Region of Interest (ROI) map on the surface of the mesh, and turn it into 3D-consistent masks that we use to control the expression of these activations. We demonstrate the effectiveness of BSD empirically and show that it can deform various meshes towards multiple objectives.
我们提出的 MeshUp 是一种针对多个目标概念对三维网格进行变形的技术，它可以直观地控制每个概念所表达的区域。我们可以使用一种新颖的分数蒸馏方法（称为混合分数蒸馏法（BSD））有效地控制概念的影响并将它们混合在一起。BSD 对扩散模型的去噪 U 网的每个注意层进行操作，因为它提取并将每个目标的激活状态注入统一的去噪管道，并从中计算出变形梯度。为了定位这些激活的表达，我们在网格表面创建了一个概率感兴趣区域（ROI）图，并将其转化为三维一致的掩码，用来控制这些激活的表达。我们通过经验证明了 BSD 的有效性，并表明它可以使各种网格向多个目标变形。
{"title":"MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation","authors":"Hyunwoo Kim, Itai Lang, Noam Aigerman, Thibault Groueix, Vladimir G. Kim, Rana Hanocka","doi":"arxiv-2408.14899","DOIUrl":"https://doi.org/arxiv-2408.14899","url":null,"abstract":"We propose MeshUp, a technique that deforms a 3D mesh towards multiple target\u0000concepts, and intuitively controls the region where each concept is expressed.\u0000Conveniently, the concepts can be defined as either text queries, e.g., \"a dog\"\u0000and \"a turtle,\" or inspirational images, and the local regions can be selected\u0000as any number of vertices on the mesh. We can effectively control the influence\u0000of the concepts and mix them together using a novel score distillation\u0000approach, referred to as the Blended Score Distillation (BSD). BSD operates on\u0000each attention layer of the denoising U-Net of a diffusion model as it extracts\u0000and injects the per-objective activations into a unified denoising pipeline\u0000from which the deformation gradients are calculated. To localize the expression\u0000of these activations, we create a probabilistic Region of Interest (ROI) map on\u0000the surface of the mesh, and turn it into 3D-consistent masks that we use to\u0000control the expression of these activations. We demonstrate the effectiveness\u0000of BSD empirically and show that it can deform various meshes towards multiple\u0000objectives.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dynamic scene reconstruction has garnered significant attention in recent years due to its capabilities in high-quality and real-time rendering. Among various methodologies, constructing a 4D spatial-temporal representation, such as 4D-GS, has gained popularity for its high-quality rendered images. However, these methods often produce suboptimal surfaces, as the discrete 3D Gaussian point clouds fail to align with the object's surface precisely. To address this problem, we propose DynaSurfGS to achieve both photorealistic rendering and high-fidelity surface reconstruction of dynamic scenarios. Specifically, the DynaSurfGS framework first incorporates Gaussian features from 4D neural voxels with the planar-based Gaussian Splatting to facilitate precise surface reconstruction. It leverages normal regularization to enforce the smoothness of the surface of dynamic objects. It also incorporates the as-rigid-as-possible (ARAP) constraint to maintain the approximate rigidity of local neighborhoods of 3D Gaussians between timesteps and ensure that adjacent 3D Gaussians remain closely aligned throughout. Extensive experiments demonstrate that DynaSurfGS surpasses state-of-the-art methods in both high-fidelity surface reconstruction and photorealistic rendering.
{"title":"DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting","authors":"Weiwei Cai, Weicai Ye, Peng Ye, Tong He, Tao Chen","doi":"arxiv-2408.13972","DOIUrl":"https://doi.org/arxiv-2408.13972","url":null,"abstract":"Dynamic scene reconstruction has garnered significant attention in recent\u0000years due to its capabilities in high-quality and real-time rendering. Among\u0000various methodologies, constructing a 4D spatial-temporal representation, such\u0000as 4D-GS, has gained popularity for its high-quality rendered images. However,\u0000these methods often produce suboptimal surfaces, as the discrete 3D Gaussian\u0000point clouds fail to align with the object's surface precisely. To address this\u0000problem, we propose DynaSurfGS to achieve both photorealistic rendering and\u0000high-fidelity surface reconstruction of dynamic scenarios. Specifically, the\u0000DynaSurfGS framework first incorporates Gaussian features from 4D neural voxels\u0000with the planar-based Gaussian Splatting to facilitate precise surface\u0000reconstruction. It leverages normal regularization to enforce the smoothness of\u0000the surface of dynamic objects. It also incorporates the as-rigid-as-possible\u0000(ARAP) constraint to maintain the approximate rigidity of local neighborhoods\u0000of 3D Gaussians between timesteps and ensure that adjacent 3D Gaussians remain\u0000closely aligned throughout. Extensive experiments demonstrate that DynaSurfGS\u0000surpasses state-of-the-art methods in both high-fidelity surface reconstruction\u0000and photorealistic rendering.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computing on graphics processing units (GPUs) has become standard in scientific computing, allowing for incredible performance gains over classical CPUs for many computational methods. As GPUs were originally designed for 3D rendering, they still have several features for that purpose that are not used in scientific computing. Among them, ray tracing is a powerful technology used to render 3D scenes. In this paper, we propose exploiting ray tracing technology to compute particle interactions with a cutoff distance in a 3D environment. We describe algorithmic tricks and geometric patterns to find the interaction lists for each particle. This approach allows us to compute interactions with quasi-linear complexity in the number of particles without building a grid of cells or an explicit kd-tree. We compare the performance of our approach with a classical approach based on a grid of cells and show that, currently, ours is slower in most cases but could pave the way for future methods.
在图形处理器（GPU）上进行计算已成为科学计算的标准，在许多计算方法中，GPU 的性能都比传统 CPU 高出许多。由于 GPU 最初是为三维渲染而设计的，因此它仍具有一些科学计算中没有使用的功能。其中，光线追踪是一项用于渲染 3D 场景的强大技术。在本文中，我们提出利用光线追踪技术来计算粒子在三维环境中与截止距离的相互作用。我们描述了为每个粒子寻找相互作用列表的算法技巧和几何模式。这种方法允许我们以粒子数量的准线性复杂度计算相互作用，而无需构建单元网格或显式 kd 树。我们比较了我们的方法和基于单元网格的经典方法的性能，结果表明，目前，我们的方法在大多数情况下速度较慢，但可以为未来的方法铺平道路。
{"title":"Exploiting ray tracing technology through OptiX to compute particle interactions with cutoff in a 3D environment on GPU","authors":"Bérenger Bramas","doi":"arxiv-2408.14247","DOIUrl":"https://doi.org/arxiv-2408.14247","url":null,"abstract":"Computing on graphics processing units (GPUs) has become standard in\u0000scientific computing, allowing for incredible performance gains over classical\u0000CPUs for many computational methods. As GPUs were originally designed for 3D\u0000rendering, they still have several features for that purpose that are not used\u0000in scientific computing. Among them, ray tracing is a powerful technology used\u0000to render 3D scenes. In this paper, we propose exploiting ray tracing\u0000technology to compute particle interactions with a cutoff distance in a 3D\u0000environment. We describe algorithmic tricks and geometric patterns to find the\u0000interaction lists for each particle. This approach allows us to compute\u0000interactions with quasi-linear complexity in the number of particles without\u0000building a grid of cells or an explicit kd-tree. We compare the performance of\u0000our approach with a classical approach based on a grid of cells and show that,\u0000currently, ours is slower in most cases but could pave the way for future\u0000methods.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yutaka Endo, Minoru Oikawa, Timothy D. Wilkinson, Tomoyoshi Shimobaba, Tomoyoshi Ito
Computer-generated holography (CGH) is a promising technology for augmented reality displays, such as head-mounted or head-up displays. However, its high computational demand makes it impractical for implementation. Recent efforts to integrate neural networks into CGH have successfully accelerated computing speed, demonstrating the potential to overcome the trade-off between computational cost and image quality. Nevertheless, deploying neural network-based CGH algorithms on computationally limited embedded systems requires more efficient models with lower computational cost, memory footprint, and power consumption. In this study, we developed a lightweight model for complex hologram generation by introducing neural network quantization. Specifically, we built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8). Our performance evaluation shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size by approximately 70% and increasing the speed fourfold. Additionally, we implemented the INT8 model on a system-on-module to demonstrate its deployability on embedded platforms and high power efficiency.
{"title":"Quantized neural network for complex hologram generation","authors":"Yutaka Endo, Minoru Oikawa, Timothy D. Wilkinson, Tomoyoshi Shimobaba, Tomoyoshi Ito","doi":"arxiv-2409.06711","DOIUrl":"https://doi.org/arxiv-2409.06711","url":null,"abstract":"Computer-generated holography (CGH) is a promising technology for augmented\u0000reality displays, such as head-mounted or head-up displays. However, its high\u0000computational demand makes it impractical for implementation. Recent efforts to\u0000integrate neural networks into CGH have successfully accelerated computing\u0000speed, demonstrating the potential to overcome the trade-off between\u0000computational cost and image quality. Nevertheless, deploying neural\u0000network-based CGH algorithms on computationally limited embedded systems\u0000requires more efficient models with lower computational cost, memory footprint,\u0000and power consumption. In this study, we developed a lightweight model for\u0000complex hologram generation by introducing neural network quantization.\u0000Specifically, we built a model based on tensor holography and quantized it from\u000032-bit floating-point precision (FP32) to 8-bit integer precision (INT8). Our\u0000performance evaluation shows that the proposed INT8 model achieves hologram\u0000quality comparable to that of the FP32 model while reducing the model size by\u0000approximately 70% and increasing the speed fourfold. Additionally, we\u0000implemented the INT8 model on a system-on-module to demonstrate its\u0000deployability on embedded platforms and high power efficiency.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daxuan Renınst, Hezi Shiınst, Jianmin Zheng, Jianfei Cai
Iso-surface extraction from an implicit field is a fundamental process in various applications of computer vision and graphics. When dealing with geometric shapes with complicated geometric details, many existing algorithms suffer from high computational costs and memory usage. This paper proposes McGrids, a novel approach to improve the efficiency of iso-surface extraction. The key idea is to construct adaptive grids for iso-surface extraction rather than using a simple uniform grid as prior art does. Specifically, we formulate the problem of constructing adaptive grids as a probability sampling problem, which is then solved by Monte Carlo process. We demonstrate McGrids' capability with extensive experiments from both analytical SDFs computed from surface meshes and learned implicit fields from real multiview images. The experiment results show that our McGrids can significantly reduce the number of implicit field queries, resulting in significant memory reduction, while producing high-quality meshes with rich geometric details.
{"title":"McGrids: Monte Carlo-Driven Adaptive Grids for Iso-Surface Extraction","authors":"Daxuan Renınst, Hezi Shiınst, Jianmin Zheng, Jianfei Cai","doi":"arxiv-2409.06710","DOIUrl":"https://doi.org/arxiv-2409.06710","url":null,"abstract":"Iso-surface extraction from an implicit field is a fundamental process in\u0000various applications of computer vision and graphics. When dealing with\u0000geometric shapes with complicated geometric details, many existing algorithms\u0000suffer from high computational costs and memory usage. This paper proposes\u0000McGrids, a novel approach to improve the efficiency of iso-surface extraction.\u0000The key idea is to construct adaptive grids for iso-surface extraction rather\u0000than using a simple uniform grid as prior art does. Specifically, we formulate\u0000the problem of constructing adaptive grids as a probability sampling problem,\u0000which is then solved by Monte Carlo process. We demonstrate McGrids' capability\u0000with extensive experiments from both analytical SDFs computed from surface\u0000meshes and learned implicit fields from real multiview images. The experiment\u0000results show that our McGrids can significantly reduce the number of implicit\u0000field queries, resulting in significant memory reduction, while producing\u0000high-quality meshes with rich geometric details.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many real-world materials are characterized by a glittery appearance. Reproducing this effect in physically based renderings is a challenging problem due to its discrete nature, especially in real-time applications which require a consistently low runtime. Recent work focuses on glittery appearance illuminated by infinitesimally small light sources only. For light sources like the sun this approximation is a reasonable choice. In the real world however, all light sources are fundamentally area light sources. In this paper, we derive an efficient method for rendering glints illuminated by spatially constant diffuse area lights in real time. To this end, we require an adequate estimate for the probability of a single microfacet to be correctly oriented for reflection from the source to the observer. A good estimate is achieved either using linearly transformed cosines (LTC) for large light sources, or a locally constant approximation of the normal distribution for small spherical caps of light directions. To compute the resulting number of reflecting microfacets, we employ a counting model based on the binomial distribution. In the evaluation, we demonstrate the visual accuracy of our approach, which is easily integrated into existing real-time rendering frameworks, especially if they already implement shading for area lights using LTCs and a counting model for glint shading under point and directional illumination. Besides the overhead of the preexisting constituents, our method adds little to no additional overhead.
{"title":"Real-Time Rendering of Glints in the Presence of Area Lights","authors":"Tom Kneiphof, Reinhard Klein","doi":"arxiv-2408.13611","DOIUrl":"https://doi.org/arxiv-2408.13611","url":null,"abstract":"Many real-world materials are characterized by a glittery appearance.\u0000Reproducing this effect in physically based renderings is a challenging problem\u0000due to its discrete nature, especially in real-time applications which require\u0000a consistently low runtime. Recent work focuses on glittery appearance\u0000illuminated by infinitesimally small light sources only. For light sources like\u0000the sun this approximation is a reasonable choice. In the real world however,\u0000all light sources are fundamentally area light sources. In this paper, we\u0000derive an efficient method for rendering glints illuminated by spatially\u0000constant diffuse area lights in real time. To this end, we require an adequate\u0000estimate for the probability of a single microfacet to be correctly oriented\u0000for reflection from the source to the observer. A good estimate is achieved\u0000either using linearly transformed cosines (LTC) for large light sources, or a\u0000locally constant approximation of the normal distribution for small spherical\u0000caps of light directions. To compute the resulting number of reflecting\u0000microfacets, we employ a counting model based on the binomial distribution. In\u0000the evaluation, we demonstrate the visual accuracy of our approach, which is\u0000easily integrated into existing real-time rendering frameworks, especially if\u0000they already implement shading for area lights using LTCs and a counting model\u0000for glint shading under point and directional illumination. Besides the\u0000overhead of the preexisting constituents, our method adds little to no\u0000additional overhead.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221919","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhenyuan Liu, Yu Guo, Xinyuan Li, Bernd Bickel, Ran Zhang
We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique designed to represent and render 3D objects with surface and volumetric materials under dynamic illumination. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects. To unify surface and volumetric material within a cohesive appearance model, we adopt a light- and view-dependent scattering representation via bidirectional spherical harmonics. Our model does not use a specific surface normal-related reflectance function, making it more compatible with volumetric representations like Gaussian splatting, where the normals are undefined. We demonstrate our method by reconstructing and rendering objects with complex materials. Using One-Light-At-a-Time (OLAT) data as input, we can reproduce photorealistic appearances under novel lighting conditions in real time.
{"title":"BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting","authors":"Zhenyuan Liu, Yu Guo, Xinyuan Li, Bernd Bickel, Ran Zhang","doi":"arxiv-2408.13370","DOIUrl":"https://doi.org/arxiv-2408.13370","url":null,"abstract":"We present Bidirectional Gaussian Primitives, an image-based novel view\u0000synthesis technique designed to represent and render 3D objects with surface\u0000and volumetric materials under dynamic illumination. Our approach integrates\u0000light intrinsic decomposition into the Gaussian splatting framework, enabling\u0000real-time relighting of 3D objects. To unify surface and volumetric material\u0000within a cohesive appearance model, we adopt a light- and view-dependent\u0000scattering representation via bidirectional spherical harmonics. Our model does\u0000not use a specific surface normal-related reflectance function, making it more\u0000compatible with volumetric representations like Gaussian splatting, where the\u0000normals are undefined. We demonstrate our method by reconstructing and\u0000rendering objects with complex materials. Using One-Light-At-a-Time (OLAT) data\u0000as input, we can reproduce photorealistic appearances under novel lighting\u0000conditions in real time.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Designing a freeform surface to reflect or refract light to achieve a target distribution is a challenging inverse problem. In this paper, we propose an end-to-end optimization strategy for an optical surface mesh. Our formulation leverages a novel differentiable rendering model, and is directly driven by the difference between the resulting light distribution and the target distribution. We also enforce geometric constraints related to fabrication requirements, to facilitate CNC milling and polishing of the designed surface. To address the issue of local minima, we formulate a face-based optimal transport problem between the current mesh and the target distribution, which makes effective large changes to the surface shape. The combination of our optimal transport update and rendering-guided optimization produces an optical surface design with a resulting image closely resembling the target, while the fabrication constraints in our optimization help to ensure consistency between the rendering model and the final physical results. The effectiveness of our algorithm is demonstrated on a variety of target images using both simulated rendering and physical prototypes.
{"title":"End-to-end Surface Optimization for Light Control","authors":"Yuou Sun, Bailin Deng, Juyong Zhang","doi":"arxiv-2408.13117","DOIUrl":"https://doi.org/arxiv-2408.13117","url":null,"abstract":"Designing a freeform surface to reflect or refract light to achieve a target\u0000distribution is a challenging inverse problem. In this paper, we propose an\u0000end-to-end optimization strategy for an optical surface mesh. Our formulation\u0000leverages a novel differentiable rendering model, and is directly driven by the\u0000difference between the resulting light distribution and the target\u0000distribution. We also enforce geometric constraints related to fabrication\u0000requirements, to facilitate CNC milling and polishing of the designed surface.\u0000To address the issue of local minima, we formulate a face-based optimal\u0000transport problem between the current mesh and the target distribution, which\u0000makes effective large changes to the surface shape. The combination of our\u0000optimal transport update and rendering-guided optimization produces an optical\u0000surface design with a resulting image closely resembling the target, while the\u0000fabrication constraints in our optimization help to ensure consistency between\u0000the rendering model and the final physical results. The effectiveness of our\u0000algorithm is demonstrated on a variety of target images using both simulated\u0000rendering and physical prototypes.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose the first comprehensive approach for modeling and analyzing the spatiotemporal shape variability in tree-like 4D objects, i.e., 3D objects whose shapes bend, stretch, and change in their branching structure over time as they deform, grow, and interact with their environment. Our key contribution is the representation of tree-like 3D shapes using Square Root Velocity Function Trees (SRVFT). By solving the spatial registration in the SRVFT space, which is equipped with an L2 metric, 4D tree-shaped structures become time-parameterized trajectories in this space. This reduces the problem of modeling and analyzing 4D tree-like shapes to that of modeling and analyzing elastic trajectories in the SRVFT space, where elasticity refers to time warping. In this paper, we propose a novel mathematical representation of the shape space of such trajectories, a Riemannian metric on that space, and computational tools for fast and accurate spatiotemporal registration and geodesics computation between 4D tree-shaped structures. Leveraging these building blocks, we develop a full framework for modelling the spatiotemporal variability using statistical models and generating novel 4D tree-like structures from a set of exemplars. We demonstrate and validate the proposed framework using real 4D plant data.
{"title":"A Riemannian Approach for Spatiotemporal Analysis and Generation of 4D Tree-shaped Structures","authors":"Tahmina Khanam, Hamid Laga, Mohammed Bennamoun, Guanjin Wang, Ferdous Sohel, Farid Boussaid, Guan Wang, Anuj Srivastava","doi":"arxiv-2408.12443","DOIUrl":"https://doi.org/arxiv-2408.12443","url":null,"abstract":"We propose the first comprehensive approach for modeling and analyzing the\u0000spatiotemporal shape variability in tree-like 4D objects, i.e., 3D objects\u0000whose shapes bend, stretch, and change in their branching structure over time\u0000as they deform, grow, and interact with their environment. Our key contribution\u0000is the representation of tree-like 3D shapes using Square Root Velocity\u0000Function Trees (SRVFT). By solving the spatial registration in the SRVFT space,\u0000which is equipped with an L2 metric, 4D tree-shaped structures become\u0000time-parameterized trajectories in this space. This reduces the problem of\u0000modeling and analyzing 4D tree-like shapes to that of modeling and analyzing\u0000elastic trajectories in the SRVFT space, where elasticity refers to time\u0000warping. In this paper, we propose a novel mathematical representation of the\u0000shape space of such trajectories, a Riemannian metric on that space, and\u0000computational tools for fast and accurate spatiotemporal registration and\u0000geodesics computation between 4D tree-shaped structures. Leveraging these\u0000building blocks, we develop a full framework for modelling the spatiotemporal\u0000variability using statistical models and generating novel 4D tree-like\u0000structures from a set of exemplars. We demonstrate and validate the proposed\u0000framework using real 4D plant data.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142221924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：