{"title":"LapisGS:用于自适应流媒体的分层渐进式 3D 高斯拼接技术","authors":"Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi","doi":"arxiv-2408.14823","DOIUrl":null,"url":null,"abstract":"The rise of Extended Reality (XR) requires efficient streaming of 3D online\nworlds, challenging current 3DGS representations to adapt to\nbandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS\nthat supports adaptive streaming and progressive rendering. Our method\nconstructs a layered structure for cumulative representation, incorporates\ndynamic opacity optimization to maintain visual fidelity, and utilizes\noccupancy maps to efficiently manage Gaussian splats. This proposed model\noffers a progressive representation supporting a continuous rendering quality\nadapted for bandwidth-aware streaming. Extensive experiments validate the\neffectiveness of our approach in balancing visual fidelity with the compactness\nof the model, with up to 50.71% improvement in SSIM, 286.53% improvement in\nLPIPS, and 318.41% reduction in model size, and shows its potential for\nbandwidth-adapted 3D streaming and rendering applications.","PeriodicalId":501480,"journal":{"name":"arXiv - CS - Multimedia","volume":"15 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming\",\"authors\":\"Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi\",\"doi\":\"arxiv-2408.14823\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The rise of Extended Reality (XR) requires efficient streaming of 3D online\\nworlds, challenging current 3DGS representations to adapt to\\nbandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS\\nthat supports adaptive streaming and progressive rendering. Our method\\nconstructs a layered structure for cumulative representation, incorporates\\ndynamic opacity optimization to maintain visual fidelity, and utilizes\\noccupancy maps to efficiently manage Gaussian splats. This proposed model\\noffers a progressive representation supporting a continuous rendering quality\\nadapted for bandwidth-aware streaming. Extensive experiments validate the\\neffectiveness of our approach in balancing visual fidelity with the compactness\\nof the model, with up to 50.71% improvement in SSIM, 286.53% improvement in\\nLPIPS, and 318.41% reduction in model size, and shows its potential for\\nbandwidth-adapted 3D streaming and rendering applications.\",\"PeriodicalId\":501480,\"journal\":{\"name\":\"arXiv - CS - Multimedia\",\"volume\":\"15 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Multimedia\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2408.14823\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Multimedia","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.14823","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
扩展现实(XR)的兴起要求高效地流式传输 3D 在线世界,这对当前的 3DGS 表示法适应带宽受限的环境提出了挑战。本文提出了一种支持自适应流媒体和渐进式渲染的分层 3DGS LapisGS。我们的方法为累积表示构建了分层结构,结合了动态不透明度优化以保持视觉保真度,并利用占位图来有效管理高斯飞溅。该模型提供了一种渐进式表示方法,支持适合带宽感知流的连续渲染质量。广泛的实验验证了我们的方法在平衡视觉保真度和模型紧凑性方面的有效性,SSIM 提高了 50.71%,LPIPS 提高了 286.53%,模型大小减少了 318.41%,并显示了它在带宽适应型 3D 流媒体和渲染应用方面的潜力。
LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming
The rise of Extended Reality (XR) requires efficient streaming of 3D online
worlds, challenging current 3DGS representations to adapt to
bandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS
that supports adaptive streaming and progressive rendering. Our method
constructs a layered structure for cumulative representation, incorporates
dynamic opacity optimization to maintain visual fidelity, and utilizes
occupancy maps to efficiently manage Gaussian splats. This proposed model
offers a progressive representation supporting a continuous rendering quality
adapted for bandwidth-aware streaming. Extensive experiments validate the
effectiveness of our approach in balancing visual fidelity with the compactness
of the model, with up to 50.71% improvement in SSIM, 286.53% improvement in
LPIPS, and 318.41% reduction in model size, and shows its potential for
bandwidth-adapted 3D streaming and rendering applications.