Jie FuUniversity of the Arts London, Creative Computing Institute, London, United Kingdom, Shun FuBloks Technology Company, Shanghai, China, Mick GriersonUniversity of the Arts London, Creative Computing Institute, London, United Kingdom
{"title":"根据单张图像生成珊瑚模型,用于虚拟现实应用","authors":"Jie FuUniversity of the Arts London, Creative Computing Institute, London, United Kingdom, Shun FuBloks Technology Company, Shanghai, China, Mick GriersonUniversity of the Arts London, Creative Computing Institute, London, United Kingdom","doi":"arxiv-2409.02376","DOIUrl":null,"url":null,"abstract":"With the rapid development of VR technology, the demand for high-quality 3D\nmodels is increasing. Traditional methods struggle with efficiency and quality\nin large-scale customization. This paper introduces a deep-learning framework\nthat generates high-precision 3D coral models from a single image. Using the\nCoral dataset, the framework extracts geometric and texture features, performs\n3D reconstruction, and optimizes design and material blending. Advanced\noptimization and polygon count control ensure shape accuracy, detail retention,\nand flexible output for various complexities, catering to high-quality\nrendering and real-time interaction needs.The project incorporates Explainable\nAI (XAI) to transform AI-generated models into interactive \"artworks,\" best\nviewed in VR and XR. This enhances model interpretability and human-machine\ncollaboration. Real-time feedback in VR interactions displays information like\ncoral species and habitat, enriching user experience. The generated models\nsurpass traditional methods in detail, visual quality, and efficiency. This\nresearch offers an intelligent approach to 3D content creation for VR, lowering\nproduction barriers, and promoting widespread VR applications. Additionally,\nintegrating XAI provides new insights into AI-generated visual content and\nadvances research in 3D vision interpretability.","PeriodicalId":501480,"journal":{"name":"arXiv - CS - Multimedia","volume":"59 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Coral Model Generation from Single Images for Virtual Reality Applications\",\"authors\":\"Jie FuUniversity of the Arts London, Creative Computing Institute, London, United Kingdom, Shun FuBloks Technology Company, Shanghai, China, Mick GriersonUniversity of the Arts London, Creative Computing Institute, London, United Kingdom\",\"doi\":\"arxiv-2409.02376\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"With the rapid development of VR technology, the demand for high-quality 3D\\nmodels is increasing. Traditional methods struggle with efficiency and quality\\nin large-scale customization. This paper introduces a deep-learning framework\\nthat generates high-precision 3D coral models from a single image. Using the\\nCoral dataset, the framework extracts geometric and texture features, performs\\n3D reconstruction, and optimizes design and material blending. Advanced\\noptimization and polygon count control ensure shape accuracy, detail retention,\\nand flexible output for various complexities, catering to high-quality\\nrendering and real-time interaction needs.The project incorporates Explainable\\nAI (XAI) to transform AI-generated models into interactive \\\"artworks,\\\" best\\nviewed in VR and XR. This enhances model interpretability and human-machine\\ncollaboration. Real-time feedback in VR interactions displays information like\\ncoral species and habitat, enriching user experience. The generated models\\nsurpass traditional methods in detail, visual quality, and efficiency. This\\nresearch offers an intelligent approach to 3D content creation for VR, lowering\\nproduction barriers, and promoting widespread VR applications. Additionally,\\nintegrating XAI provides new insights into AI-generated visual content and\\nadvances research in 3D vision interpretability.\",\"PeriodicalId\":501480,\"journal\":{\"name\":\"arXiv - CS - Multimedia\",\"volume\":\"59 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Multimedia\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.02376\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Multimedia","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.02376","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
随着虚拟现实技术的迅猛发展,对高质量 3D 模型的需求与日俱增。传统方法在大规模定制中难以保证效率和质量。本文介绍了一种深度学习框架,它能从单张图像生成高精度的三维珊瑚模型。该框架使用珊瑚数据集提取几何和纹理特征,执行三维重建,并优化设计和材料混合。先进的优化和多边形数量控制确保了形状的准确性、细节的保留以及各种复杂性的灵活输出,满足了高质量渲染和实时交互的需求。该项目结合了可解释人工智能(XAI),将人工智能生成的模型转化为交互式 "艺术品",在 VR 和 XR 中观看效果最佳。这增强了模型的可解释性和人机协作。VR 交互中的实时反馈显示了珊瑚种类和栖息地等信息,丰富了用户体验。生成的模型在细节、视觉质量和效率方面都超越了传统方法。这项研究为 VR 3D 内容创建提供了一种智能方法,降低了制作门槛,促进了 VR 应用的普及。此外,整合 XAI 还为人工智能生成的视觉内容提供了新的见解,并推动了 3D 视觉可解释性方面的研究。
Coral Model Generation from Single Images for Virtual Reality Applications
With the rapid development of VR technology, the demand for high-quality 3D
models is increasing. Traditional methods struggle with efficiency and quality
in large-scale customization. This paper introduces a deep-learning framework
that generates high-precision 3D coral models from a single image. Using the
Coral dataset, the framework extracts geometric and texture features, performs
3D reconstruction, and optimizes design and material blending. Advanced
optimization and polygon count control ensure shape accuracy, detail retention,
and flexible output for various complexities, catering to high-quality
rendering and real-time interaction needs.The project incorporates Explainable
AI (XAI) to transform AI-generated models into interactive "artworks," best
viewed in VR and XR. This enhances model interpretability and human-machine
collaboration. Real-time feedback in VR interactions displays information like
coral species and habitat, enriching user experience. The generated models
surpass traditional methods in detail, visual quality, and efficiency. This
research offers an intelligent approach to 3D content creation for VR, lowering
production barriers, and promoting widespread VR applications. Additionally,
integrating XAI provides new insights into AI-generated visual content and
advances research in 3D vision interpretability.