Computer Animation and Virtual Worlds最新文献

This research introduces a novel approach to generate three-dimensional objects using human voice as input for a machine learning model. The spoken input is first converted into text using a Google API, which then guides the creation of the desired three-dimensional object. This approach is particularly beneficial for individuals without design expertise, paving the way for broader participation in the evolving Metaverse—a virtual reality that transcends our physical realm. The three-dimensional object generation model comprises three primary components: Neural Radiance Fields (NeRF), Low-Rank Adaptation (LoRA), and Stable Diffusion. When combined, these components facilitate the creation of a diverse range of three-dimensional objects. Our method presents an innovative approach to harnessing speech recognition for generating three-dimensional objects within the Metaverse, while demonstrating competitive quality and practical efficiency.

Text-driven human motion generation aims to synthesize coherent, natural human actions from natural-language descriptions. While current techniques are effective for short sequences, they often suffer from abrupt transitions, detail loss, and temporal misalignment when extended to long sequences. We present SD-T2LM, a unified framework that couples transition optimization, detail enhancement, and temporal alignment to produce natural, detail-rich long motion sequences. First, a dynamic SLERP interpolation selects transition length conditioned on start/end pose similarity to suppress discontinuities while preserving physical plausibility. Second, a dynamic masking mechanism progressively reduces the mask ratio from 60% to 30% during training, enabling staged learning-global structure early, fine-grained details later. Third, a Temporal Flexible Alignment loss based on Soft-DTW aligns generated and ground-truth rhythms without enforcing point-wise synchrony. On the HumanML3D dataset and KIT-ML dataset, SD-T2LM showed significant improvements in FID and diversity compared to the baseline model, while reducing transition distance and jerk.

Reliable estimation of torso direction is essential for virtual locomotion as it controls the locomotion direction in virtual environments. Existing approaches rely on external sensors to track torso direction or use the poses and/or movements from the head, hands, or feet to control the direction of travel. In addition, deep learning methods based on a head-mounted display (HMD) and its controllers were proposed for torso direction estimation but are unable to produce continuous estimates or that the estimation errors were not systematically evaluated. On the other hand, 3D full-body human motion generation methods based on an HMD and its controllers are capable of estimating torso direction but this capability was largely ignored for its use in virtual steering. We proposed a new method called TCNPoser that can perform torso direction estimation and generate 3D full-body human motion. Through offline evaluation and a user study, we show that the proposed method achieves the state-of-the-art (SOTA) performance for real-time torso direction estimation.

The COVID-19 pandemic has profoundly disrupted traditional face-to-face education, accelerating the shift towards digital learning environments and prompting the adoption of innovative pedagogical approaches. Among these, Augmented Reality (AR) has emerged as a transformative educational tool, offering immersive and interactive platforms that enhance students' understanding of complex concepts. This study investigates the integration of AR technology into university-level beverage preparation courses, with the aim of fostering creativity and strengthening professional competencies. The findings indicate that AR has considerable potential to enhance creative performance, reshape pedagogical practices, and promote self-directed learning. By addressing prevalent learning challenges and providing actionable insights into students' developmental progress, AR supports the design of tailored teaching strategies while cultivating industry-relevant skills. This research highlights the role of AR in advancing inclusive, equitable, and high-quality education, in alignment with Sustainable Development Goal 4 (SDG 4), and underscores its broader significance in redefining educational paradigms in the post-pandemic era.

The convergence of deep learning and the Metaverse represents a pivotal frontier in the evolution of intelligent digital ecosystems. This paper presents a comprehensive survey of how deep learning techniques spanning convolutional, generative, transformer-based, and reinforcement architectures collectively enable perception, creation, cognition, and governance within immersive virtual worlds. Building upon this synthesis, we propose the Deep Learning-Empowered Metaverse Intelligence (DL-MI) framework, which unifies sensory intelligence, generative world-building, adaptive reasoning, and ethical-social governance into a cohesive architecture. The study illustrates how deep learning facilitates realistic avatar synthesis, dynamic environmental rendering, emotion-aware interaction, and predictive personalization, thereby transforming the Metaverse from reactive systems to anticipatory, self-evolving spaces. Key challenges such as data privacy, algorithmic bias, and computational sustainability are critically examined alongside emerging paradigms, including quantum-augmented AI and federated collaboration. By integrating technical, ethical, and societal dimensions, this survey provides a structured foundation for developing scalable, transparent, and human-centered Metaverse intelligence.

The proposed MotionBlend GAN model marks a significant step forward in video synthesis by blending the motion from a source video with the appearance of a target person's image. As training progresses, the model improves video creation by enhancing the smoothness and natural flow of motion, resulting in more coherent and lifelike videos. Using advanced techniques like MoBConv blocks of EfficientNet-B7, OpenPose for precise pose detection, ResNet blocks for feature integration, and a 3D CNN discriminator, the model produces high-quality videos that maintain both spatial and temporal consistency. After 200 epochs, the model achieved an adversarial loss of 0.2265, with metrics like PSNR at 20.246, SSIM at 0.867, and LPIPS at 0.178. The high PSNR and SSIM values, along with the low LPIPS, show that the generated frames are well aligned and preserve important details. These results highlight the model's strong performance over time, consistently generating visually convincing videos of human activities using a reference image and source video. The model effectively transfers motion from video to image, creating realistic videos of human activity in comparison to existing models.

Dressing animations have broad applications in film and animation, but current methods require retraining networks for new styles, which is resource-intensive. While 2D pattern parameters are used for static 3D clothing modeling and editing, applying them to control style variations in dynamic clothing deformation is challenging due to the uncertainty introduced by multiple parameters. Ensuring consistent and stable style features across time-varying deformation sequences is difficult. Therefore, we present StyleGarNet, a novel approach for style-parameter-controlled dressing animation generation and editing. We employ a conditional variational autoencoder architecture to build our network, with style parameters serving as constraint information. This allows us to learn the probabilistic distribution model of deformations under style constraints, crucial for enhancing the robust representation of styles during the deformation process. Simultaneously, considering the temporal nature of motion, we introduce a transformer layer to capture the temporal dependencies of both motion and clothing deformation, thereby enhancing the stability of clothing deformation. Ultimately, our approach enables flexible manipulation of dressing animation generation through inputting style and motion features. Evaluation results demonstrate the efficacy of our approach and show that StyleGarNet outperforms existing methods in terms of prediction speed, accuracy, and stability of deformation sequences.

Co-speech gestures generation plays a key role in the field of virtual reality interaction, synthesizing proper digital human's actions passively with speech. The generative algorithm-based method creates realistic gestures accompanying speech's rhythm and semantic content, improving the interactive experience. To match the persona of digital humans, gestures from algorithms often require additional modification before being applied to a virtual character. However, motion sequences are difficult to edit when generated from hidden motion representations. To make motion synthesis editable, the proposed method develops a two-stage controllable gesture generation pipeline for the c-speech ge-ture generating problem. In stage 1, we design a novel large language model based -K-Decoder th-t takes speech and style label as input to synthesize inverse ki-ematic s-yle control points, which are highly editable. In stage 2, we divide the motion sequence into the body or fingers part for VQ-based latent motion representation learning relatively. And a diffusion-based IK-Denoiser is proposed for'latent motion representation synthesis under the condition of control points. Compared to other representative algorithms, the proposed method gets a competitive performance of metrics such as Fréchet Gesture Distance, Beat Consistency, and Diversity. To demonstrate controllability, it provides three explicit control strategies for motion editing. With these control points, we provide a new co-speech gesture generation paradigm.