Computer Animation and Virtual Worlds最新文献

The material point method (MPM) has attracted more and more attention in computer graphics. It is very successful in simulating both fluid flow and solid deformation, but may fail in simulating multiple fluids and solids coupling. We propose a unified MPM solver for multi-species simulations. Compared to traditional MPM, we extend the degree of freedom on background grid to store information of multiple materials, so that our framework is able to deal with multiple materials well. The proposed method leverages the advantages of MPM as a hybrid method. We introduce the mixture model into the framework, which was the most widely used for grid-based multi-fluid flows. This enables MPM to capture the interaction and relative motion, and animates complex and coupled fluids and solids in a unified manner. A series of experiments are presented to demonstrate effectiveness of our method.

Face image generation plays an important role in generating innovative and unique multimedia content using the GAN model. With these qualities of the GAN model, they have numerous challenges in the human face image generation. The problems encountered in the generation of facial images are like blurriness in images, incomplete details in the generated facial images, high computational power requirements, and so forth. In this manuscript, we proposed a GAN model that utilizes the composite strength of VGG-16 and ResNet-50's models to overcome those difficulties. It uses VGG-16 to build a discriminator model to discriminate between real and fake images. The generator model utilizes a combination of components from the ResNet-50 and VGG-16 models to enhance the image generation process at each iteration, resulting in the creation of realistic face images. The proposed DRFI GAN (Diversified and Realistic Face Image Generation GAN) model's generator achieves an impressive low FID score of 20.50, which is less than existing state-of-the-art approaches. Furthermore, our findings indicate that the images generated by the DRFI GAN model exhibit 10%–15% greater efficiency and realism with reduced training time compared to existing state-of-the-art methods with lower FID scores.

This study primarily focuses on investigating whether virtual reality scenarios can authentically replicate real-life audio-visual environments. The authenticity of audio-visual environments plays a crucial role in both the design and VR fields today. Only when the authenticity of audio-visual interactive experiences is validated as feasible can virtual reality technology demonstrate positive impacts. We assessed the annoyance levels subjectively under different audio-visual conditions: a real cafeteria environment and a simulated cafeteria environment. Participants were tasked with the same activities in both environments. After each experiment, they indicated their levels of annoyance by completing a questionnaire. The results indicated a significant positive correlation between the overall subjective annoyance levels in both experiments and the subjective annoyance levels associated with different behaviors. This suggests that under identical audio conditions, virtual reality scenarios more effectively replicate the real noise environment. Furthermore, we have uncovered that certain objective factors influence the expression of authenticity. Optimizing these factors may potentially further enhance the feasibility of virtual reality technology in audio-visual environments.

This paper presents a novel real-time facial feature extraction algorithm, producing a small feature set, suitable for implementing emotion recognition with online game and metaverse avatars. The algorithm aims to reduce data transmission and storage requirements, hurdles in the adoption of emotion recognition in these mediums. The early results presented show a facial emotion recognition accuracy of up to 92% on one benchmark dataset, with an overall accuracy of 77.2% across a wide range of datasets, demonstrating the early promise of the research.

Coloring an animation sketch sequence is a challenging task in computer vision since the information contained in line sketches is too sparse, and the colors need to be uniform between continuous frames. Many the existing colorization algorithms can only be applied to one image and can be considered color filling algorithms. Such algorithms only provide a color result that fits within a reasonable range and can not be applied to the coloring of frame sequences. This paper proposes an end-to-end two-stage optical flow colorization network to solve the animation frame sequence colorization problem. The first stage of the network finds the direction of the color pixel flow from the detail change between a given reference frame and the next frame of line artwork and then completes the initial coloring process. The second stage of the network performs color correction and clarifies the output of the first stage. Since our algorithm does not directly colorize the image but finds the path of the color change to colorize it, it ensures a consistent color space for the sequence frames after colorization. We conduct experiments on an animation dataset, and the results show that our algorithm is effective. The code is available at https://github.com/silenye/Colorization.

Marker-based optical motion capture (MoCap) aims to localize 3D human motions from a sequence of input raw markers. It is widely used to produce physical movements for virtual characters in various games such as the role-playing game, the fighting game, and the action-adventure game. However, the conventional MoCap cleaning and solving process is extremely labor-intensive, time-consuming, and usually the most costly part of game animation production. Thus, there is a high demand for automated algorithms to replace costly manual operations and achieve accurate MoCap cleaning and solving in the game industry. In this article, we design a divide-and-conquer-based MoCap solving network, dubbed MarkerNet, to estimate human skeleton motions from sequential raw markers effectively. In a nutshell, our key idea is to decompose the task of direct solving of global motion from all markers into first modeling sub-motions of local parts from the corresponding marker subsets and then aggregating sub-motions into a global one. In this manner, our model can effectively capture local motion patterns w.r.t. different marker subsets, thus producing more accurate results compared to the existing methods. Extensive experiments on both real and synthetic data verify the effectiveness of the proposed method.

Face-to-face communication relies extensively on non-verbal cues (NVCs) which complement, or at times dominate, the communicative process as they convey emotions with intense salience, thus definitively affecting interpersonal communication. The capture, transference, and subsequent interpretation of NVCs becomes complicated in computer-mediated communicative processes, particularly in shared virtual worlds, for which there is growing interest both in regard to NVCs technological integration and their affective impact. This paper presents a between-groups experimental setup which is facilitated in immersive virtual reality (IVR), and examines NVCs effects on user experience, with special emphasis on degree of attention toward each NVC as an isolated controlled variable of a scripted performance by a virtual character (VC). This study aims to evaluate NVCs fidelity based on the capabilities of the motion-capture technologies utilized to address cue integration developmental challenges and examines NVCs impact on users' perceived realism of the VC, their empathy toward him, and the degree of social presence experienced. To meet the objectives set the affective impact of low-fidelity automated NVCs and high-fidelity real-time captured NVCs were compared. The findings of the evaluation suggest that although NVCs do impact user experience to an extent, their effects are notably more subtle compared to previous studies.

Talking head generation aims to synthesize a photo-realistic speaking video with accurate lip motion. While this field has attracted more attention in recent audio-visual researches, most existing methods do not achieve the simultaneous improvement of lip synchronization and visual quality. In this paper, we propose Wav2Lip-HR, a neural-based audio-driven high-resolution talking head generation method. With our technique, all required to generate a clear high-resolution lip sync talking video is an image/video of the target face and an audio clip of any speech. The primary benefit of our method is that it generates clear high-resolution videos with sufficient facial details, rather than the ones just be large-sized with less clarity. We first analyze key factors that limit the clarity of generated videos and then put forth several important solutions to address the problem, including data augmentation, model structure improvement and a more effective loss function. Finally, we employ several efficient metrics to evaluate the clarity of images generated by our proposed approach as well as several widely used metrics to evaluate lip-sync performance. Numerous experiments demonstrate that our method has superior performance on visual quality and lip synchronization when compared to other existing schemes.

The color change of plant fruit in ripening is a typical time-varying phenomenon involving various factors. Due to its complexity and biodiversity, it is challenging to model this phenomenon. To address this issue, we take the tomato as an example and propose a botanical-based framework considering variety, environment, phytohormone, and genes to simulate fruit color change during the ripening process. Specifically, we propose a first-order kinetic model that integrates varietal, environmental, and phytohormonal factors to represent the variation of pigment concentrations in the pericarp. Moreover, we introduce a logistic model to describe the change in pigment concentration in the epidermis. Based on the gene expression pathway of tomato color in botany, we propose a genotype-to-phenotype simulation method to represent its biodiversity. An improved method is proposed to convert pigment concentrations into color accurately. Furthermore, we propose a gradient descent-based method to assist the user in quickly setting pigment concentration parameters. Experiments verified that the proposed framework can simulate a wide range of tomato colors. Both qualitative and quantitative experiments validated the proposed method. Furthermore, our framework can be applied to more fruits.