Graphics and Visual Computing最新文献

How do I look if I have the same nose shape as my favorite star? Existing methods for face image manipulation generally focus on modifying predefined facial attributes, editing expressions and changing image styles, where users cannot control the shapes of specific semantic facial parts freely in the generated face image. The facial part shapes are described by their geometries and need to be controlled by continuously generating geometric parameters. Therefore, the existing methods that work with discretely labelled attributes are not applicable on this task. In this paper, we propose a novel approach to learn the disentangled shape representation for a face image, namely the FaceShapeGene, which encodes the shape information of the semantic facial parts into separate chunks in the latent space. It allows users to freely recombine the part-wise latent chunks of a face image from other individuals to transfer a specified facial part shape, just like gene editing. Experimental results on several tasks demonstrate that the proposed FaceShapeGene representation correctly disentangles the shape features of different semantic parts. Comparisons to existing methods show the superiority of the proposed method on facial parts editing tasks.

In this paper, we propose a lookup table method to simulate a physically plausible large-scale forest animation. Our method is rooted in the FEM simulator to simulate plant deformation, and establish a table through pre-computed dynamic equation data. Based on this table, an efficient physical state similarity algorithm for traversing the table is proposed. Real-time search for the vertices displacement of the model from the table, and constantly update the current gesture of the model. The forest can react to the environment at an interactive rate by calculating the effects of random wind. Experiments show that our method can simulate a single tree with high efficiency, and can simulate a physically plausible forest with 500 trees at a frame rate of 34(fps).

This paper presents an optimization-based approach for designing virtual reality game level layouts, based on the layout of a real environment. Our method starts by asking the user to define the shape of the real environment and the obstacles (e.g., furniture) located in it. Then, by representing a game level as an assembly of chunks and defining the game level layout design decisions in cost terms (mapping, fitting, variations, and accessibility) in a total cost function, our system automatically synthesizes a game level layout that fulfills the real environment layout and its constraints as well as the user-defined design decisions. To evaluate the proposed method, a user study was conducted. The results indicated that the proposed method: (1) enhanced the levels of presence; (2) enhanced the levels of involvement of participants in the virtual environment; and (3) reduced the fear of collision with the real environment and its constraints. Limitations and future research directions are also discussed.

Convolutional Neural Network (CNN) accelerator design on resource limited platform faces the challenge of lacking efficient design space exploration (DSE) method because of its huge and irregular design space. Numerous parameters belong to accelerator architecture and dataflow mode jointly construct a huge design space while power and resource constrains make the design space become quite irregular. Under such circumstances, traditional DSE methods based on exhaustive search is infeasible for the non-trivial design space and methods based on general optimization algorithms will also be inefficient because of the irregular distribution of design points. In this paper, we provide an efficient DSE method named ERDSE for CNN accelerator design on resource limited platform. ERDSE is based on reinforcement learning algorithm REINFORCE but refines it to adapt the complex design space. ERDSE implements off-policy strategy to decouple sampling and learning phase, then separately refines them to further improve exploration ability and samples utilization. We implement ERDSE to optimize the computing latency of CNN accelerator for VGG-16 and MobileNet-V3. Under the tightest constraints, ERDSE achieves 1.2x-1.7x (on VGG-16) and 2.3-4.9x (on MobileNet-V3) latency improvement compared with other DSE methods, which demonstrates the efficiency of ERDSE.

In consideration of interdependency between image sensing/recognition (computer vision, CV) and 3D image synthesis (computer graphics, CG) in visual computing, Keio University, Department of Information and Computer Science, reorganized its first course on CV and CG in the undergraduate program into a series of three courses on visual computing in the 2019 academic year. One salient feature of these courses is its newly introduced programming assignment with two specific goals: to experience GPU computing and to understand the path tracing algorithm. The purpose is to help students easily understand the trend in visual computing and vividly envision the future CG area. Specifically, two types of tasks were given to students: material design and analysis of the relationship between sampling and noise. The educational material builds on Google Colaboratory, a cloud-based development environment, to be independent of the students’ hardware. Owing to the judicious design, students can unhesitatingly work on the programming assignment with relatively inexpensive hardware, such as laptop PCs, tablets, or even smartphones, whenever they have a standard off-campus network environment. In the second academic year (2020) of these courses, this type of exercise was especially valuable for students who had to take the courses online because of the COVID-19 pandemic. The effects of the new courses with the educational materials were empirically proven in terms of quantitative and qualitative perspectives.

In the study of spatiotemporal data visualization, compression and morphing of density maps are challenging tasks. Many existing methods require adjustment of multiple parameters and rich experience, but still cannot get accuracy or smooth results. In this paper, we propose a GAN-based method (LatentMap) to explore the latent space of density maps, which is an end-to-end method. First, we find that small latent codes can be used as the results of compression, which can greatly save transmission time in a front-end system. We collect density maps to make a dataset. Our model learns from the dataset and samples from the Gaussian distribution to encode and decode density maps. Second, based on the latent codes, we explore the smooth dynamic visualization of density maps, and our method can generate dynamic and smooth results. We show the results of our method in a variety of situations and evaluations from multiple aspects. The results demonstrate the effectiveness and practicality of our approach. Our method has practical applications, such as speed up front-end loading, completing or predicting stream data information and visual query.

We present a new visual analytics system, called CrossVis, that allows flexible exploration of multivariate data with heterogeneous data types. After presenting the design requirements, which were derived from prior collaborations with domain experts, we introduce key features of CrossVis beginning with a tabular data model that coordinates multiple linked views and performance enhancements that enable scalable exploration of complex data. Next, we introduce extensions to the parallel coordinates plot, which include new axis representations for numerical, temporal, categorical, and image data, an embedded bivariate axis option, dynamic selections, focus+context axis scaling, and graphical indicators of key statistical values. We demonstrate the practical effectiveness of CrossVis through two scientific use cases; one focused on understanding neural network image classifications from a genetic engineering project and another involving general exploration of a large and complex data set of historical hurricane observations. We conclude with discussions regarding domain expert feedback, future enhancements to address limitations, and the interdisciplinary process used to design CrossVis.

Immersive media such as virtual and augmented reality pose some interesting new challenges for non-photorealistic animation: we must not only balance the screen-space rules of a 2D visual style against 3D motion coherence, but also account for stereo spatialization and interactive camera movement, at a rate of 90 frames per second. We introduce two new real-time rendering techniques: MetaTexture, an example-based texturing method that adheres to the movement of 3D geometry while preserving the texture’s screen-space characteristics, and Edge Breakup, a method for roughening edges by warping with structured noise. We also describe a custom rendering pipeline featuring art-directable coloring, shadow filtering, and texture indication, and our approach to animating and rendering a painterly ocean in real time. We show how we have used these techniques to achieve the “moving illustration” style of the real-time immersive short film “Age of Sail”.