IEEE transactions on visualization and computer graphics最新文献

Reconstructing 3D scenes and synthesizing novel views from sparse input views is a highly challenging task. Recent advances in video diffusion models have demonstrated strong temporal reasoning capabilities, making them a promising tool for enhancing reconstruction quality under sparse-view settings. However, existing approaches are primarily designed for modest viewpoint variations, which struggle in capturing fine-grained details in close-up scenarios since input information is severely limited. In this paper, we present a diffusion-based framework, called CloseUpShot, for close-up novel view synthesis from sparse inputs via point-conditioned video diffusion. Specifically, we observe that pixel-warping conditioning suffers from severe sparsity and background leakage in close-up settings. To address this, we propose hierarchical warping and occlusion-aware noise suppression, enhancing the quality and completeness of the conditioning images for the video diffusion model. Furthermore, we introduce global structure guidance, which leverages a dense fused point cloud to provide consistent geometric context to the diffusion process, to compensate for the lack of globally consistent 3D constraints in sparse conditioning inputs. Extensive experiments on multiple datasets demonstrate that our method outperforms existing approaches, especially in close-up novel view synthesis, clearly validating the effectiveness of our design.

Artificial facial mimicry (AFM) is increasingly used to enhance social interaction with virtual agents in immersive virtual reality. However, its psychological and ethical implications remain insufficiently explored. This article conceptualizes AFM as an effective and embodied intervention, examining the role of emotional congruence, individual differences, and clinical vulnerability in shaping user responses. We further outline methodological directions involving physiological measures and embodied coordination. By framing AFM within affective computing and embodied cognition, this work contributes to the responsible design of emotionally adaptive virtual agents.

Indoor scene synthesis is essential for creative industries, recent advances in scene synthesis using diffusion and autoregressive models have shown promising results. However, existing models struggle to simultaneously achieve real-time performance, high visual fidelity, and flexible scene editability. To tackle these challenges, we propose MaskScene, a novel hierarchical conditional masked model for real-time 3D indoor scene synthesis and editing. Specifically, MaskScene introduces a hierarchical scene representation that explicitly encodes scene relationships, semantics, and tokenization. Based on this representation, we design a hierarchical conditional masked modeling architecture that enables parallel and iterative decoding, conditioned on both semantics and relationships. By masking local objects and leveraging the hierarchical structure of the scene, the model learns to infer and synthesize missing regions from partial observations, enabling rapid construction of 3D indoor environments that more accurately reflect real-world scenes. Compared to state-of-the-art methods, MaskScene achieves 80× faster generation speed and improves scene quality by 10%, while also supporting zero-shot editing, such as scene completion and rearrangement, without extra fine-tuning. Our project and dataset will be public.

We explore the potential of visualization to support musicians in instrument practice through real-time feedback and reflection on their playing. Musicians often struggle to observe patterns in their playing and interpret them with respect to their goals. Our premise is that these patterns can be made visible with interactive visualization: we can make the unhearable visible. However, understanding the design of such visualizations is challenging: the diversity of needs, including different instruments, skills, musical attributes, and genres, means that any single use case is unlikely to illustrate the broad potential and opportunities. To address this challenge, we conducted a design exploration where we created and iterated on 33 designs, each focusing on a subset of needs, for example, only one musical skill. Our designs are grounded in our own experience as musicians and the ideas and feedback of 18 musicians with various musical backgrounds and we evaluated them with 13 music learners and teachers. This paper presents the results of our exploration, focusing on a few example designs as instances of possible instrument practice visualizations. From our work, we draw design considerations that contribute to future research and products for visual instrument education. Supplemental materials are available at github.com/visvar/mila.

Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots that visually capture some properties of the data. Because visually determining the accuracy of these plots is challenging, researchers often use quality metrics to measure the projection's accuracy and faithfulness to the original data. One of the most commonly employed metrics, normalized stress, is sensitive to uniform scaling (stretching, shrinking) of the projection, despite this act not meaningfully changing anything about the projection. Another quality metric, the Kullback-Leibler (KL) divergence used in the popular t-Distributed Stochastic Neighbor Embedding (t-SNE) technique, is also susceptible to this scale sensitivity. We investigate the effect of scaling on stress and KL divergence analytically and empirically by showing just how much the values change and how this affects dimension reduction technique evaluations. We introduce a simple technique to make both metrics scale-invariant and show that it accurately captures expected behavior on a small benchmark.

Generating full-body humans in 360° has broad applications in digital entertainment, online education and art design. Existing works primarily rely on coarse conditions such as body pose to guide the generation, lacking detailed control over the synthesized results. Regarding this limitation, sketches offer a promising alternative as an expressive condition that enables more explicit and precise control. However, current sketch-based generation methods focus on faces or common objects, how to transfer sketches into 360° full-body humans remains unexplored. We first propose two straightforward strategies: adapting sketch-based 3D face generation to full-body human or lifting sketch-based 2D human generation to 3D format through a two-stage approach. Unfortunately, both methods result in unsatisfactory degradation of generation quality. To bridge this gap, in this work, we propose Sketch2Avatar, the first generative model to achieve 3D full-body human generation from hand-drawn sketches. Our model is capable of synthesizing sketch-aligned and 360°-consistent full-body human images by leveraging the geometry information extracted from sketches to guide the 3D representation generation and neural rendering. Specifically, we propose sketch-guided 3D representation generation to model the 3D human and maintain the alignment between input sketches and generated humans. Our transformer-based generator incorporates spatial feature guidance and latent modulation derived from sketches to produce high-quality 3D representations. Additionally, our designed body aware neural rendering utilizes 3D human body priors from sketches, simplifying the learning of articulated body poses and complex body shapes. To train and evaluate our model, we construct a large-scale dataset comprising approximately 19K 2D full-body human images and their corresponding sketches in a hand-drawn style. Experimental results demonstrate that our Sketch2Avatar can transfer hand-drawn sketches into photo-realistic 360° full-body human images with precise sketch-human alignment. Ablation studies further validate the effectiveness of our design choices. Our project is publicly available at: https://richardchen20.github.io/Sketch2Avatar.

Text-driven 3D editing is an emerging task that focuses on modifying scenes based on text prompts. Current methods often adapt pre-trained 2D image editors to multi-view observations, using specific strategies to combine information across views. However, these approaches still struggle with ensuring consistency across views, as they lack precise control over the sharing of information, resulting in edits with insufficient visual changes and blurry details. In this paper, we propose CoreEditor, a novel framework for consistent text-to-3D editing. At the core of our approach is a novel correspondence-constrained attention mechanism, which enforces structured interactions between corresponding pixels that are expected to remain visually consistent during the diffusion denoising process. Unlike conventional wisdom that relies solely on scene geometry, we enhance the correspondence by incorporating semantic similarity derived from the diffusion denoising process. This combined support from both geometry and semantics ensures a robust multi-view editing process. Additionally, we introduce a selective editing pipeline that enables users to choose their preferred edits from multiple candidates, creating a more flexible and user-centered 3D editing process. Extensive experiments demonstrate the effectiveness of CoreEditor, showing its ability to generate high-quality 3D edits, significantly outperforming existing methods.

Virtual reality is becoming increasingly popular, and modern haptic equipment, such as vibrotactile suits, haptic gloves, and force-feedback controllers, offers new means of interaction within virtual environments, significantly enhancing user experience. When interacting with virtual objects, combined visual and haptic feedback simulates the physical sensations of grasping, lifting, or moving real objects. This sensorimotor feedback is essential for inducing a sense of presence and agency, yet it remains challenging to reproduce in the absence of reliable haptic cues. In this study, we design and evaluate several haptic metaphors using combinations of vibrotactile design parameters to simulate the lifting effort associated with light to heavy objects. These parameters include primitive signals, intensity, spatial density, propagation, and temporal density. Our contribution is threefold. First, we propose a method for modulating perceived physical effort by extending signal intensity with spatial and temporal density, which together reflect the effort required to lift an object. Second, we present a user study in which participants compared haptic effects and ranked them according to perceived lifting effort, comfort, and confidence, allowing us to assess the influence of each parameter. Third, we report the results of a second study in which participants evaluated vibrotactile effects when lifting different virtual objects. The findings confirm the importance of intensity and spatial density, as well as the influence of graphical representation on perceived effort. This research provides practical insights for designing haptic-enabled virtual reality systems and offers guidance for developers seeking to create more expressive and believable vibrotactile interactions.

Machine Unlearning (MU) aims to remove target training data from a trained model so that the removed data no longer influences the model's behavior, fulfilling "right to be forgotten" obligations under data privacy laws. Yet, we observe that researchers in this rapidly emerging field face challenges in analyzing and understanding the behavior of different MU methods, especially in terms of three fundamental principles in MU: accuracy, efficiency, and privacy. Consequently, they often rely on aggregate metrics and ad-hoc evaluations, making it difficult to accurately assess the trade-offs between methods. To f ill this gap, we introduce a visual analytics system, Unlearning Comparator, designed to facilitate the systematic evaluation of MU methods. Our system supports two important tasks in the evaluation process: model comparison and attack simulation. First, it allows the user to compare the behaviors of two models, such as a model generated by a certain method and a retrained baseline, at class-, instance-, and layer-levels to better understand the changes made after unlearning. Second, our system simulates membership inference attacks (MIAs) to evaluate the privacy of a method, where an attacker attempts to determine whether specific data samples were part of the original training set. We evaluate our system through a case study visually analyzing prominent MU methods and demonstrate that it helps the user not only understand model behaviors but also gain insights that can inform the improvement of MU methods.

This short paper presents a general approach for computing robust Wasserstein barycenters [2], [80], [81] of persistence diagrams. The classical method consists in computing assignment arithmetic means after finding the optimal transport plans between the barycenter and the persistence diagrams. However, this procedure only works for the transportation cost related to the $q$-Wasserstein distance $W_{q}$ when $q=2$. We adapt an alternative fixed-point method [76] to compute a barycenter diagram for generic transportation costs ($q gt 1$), in particular those robust to outliers , $q in (1,2)$. We show the utility of our work in two applications : (i) the clustering of persistence diagrams on their metric space and (ii) the dictionary encoding of persistence diagrams [73]. In both scenarios, we demonstrate the added robustness to outliers provided by our generalized framework. Our Python implementation is available at this address: https://github.com/Keanu-Sisouk/RobustBarycenter.