IEEE Computer Graphics and Applications最新文献

Large language models (LLMs) are now being applied to the tasks of visualization generation and understanding, demonstrating these models' ability to be "visually literate." On the generation side, LLMs have shown promise in powering natural languages' interfaces for visualization authoring while also suffering from usability and inconsistency issues. On the interpretation side, models (especially vision-language models) can answer basic questions about visualizations, synthesize visual and textual information, and detect misleading visual designs. However, models also tend to struggle with certain analytic tasks, and their takeaways from reading visualizations often differ from those of humans. We aim to both illuminate the state of the art in LLMs' visualization literacy and speculate on where such work may, and perhaps ought to, take us next.

Object recognition is a fundamental challenge in computer vision, particularly for fine-grained object classification, where classes differ in minor features. Improved fine-grained object classification requires a teaching system with numerous classes and instances of data. As the number of hierarchical levels and instances grows, debugging these models becomes increasingly complex. Moreover, different types of debugging tasks require varying approaches, explanations, and levels of detail. We present MuCHEx, a multimodal conversational system that blends natural language and visual interaction for interactive debugging of hierarchical object classification. Natural language allows users to flexibly express high-level questions or debugging goals without needing to navigate complex interfaces, while adaptive explanations surface only the most relevant visual or textual details based on the user's current task. This multimodal approach combines the expressiveness of language with the precision of direct manipulation, enabling context-aware exploration during model debugging.

Visual comparison of text embeddings is crucial for analyzing semantic differences and comparing embedding models. Existing methods fail to maintain visual consistency in comparative regions and lack AI-assisted analysis, leading to high cognitive loads and time-consuming exploration processes. In this article, we propose AnchorTextVis, a visual analytics approach based on AnchorMap-our dynamic projection algorithm balancing spatial quality and temporal coherence and large language models (LLMs) to preserve users' mental map and accelerate the exploration process. We introduce the use of comparable dimensionality reduction algorithms that maintain visual consistency, such as AnchorMap from our previous work and Joint t-SNE. Building on this foundation, we leverage LLMs to compare and summarize, offering users insights. For quantitative comparisons, we define two complementary metrics, Shared k-nearest neighbors (KNN) and Coordinate distance. Besides, we have also designed intuitive representation and rich interactive tools to compare clusters of texts and individual texts. We demonstrate the effectiveness and usefulness of our approach through three case studies and expert feedback.

Autonomous agents powered by large language models are transforming artificial intelligence (AI), creating an imperative for the visualization area. However, our field's focus on a human in the sensemaking loop raises critical questions about autonomy, delegation, and coordination for such agentic visualization that preserve human agency while amplifying analytical capabilities. This article addresses these questions by reinterpreting existing visualization systems with semiautomated or fully automatic AI components through an agentic lens. Based on this analysis, we extract a collection of design patterns for agentic visualization, including agentic roles, communication, and coordination. These patterns provide a foundation for future agentic visualization systems that effectively harness AI agents while maintaining human insight and control.

This paper presents MidSurfer, a novel parameter-free method for extracting mid-surfaces from segmented volumetric data. The method generates uniformly triangulated, smooth meshes that accurately capture structural features. The process begins with the Ridge Field Transformation step that transforms the segmented input data, followed by the Mid-Polyline Extraction Algorithm that works on individual volume slices. Based on the connectivity of components, this step can result in either single or multiple polyline segments that represent the structural features. These segments form a coherent series, creating a backbone of regularly spaced points representing the mid-surface. Subsequently, we employ a Polyline Zipper Algorithm for triangulation that connects these polyline segments across neighboring slices, yielding a detailed triangulated mid-surface mesh. Results show that this method outperforms previous techniques in versatility, simplicity, and accuracy. Our approach is publicly available as a ParaView plugin at https://github.com/kaust-vislab/MidSurfer.

Accessible design for some may still produce barriers for others. This tension, called access friction, creates challenges for both designers and end-users with disabilities. To address this, we present the concept of softerware, a system design approach that provides end users with agency to meaningfully customize and adapt interfaces to their needs. To apply softerware to visualization, we assembled 195 data visualization customization options centered on the barriers we expect users with disabilities will experience. We built a prototype that applies a subset of these options and interviewed practitioners for feedback. Lastly, we conducted a design probe study with blind and low vision accessibility professionals to learn more about their challenges and visions for softerware. We observed access frictions between our participant's designs and they expressed that for softerware's success, current and future systems must be designed with accessible defaults, interoperability, persistence, and respect for a user's perceived effort-to-outcome ratio.

Capturing indoor environments with 360° images provides a cost-effective method for creating immersive content. However, virtual staging - removing existing furniture and inserting new objects with realistic lighting - remains challenging. We present VISPI (Virtual Staging Pipeline for Single Indoor Panoramic Images), a framework that enables interactive restaging of indoor scenes from a single panoramic image. Our approach combines multi-task deep learning with real-time rendering to extract geometric, semantic, and material information from cluttered scenes. The system includes: i) a vision transformer that simultaneously predicts depth, normals, semantics, albedo, and material properties; ii) spherical Gaussian lighting estimation; iii) real-time editing for interactive object placement; iv) stereoscopic Multi-Center-Of-Projection generation for Head Mounted Display exploration. The framework processes input through two pathways: extracting clutter-free representations for virtual staging and estimating material properties including metallic and roughness signals. We evaluate VISPI on Structured3D and FutureHouse datasets, demonstrating applications in real estate visualization, interior design, and virtual environment creation.

In the field of digital orthodontics, dental models with complete roots are essential digital assets, particularly for visualization and treatment planning. However, intraoral scans typically capture only dental crowns, leaving roots missing. In this paper, we introduce a meticulously designed algorithmic pipeline to complete dental models while preserving crown geometry and mesh topology. Our pipeline begins with learning-based point cloud completion applied to existing dental crowns. We then reconstruct a complete tooth model, encompassing both the crown and root, to guide subsequent processing steps. Next, we restore the crown's original geometry and mesh topology using a strong Delaunay meshing structure; the correctness of this approach has been thoroughly established in existing literature. Finally, we optimize the transition region between crown and root using bi-harmonic smoothing. A key advantage of our approach is that the completed tooth model accurately maintains the geometry and mesh topology of the original crown, while also ensuring high-quality triangulation of dental roots.

A goal of data visualization is to advance the understanding of multiparameter, large-scale datasets. In astrophysics, scientists map celestial objects to understand the hierarchical structure of the universe. In biology, genetic sequences and biological characteristics uncover evolutionary relationships and patterns (e.g., variation within species and ecological associations). Our highly interdisciplinary project entitled "A Cosmic View of Life on Earth" adapts an immersive astrophysics visualization platform called OpenSpace to contextualize diverse biological data. Dimensionality reduction techniques harmonize biological information to create spatial representations in which data are interactively explored on flat screens and planetarium domes. Visualizations are enriched with geographic metadata, 3-D scans of specimens, and species-specific sonifications (e.g., bird songs). The "Cosmic View" project eases the dissemination of stories related to biological domains (e.g., insects, birds, mammals, and human migrations) and facilitates scientific discovery.

In mass spectrometry-based proteomics, experts usually project data onto a single set of reference sequences, overlooking the influence of common haplotypes (combinations of genetic variants inherited together from a parent). We recently introduced ProHap, a tool for generating customized protein haplotype databases. Here, we present ProHap Explorer, a visualization interface designed to investigate the influence of common haplotypes on the human proteome. It enables users to explore haplotypes, their effects on protein sequences, and the identification of noncanonical peptides in public mass spectrometry datasets. The design builds on well-established representations in biological sequence analysis, ensuring familiarity for domain experts while integrating novel interactive elements tailored to proteogenomic data exploration. User interviews with proteomics experts confirmed the tool's utility, highlighting its ability to reveal whether haplotypes affect proteins of interest. By facilitating the intuitive exploration of proteogenomic variation, ProHap Explorer supports research in personalized medicine and the development of targeted therapies.