IEEE Computer Graphics and Applications最新文献

The continuous evolution of artificial intelligence and advanced algorithms capable of generating information from simplified input creates new opportunities for several scientific fields. Currently, the applicability of such technologies is limited to art and medical domains, but it can be applied to engineering domains to help the architects and urban planners design environmentally friendly solutions by proposing several alternatives in a short time. This work utilizes the image-inpainting algorithm for suggesting several alternative solutions to four European cities. In addition, this work suggests the utilization of a voting-based framework for finding the most preferred solution for each case study. The voting-based framework involves the participation of citizens and, as a result, decentralizes and democratizes the urban planning process. Finally, this research indicates the importance of deploying generative models in engineering applications by proving that generative AI models are capable of supporting the architects and urban planners in urban planning procedures.

Dashboards have arguably been the most used visualizations during the COVID-19 pandemic. They were used to communicate its evolution to national governments for disaster mitigation, to the public domain to inform about its status, and to epidemiologists to comprehend and predict the evolution of the disease. Each design had to be tailored for different tasks and to varying audiences-in many cases set up in a very short time due to the urgent need. In this article, we collect notable examples of dashboards and reflect on their use and design during the pandemic from a user-oriented perspective. We interview a group of researchers with varying visualization expertise who actively used dashboards during the pandemic as part of their daily workflow. We discuss our findings and compile a list of lessons learned to support future visualization researchers and dashboard designers.

Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. We demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network solving a simple convection problem.

HoloJig offers an interactive, speech-to-virtual reality (VR), VR experience that generates diverse environments in real time based on live spoken descriptions. Unlike traditional VR systems that rely on prebuilt assets, HoloJig dynamically creates personalized and immersive virtual spaces with depth-based parallax 3-D rendering, allowing users to define the characteristics of their immersive environment through verbal prompts. This generative approach opens up new possibilities for interactive experiences, including simulations, training, collaborative workspaces, and entertainment. In addition to speech-to-VR environment generation, a key innovation of HoloJig is its progressive visual transition mechanism, which smoothly dissolves between previously generated and newly requested environments, mitigating the delay caused by neural computations. This feature ensures a seamless and continuous user experience, even as new scenes are being rendered on remote servers.

This work explores the integration of generative AI models for automatically generating synthetic image-labeled data. Our approach leverages controllable diffusion models to generate synthetic variations of semantically labeled images. Synthetic datasets for semantic segmentation struggle to represent real-world subtleties, such as different weather conditions or fine details, typically relying on costly simulations and rendering. However, diffusion models can generate diverse images using input text prompts and guidance images, such as semantic masks. Our work introduces and tests a novel methodology for generating labeled synthetic images, with an initial focus on semantic segmentation, a demanding computer vision task. We showcase our approach in two distinct image segmentation domains, outperforming traditional computer graphics simulations in efficiently creating diverse datasets and training downstream models. We leverage generative models for crafting synthetically labeled images, posing the question: "Should I render or should AI generate?" Our results endorse a paradigm shift toward controlled generation models.

To facilitate comparative analysis of artificial intelligence (AI) and human paintings, we present a unified computational framework combining neural embedding and computational aesthetic features. We first exploit CLIP embedding to provide a projected overview for human and AI painting datasets, and we next leverage computational aesthetic metrics to obtain explainable features of paintings. On that basis, we design a visual analytics system that involves distribution discrepancy measurement for quantifying dataset differences and evolutionary analysis for comparing artists with AI models. Case studies comparing three AI-generated datasets with three human paintings datasets, and analyzing the evolutionary differences between authentic Picasso paintings and AI-generated ones, show the effectiveness of our framework.

Our society increasingly depends on intelligent systems to solve complex problems, ranging from recommender systems suggesting the next movie to watch to AI models assisting in medical diagnoses for hospitalized patients. With the iterative improvement of diagnostic accuracy and efficiency, AI holds significant potential to mitigate medical misdiagnoses by preventing numerous deaths and reducing an economic burden of approximately € 450 billion annually. However, a key obstacle to AI adoption lies in the lack of transparency, that is, many automated systems provide predictions without revealing the underlying processes. This opacity can hinder experts' ability to trust and rely on AI systems. Visual analytics (VA) provides a compelling solution by combining AI models with interactive visualizations. These specialized charts and graphs empower users to incorporate their domain expertise to refine and improve the models, bridging the gap between AI and human understanding. In this work, the author defines, categorizes, and explores how VA solutions can foster trust across the stages of a typical AI pipeline. The author proposes a design space for innovative visualizations and presents an overview of our previously developed VA dashboards, which support critical tasks within the various pipeline stages, including data processing, feature engineering, hyperparameter tuning, understanding, debugging, refining, and comparing models.

Remote assistance has become integral in today's work environments, enabling offsite experts to guide team members in need of assistance. One technology that has gained momentum is mixed reality (MR), combining virtual and augmented reality elements, bridging physical distances, and facilitating knowledge sharing. One critical area is the representation of remote experts. Understanding how different representations influence the learning process is essential for improving the shared experience. Effective representation can enhance communication, trust, and the perceived expertise of the remote expert, thereby improving the learning and collaboration experiences. This study focuses on evaluating how different representations affect various dimensions of collaboration. The study involved 57 participants and examined six different conditions: no visual representation, representation using emojis, representation using a cartoon avatar, representation using a robot avatar, representation using a realistic avatar, and live stream using video.

We present Meet-in-Style-a new approach to real-time stylization of live video streams using text prompts. In contrast to previous text-based techniques, our system is able to stylize input video at 30 fps on commodity graphics hardware while preserving structural consistency of the stylized sequence and minimizing temporal flicker. A key idea of our approach is to combine diffusion-based image stylization with a few-shot patch-based training strategy that can produce a custom image-to-image stylization network with real-time inference capabilities. Such a combination not only allows for fast stylization, but also greatly improves consistency of individual stylized frames compared to a scenario where diffusion is applied to each video frame separately. We conducted a number of user experiments in which we found our approach to be particularly useful in video conference scenarios enabling participants to interactively apply different visual styles to themselves (or to each other) to enhance the overall chatting experience.

Trame is an open-source, Python-based, scalable integration framework for visual analytics. It is the culmination of decades of work-by a large and active community-beginning with the creation of VTK, the growth of ParaView as a premier high-performance, client-server computing system, and more recently the creation of web tools, such as VTK.js and VTK.wasm. As an integration environment, trame relies on open-source standards and tools that can be easily combined into effective computing solutions. We have long recognized that impactful analytics tools must be ubiquitous-meaning they run on all major computing platforms-and integrate/interoperate easily with external packages, such as data systems and processing tools, application UI frameworks, and 2-D/3-D graphical libraries. In this article, we present the architecture and use of trame for applications ranging from simple dashboards to complex workflow-based applications. We also describe examples that readily incorporate external tools and run without coding changes on desktop, mobile, cloud, client-server, and interactive computing notebooks, such as Jupyter.