Visual Informatics最新文献

The digital twin brain (DTB) computing model from brain-inspired computing research is an emerging artificial intelligence technique, which is realized by a computational modeling approach of hardware and software. It can achieve various cognitive abilities and their synergistic mechanisms in a manner similar to the human brain. Given that the task of the DTB is to simulate the functions of the human brain, comparing the similarities and differences between the two is crucial. However, the visualization study of the DTB is still under-researched. Moreover, the complexity of the datasets (multilevel spatiotemporal granularity and different types of comparison tasks) presents new challenges to the analysis and exploration of visualization. Therefore, in this study, we proposed DTBVis, a visual analytics system that supports comparison tasks for the DTB. DTBVis supports iterative explorations from different levels and at different granularities. Combined with automatic similarity recommendation, and high-dimensional exploration, DTBVis can assist experts in understanding the similarities and differences between the DTB and the human brain, thus helping them adjust their model and enhance its functionality. The highest level of DTBVis shows an overview of the datasets from the brain, which is used for comparison and exploration of the function and structure of the DTB and the human brain. The medium level is used for the comparison and exploration of a designated brain region. The low level can analyze a designated brain voxel. We worked closely with experts of brain science and held regular seminars with them. Feedback from the experts indicates that our approach helps them conduct comparative studies of the DTB and human brain and make modeling adjustments of the DTB through intuitive visual comparisons and interactive explorations.

Purpose:

This paper aims to develop a navigation system based on mixed reality, which can display multimodal medical images in an immersive environment and help surgeons locate the target area and surrounding important tissues precisely.

Methods:

To be displayed properly in mixed reality, medical images are processed in this system. High-quality cerebral vessels and nerve fibers with proper colors are reconstructed and exported to mixed reality environment. Multimodal images and models are registered and fused, extracting their key information. The multiple processed images are fused with the real patient in the same coordinate system to guide the surgery.

Results:

The multimodal image system is designed and validated properly. In phantom experiments, the average error of preoperative registration is 1.003 mm and the standard deviation is 0.096 mm. The average proportion of well-registered areas is 94.9%. In patient experiments, the surgeons who participated in the experiments generally indicated that the system had excellent performance and great application prospect for neurosurgery.

Conclusion:

This article proposes a navigation system of multimodal images for neurosurgery based on mixed reality. Compared with other navigation methods, this system can help surgeons locate the target area and surrounding important tissues more precisely and rapidly.

Exploring flow features and patterns hidden behind the data has received extensive academic attention in flow visualization. In this paper, we introduce an importance-guided surface generation and exploration scheme to explore the features and their connections. The features are expressed as an importance field, which can either be derived from a scalar field or be specified as a flow pattern. Guided by the importance field, we sample a pool of seeding curves along the binormal direction and construct stream surfaces to fit the regions of high- importance values. Our scheme evaluates candidate seeding curves by collecting importance scores from the curve and corresponding streamlines. The candidate seeding curves are refined using the high-score segments to identify the optimal surfaces. Comparative visualization among different kinds of flow features across time steps can be easily derived for flow structure analysis. In order to reduce the visual complexity, we leverage SurfRiver to achieve clearer observation by flattening and aligning the surface. Finally, we apply our surface generation scheme guided by flow patterns and scalar fields to evaluate the effectiveness of the proposed tool.

Digital phenotyping is the characterization of human behavior patterns based on data from digital devices such as smartphones in order to gain insights into the users’ state and especially to identify ailments. To support supervised machine learning, digital phenotyping requires gathering data from study participants’ smartphones as they live their lives. Periodically, participants are then asked to provide ground truth labels about their health status. Analyzing such complex data is challenging due to limited contextual information and imperfect health/wellness labels. We propose INteractive PHOne-o-typing VISualization (INPHOVIS), an interactive visual framework for exploratory analysis of smartphone health data to study phone-o-types. Prior visualization work has focused on mobile health data with clear semantics such as steps or heart rate data collected using dedicated health devices and wearables such as smartwatches. However, unlike smartphones which are owned by over 85 percent of the US population, wearable devices are less prevalent thus reducing the number of people from whom such data can be collected. In contrast, the “low-level” sensor data (e.g., accelerometer or GPS data) supported by INPHOVIS can be easily collected using smartphones. Data visualizations are designed to provide the essential contextualization of such data and thus help analysts discover complex relationships between observed sensor values and health-predictive phone-o-types. To guide the design of INPHOVIS, we performed a hierarchical task analysis of phone-o-typing requirements with health domain experts. We then designed and implemented multiple innovative visualizations integral to INPHOVIS including stacked bar charts to show diurnal behavioral patterns, calendar views to visualize day-level data along with bar charts, and correlation views to visualize important wellness predictive data. We demonstrate the usefulness of INPHOVIS with walk-throughs of use cases. We also evaluated INPHOVIS with expert feedback and received encouraging responses.

Self-supervised learning aims to learn a universal feature representation without labels. To date, most existing self-supervised learning methods are designed and optimized for image classification. These pre-trained models can be sub-optimal for dense prediction tasks due to the discrepancy between image-level prediction and pixel-level prediction. To fill this gap, we aim to design an effective, dense self-supervised learning framework that directly works at the level of pixels (or local features) by taking into account the correspondence between local features. Specifically, we present dense contrastive learning (DenseCL), which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images. Compared to the supervised ImageNet pre-training and other self-supervised learning methods, our self-supervised DenseCL pre-training demonstrates consistently superior performance when transferring to downstream dense prediction tasks including object detection, semantic segmentation and instance segmentation. Specifically, our approach significantly outperforms the strong MoCo-v2 by 2.0% AP on PASCAL VOC object detection, 1.1% AP on COCO object detection, 0.9% AP on COCO instance segmentation, 3.0% mIoU on PASCAL VOC semantic segmentation and 1.8% mIoU on Cityscapes semantic segmentation. The improvements are up to 3.5% AP and 8.8% mIoU over MoCo-v2, and 6.1% AP and 6.1% mIoU over supervised counterpart with frozen-backbone evaluation protocol.

Code and models are available at: https://git.io/DenseCL

Reconstructing 3D models for single objects with complex backgrounds has wide applications like 3D printing, AR/VR, and so on. It is necessary to consider the tradeoff between capturing data at low cost and getting high-quality reconstruction results. In this work, we propose a voxel-based modeling pipeline with sparse RGB-D images to effectively and efficiently reconstruct a single real object without the geometrical post-processing operation on background removal. First, referring to the idea of VisualHull, useless and inconsistent voxels of a targeted object are clipped. It helps focus on the target object and rectify the voxel projection information. Second, a modified TSDF calculation and voxel filling operations are proposed to alleviate the problem of depth missing in the depth images. They can improve TSDF value completeness for voxels on the surface of the object. After the mesh is generated by the MarchingCube, texture mapping is optimized with view selection, color optimization, and camera parameters fine-tuning. Experiments on Kinect capturing dataset, TUM public dataset, and virtual environment dataset validate the effectiveness and flexibility of our proposed pipeline.

Digital learning is becoming increasingly important in the crisis COVID-19 and is widespread in most countries. The proliferation of smart devices and 5G telecommunications systems are contributing to the development of digital learning systems as an alternative to traditional learning systems. Digital learning includes blended learning, online learning, and personalized learning which mainly depends on the use of new technologies and strategies, so digital learning is widely developed to improve education and combat emerging disasters such as COVID-19 diseases. Despite the tremendous benefits of digital learning, there are many obstacles related to the lack of digitized curriculum and collaboration between teachers and students. Therefore, many attempts have been made to improve the learning outcomes through the following strategies: collaboration, teacher convenience, personalized learning, cost and time savings through professional development, and modeling. In this study, facial expressions and heart rates are used to measure the effectiveness of digital learning systems and the level of learners’ engagement in learning environments. The results showed that the proposed approach outperformed the known related works in terms of learning effectiveness. The results of this research can be used to develop a digital learning environment.

The Parallel Coordinate Plot (PCP) is a complex visual design commonly used for the analysis of high-dimensional data. Increasing data size and complexity may make it challenging to decipher and uncover trends and outliers in a confined space. A dense PCP image resulting from overlapping edges may cause patterns to be covered. We develop techniques aimed at exploring the relationship between data dimensions to uncover trends in dense PCPs. We introduce correlation glyphs in the PCP view to reveal the strength of the correlation between adjacent axis pairs as well as an interactive glyph lens to uncover links between data dimensions by investigating dense areas of edge intersections. We also present a subtraction operator to identify differences between two similar multivariate data sets and relationship-guided dimensionality reduction by collapsing axis pairs. We finally present a case study of our techniques applied to ensemble data and provide feedback from a domain expert in epidemiology.

We introduce a concept of episode referring to a time interval in the development of a dynamic phenomenon that is characterized by multiple time-variant attributes. A data structure representing a single episode is a multivariate time series. To analyse collections of episodes, we propose an approach that is based on recognition of particular patterns in the temporal variation of the variables within episodes. Each episode is thus represented by a combination of patterns. Using this representation, we apply visual analytics techniques to fulfil a set of analysis tasks, such as investigation of the temporal distribution of the patterns, frequencies of transitions between the patterns in episode sequences, and co-occurrences of patterns of different variables within same episodes. We demonstrate our approach on two examples using real-world data, namely, dynamics of human mobility indicators during the COVID-19 pandemic and characteristics of football team movements during episodes of ball turnover.