Virtual Reality Intelligent Hardware最新文献

Background

With the rapid development of Web3D, virtual reality, and digital twins, virtual trajectories and decision data considerably rely on the analysis and understanding of real video data, particularly in emergency evacuation scenarios. Correctly and effectively evacuating crowds in virtual emergency scenarios are becoming increasingly urgent. One good solution is to extract pedestrian trajectories from videos of emergency situations using a multi-target tracking algorithm and use them to define evacuation procedures.

Methods

To implement this solution, a trajectory extraction and optimization framework based on multi-target tracking is developed in this study. First, a multi-target tracking algorithm is used to extract and preprocess the trajectory data of the crowd in a video. Then, the trajectory is optimized by combining the trajectory point extraction algorithm and Savitzky–Golay smoothing filtering method. Finally, related experiments are conducted, and the results show that the proposed approach can effectively and accurately extract the trajectories of multiple target objects in real time.

Results

In addition, the proposed approach retains the real characteristics of the trajectories as much as possible while improving the trajectory smoothing index, which can provide data support for the analysis of pedestrian trajectory data and formulation of personnel evacuation schemes in emergency scenarios.

Conclusions

Further comparisons with methods used in related studies confirm the feasibility and superiority of the proposed framework.

Background

In recent years, with the rapid development of mobile Internet and Web3D technologies, a large number of web-based online 3D visualization applications have emerged. Web3D applications, including Web3D online tourism, Web3D online architecture, Web3D online education environment, Web3D online medical care, and Web3D online shopping are examples of these applications that leverage 3D rendering on the web. These applications have pushed the boundaries of traditional web applications that use text, sound, image, video, and 2D animation as their main communication media, and resorted to 3D virtual scenes as the main interaction object, enabling a user experience that delivers a strong sense of immersion. This paper approached the emerging Web3D applications that generate stronger impacts on people's lives through “real-time rendering technology”, which is the core technology of Web3D. This paper discusses all the major 3D graphics APIs of Web3D and the well-known Web3D engines at home and abroad and classify the real-time rendering frameworks of Web3D applications into different categories.

Results

Finally, this study analyzed the specific demand posed by different fields to Web3D applications by referring to the representative Web3D applications in each particular field.

Conclusions

Our survey results show that Web3D applications based on real-time rendering have in-depth sectors of society and even family, which is a trend that has influence on every line of industry.

Background

In computer vision, simultaneously estimating human pose, shape, and clothing is a practical issue in real life, but remains a challenging task owing to the variety of clothing, complexity of deformation, shortage of large-scale datasets, and difficulty in estimating clothing style.

Methods

We propose a multistage weakly supervised method that makes full use of data with less labeled information for learning to estimate human body shape, pose, and clothing deformation. In the first stage, the SMPL human-body model parameters were regressed using the multi-view 2D key points of the human body. Using multi-view information as weakly supervised information can avoid the deep ambiguity problem of a single view, obtain a more accurate human posture, and access supervisory information easily. In the second stage, clothing is represented by a PCAbased model that uses two-dimensional key points of clothing as supervised information to regress the parameters. In the third stage, we predefine an embedding graph for each type of clothing to describe the deformation. Then, the mask information of the clothing is used to further adjust the deformation of the clothing. To facilitate training, we constructed a multi-view synthetic dataset that included BCNet and SURREAL.

Results

The Experiments show that the accuracy of our method reaches the same level as that of SOTA methods using strong supervision information while only using weakly supervised information. Because this study uses only weakly supervised information, which is much easier to obtain, it has the advantage of utilizing existing data as training data. Experiments on the DeepFashion2 dataset show that our method can make full use of the existing weak supervision information for fine-tuning on a dataset with little supervision information, compared with the strong supervision information that cannot be trained or adjusted owing to the lack of exact annotation information.

Conclusions

Our weak supervision method can accurately estimate human body size, pose, and several common types of clothing and overcome the issues of the current shortage of clothing data.

Image denoising is an important topic in the digital image processing field. This paper theoretically studies the validity of the classical non-local mean filter (NLM) for removing Gaussian noise from a novel statistic perspective. By regarding the restored image as an estimator of the clear image from the statistical view, we gradually analyse the unbiasedness and effectiveness of the restored value obtained by the NLM filter. Then, we propose an improved NLM algorithm called the clustering-based NLM filter (CNLM) that derived from the conditions obtained through the theoretical analysis. The proposed filter attempts to restore an ideal value using the approximately constant intensities obtained by the image clustering process. Here, we adopt a mixed probability model on a prefiltered image to generate an estimator of the ideal clustered components. The experimental results show that our algorithm obtains considerable improvement in peak signal-to-noise ratio (PSNR) values and visual results when removing Gaussian noise. On the other hand, the considerable practical performance of our filter shows that our method is theoretically acceptable as it can effectively estimates ideal images.

Background

At present, the teaching of experiments in primary and secondary schools is affected by cost and security factors. The existing research on virtual-experiment platforms alleviates this problem. However, the lack of real experimental equipment and the use of a single channel to understand users’ intentions weaken these platforms operationally and degrade the naturalness of interactions. To slove the above problems,we propose an intelligent experimental container structure and a situational awareness algorithm,both of which are verified and then applied to a chemical experiment involving virtual-real fusion. First, acquired images are denoised in the visual channel, using maximum diffuse reflection chroma to remove overexposures. Second, container situational awareness is realized by segmenting the image liquid level and establishing a relation-fitting model. Then, strategies for constructing complete behaviors and making priority comparisons among behaviors are adopted for information complementarity and information independence, respectively. A multichannel intentional understanding model and an interactive paradigm fusing vision, hearing and touch are proposed. The results show that the designed experimental container and algorithm in a virtual chemical experiment platform can achieve a natural level of human-computer interaction, enhance the user's sense of operation, and achieve high user satisfaction.

In terrorist attack simulations, existing methods do not describe individual differences, which means different individuals will not have different behaviors. To address this problem, we propose a framework to model people’s heterogeneous behaviors in terrorist attack. For pedestrian, we construct an emotional model that takes into account its personality and visual perception. The emotional model is then combined with pedestrians' relationship networks to make the decision-making model. With the proposed decision-making model, pedestrian may have altruistic behaviors. For terrorist, a mapping model is developed to map its antisocial personality to its attacking strategy. The experiments show that the proposed algorithm can generate realistic heterogeneous behaviors that are consistent with existing psychological research findings.

Background

Gland cancer is a high-incidence disease endangering human health, and its early detection and treatment need efficient, accurate and objective intelligent diagnosis methods. In recent years, the advent of machine learning techniques has yielded satisfactory results in the intelligent gland cancer diagnosis based on clinical images, greatly improving the accuracy and efficiency of medical image interpretation while reducing the workload of doctors. The foci of this paper is to review, classify and analyze the intelligent diagnosis methods of imaging gland cancer based on machine learning and deep learning. To start with, the paper presents a brief introduction about some basic imaging principles of multi-modal medical images, such as the commonly used CT, MRI, US, PET, and pathology. In addition, the intelligent diagnosis methods of imaging gland cancer are further classified into supervised learning and weakly-supervised learning. Supervised learning consists of traditional machine learning methods like KNN, SVM, multilayer perceptron, etc. and deep learning methods evolving from CNN, meanwhile, weakly-supervised learning can be further categorized into active learning, semi-supervised learning and transfer learning. The state-of-the-art methods are illustrated with implementation details, including image segmentation, feature extraction, the optimization of classifiers, and their performances are evaluated through indicators like accuracy, precision and sensitivity. To conclude, the challenges and development trend of intelligent diagnosis methods of imaging gland cancer are addressed and discussed.

Background

This work aims to provide an overview of the Mixed Reality (MR) technology’s use in maritime industry for training purposes. Current training procedures cover a broad range of procedural operations for Life-Saving Appliances (LSA) lifeboats; however, several gaps and limitations have been identified related to the practical training that can be addressed through the use of MR. Augmented, Virtual and Mixed Reality applications are already used in various fields in maritime industry, but its full potential has not been yet exploited. SafePASS project aims to exploit MR advantages in the maritime training by introducing a relevant application focusing on use and maintenance of LSA lifeboats.

Methods

An MR Training application is proposed supporting the training of crew members in equipment usage and operation, as well as in maintenance activities and procedures. The application consists of the training tool that trains crew members on handling lifeboats, the training evaluation tool that allows trainers to assess the performance of trainees and the maintenance tool that supports crew members to perform maintenance activities and procedures on lifeboats. For each tool, an indicative session and scenario workflow are implemented, along with the main supported interactions of the trainee with the equipment.

Results

The application has been tested and validated both in lab environment and using a real LSA lifeboat, resulting to improved experience for the users that provided feedback and recommendations for further development. The application has also been demonstrated onboard a cruise ship, showcasing the supported functionalities to relevant stakeholders that recognized the added value of the application and suggested potential future exploitation areas.

Conclusions

The MR Training application has been evaluated as very promising in providing a user-friendly training environment that can support crew members LSA lifeboat operation and maintenance, while it is still subject to improvement and further expansion.

This work emphasizes the current research status of the urban Digital Twins to establish an intelligent spatiotemporal framework. A Geospatial Artificial Intelligent (GeoAI) system is developed based on the Geographic Information System and Artificial Intelligence. It integrates multi-video technology and Virtual City in urban Digital Twins. Besides, an improved small object detection model is proposed: YOLOv5-Pyramid, and Siamese network video tracking models, namely MPSiam and FSSiamese, are established. Finally, an experimental platform is built to verify the georeferencing correction scheme of video images. The experimental results show that the Multiply-Accumulate value of MPSiam is 0.5B, and that of ResNet50-Siam is 4.5B. Besides, the model is compressed by 4.8 times. The inference speed has increased by 3.3 times, reaching 83 Frames Per Second. 3% of the Average Expectation Overlap is lost. Therefore, the urban Digital Twins-oriented GeoAI framework established here has excellent performance for video georeferencing and target detection problems.

Background

Navigation assistance is very important for users when roaming in virtual reality scenes, however, the traditional navigation method requires users to manually request a map for viewing, which leads to low immersion and poor user experience.

Methods

To address this issue, first, we collected data when users need navigation assistance in a virtual reality environment, including various eye movement features such as gaze fixation, pupil size, and gaze angle, etc. After that, we used the Boostingbased XGBoost algorithm to train a prediction model, and finally used it to predict whether users need navigation assistance in a roaming task.

Results

After evaluating the performance of the model, the accuracy, precision, recall, and F1-score of our model reached about 95%. In addition, by applying the model to a virtual reality scene, an adaptive navigation assistance system based on the user’s real-time eye movement data was implemented.

Conclusions

Compared with traditional navigation assistance methods, our new adaptive navigation assistance could enable the user to be more immersive and effective during roaming in VR environment.