Frontiers in Big Data最新文献

The concept of the "metaverse" has garnered significant attention recently, positioned as the "next frontier" of the internet. This emerging digital realm carries substantial economic and financial implications for both IT and non-IT industries. However, the integration and evolution of these virtual universes bring forth a multitude of intricate issues and quandaries that demand resolution. Within this research endeavor, our objective was to delve into and appraise the array of challenges, privacy concerns, and security issues that have come to light during the development of metaverse virtual environments in the wake of the COVID-19 pandemic. Through a meticulous review and analysis of literature spanning from January 2020 to December 2022, we have meticulously identified and scrutinized 29 distinct challenges, along with 12 policy, privacy, and security matters intertwined with the metaverse. Among the challenges we unearthed, the foremost were concerns pertaining to the costs associated with hardware and software, implementation complexities, digital disparities, and the ethical and moral quandaries surrounding socio-control, collectively cited by 43%, 40%, and 33% of the surveyed articles, respectively. Turning our focus to policy, privacy, and security issues, the top three concerns that emerged from our investigation encompassed the formulation of metaverse rules and principles, the encroachment of privacy threats within the metaverse, and the looming challenges concerning data management, all mentioned in 43%, 40%, and 33% of the examined literature. In summation, the development of virtual environments within the metaverse is a multifaceted and dynamically evolving domain, offering both opportunities and hurdles for researchers and practitioners alike. It is our aspiration that the insights, challenges, and recommendations articulated in this report will catalyze extensive dialogues among industry stakeholders, governmental bodies, and other interested parties concerning the metaverse's destiny and the world they aim to construct or bequeath to future generations.

With the popularization of big data technology, agricultural data processing systems have become more intelligent. In this study, a data processing method for farmland environmental monitoring based on improved Spark components is designed. It introduces the FAST-Join (Join critical filtering sampling partition optimization) algorithm in the Spark component for equivalence association query optimization to improve the operating efficiency of the Spark component and cluster. The experimental results show that the amount of data written and read in Shuffle by Spark optimized by the FAST-join algorithm only accounts for 0.958 and 1.384% of the original data volume on average, and the calculation speed is 202.11% faster than the original. The average data processing time and occupied memory size of the Spark cluster are reduced by 128.22 and 76.75% compared with the originals. It also compared the cluster performance of the FAST-join and Equi-join algorithms. The Spark cluster optimized by the FAST-join algorithm reduced the processing time and occupied memory size by an average of 68.74 and 37.80% compared with the Equi-join algorithm, which shows that the FAST-join algorithm can effectively improve the efficiency of inter-data table querying and cluster computing.

Cardiovascular diseases, such as heart attack and congestive heart failure, are the leading cause of death both in the United States and worldwide. The current medical practice for diagnosing cardiovascular diseases is not suitable for long-term, out-of-hospital use. A key to long-term monitoring is the ability to detect abnormal cardiac rhythms, i.e., arrhythmia, in real-time. Most existing studies only focus on the accuracy of arrhythmia classification, instead of runtime performance of the workflow. In this paper, we present our work on supporting real-time arrhythmic detection using convolutional neural networks, which take images of electrocardiogram (ECG) segments as input, and classify the arrhythmia conditions. To support real-time processing, we have carried out extensive experiments and evaluated the computational cost of each step of the classification workflow. Our results show that it is feasible to achieve real-time arrhythmic detection using convolutional neural networks. To further demonstrate the generalizability of this approach, we used the trained model with processed data collected by a customized wearable sensor from a lab setting, and the results shown that our approach is highly accurate and efficient. This research provides the potentials to enable in-home real-time heart monitoring based on 2D image data, which opens up opportunities for integrating both machine learning and traditional diagnostic approaches.

Numerous networks in the real world change with time, producing dynamic graphs such as human mobility networks and brain networks. Typically, the "dynamics on graphs" (e.g., changing node attribute values) are visible, and they may be connected to and suggestive of the "dynamics of graphs" (e.g., evolution of the graph topology). Due to two fundamental obstacles, modeling and mapping between them have not been thoroughly explored: (1) the difficulty of developing a highly adaptable model without solid hypotheses and (2) the ineffectiveness and slowness of processing data with varying granularity. To solve these issues, we offer a novel scalable deep echo-state graph dynamics encoder for networks with significant temporal duration and dimensions. A novel neural architecture search (NAS) technique is then proposed and tailored for the deep echo-state encoder to ensure strong learnability. Extensive experiments on synthetic and actual application data illustrate the proposed method's exceptional effectiveness and efficiency.

Satellite microwave sensors are well suited for monitoring landscape freeze-thaw (FT) transitions owing to the strong brightness temperature (TB) or backscatter response to changes in liquid water abundance between predominantly frozen and thawed conditions. The FT retrieval is also a sensitive climate indicator with strong biophysical importance. However, retrieval algorithms can have difficulty distinguishing the FT status of soils from that of overlying features such as snow and vegetation, while variable land conditions can also degrade performance. Here, we applied a deep learning model using a multilayer convolutional neural network driven by AMSR2 and SMAP TB records, and trained on surface (~0-5 cm depth) soil temperature FT observations. Soil FT states were classified for the local morning (6 a.m.) and evening (6 p.m.) conditions corresponding to SMAP descending and ascending orbital overpasses, mapped to a 9 km polar grid spanning a five-year (2016-2020) record and Northern Hemisphere domain. Continuous variable estimates of the probability of frozen or thawed conditions were derived using a model cost function optimized against FT observational training data. Model results derived using combined multi-frequency (1.4, 18.7, 36.5 GHz) TBs produced the highest soil FT accuracy over other models derived using only single sensor or single frequency TB inputs. Moreover, SMAP L-band (1.4 GHz) TBs provided enhanced soil FT information and performance gain over model results derived using only AMSR2 TB inputs. The resulting soil FT classification showed favorable and consistent performance against soil FT observations from ERA5 reanalysis (mean percent accuracy, MPA: 92.7%) and in situ weather stations (MPA: 91.0%). The soil FT accuracy was generally consistent between morning and afternoon predictions and across different land covers and seasons. The model also showed better FT accuracy than ERA5 against regional weather station measurements (91.0% vs. 86.1% MPA). However, model confidence was lower in complex terrain where FT spatial heterogeneity was likely beneath the effective model grain size. Our results provide a high level of precision in mapping soil FT dynamics to improve understanding of complex seasonal transitions and their influence on ecological processes and climate feedbacks, with the potential to inform Earth system model predictions.