Big Data Research最新文献

Energy consumption prediction has become an integral part of a smart and sustainable environment. With future demand forecasts, energy production and distribution can be optimized to meet the needs of the growing population. However, forecasting the demand of individual households is a challenging task due to the diversity of energy consumption patterns. Recently, it has become popular with artificial intelligence-based smart energy-saving designs, smart grid planning and social Internet of Things (IoT) based smart homes. Despite existing approaches for energy demand forecast, predominantly, such systems are based on one-step forecasting and have a short forecasting period. For resolving this issue and obtain high prediction accuracy, this study follows the prediction of household appliances' power in two phases. In the first phase, a long short-term memory (LSTM) based model is used to predict total generative active power for the coming 500 hours. The second phase employs a hybrid deep learning model that combines convolutional characteristics of neural network with LSTM for household electrical energy consumption forecasting of the week ahead utilizing Social IoT-based smart meter readings. Experimental results reveal that the proposed convolutional LSTM (ConvLSTM) architecture outperforms other models with the lowest root mean square error value of 367 kilowatts for weekly household power consumption.

The proliferation of geospatial applications has tremendously increased the variety, velocity, and volume of spatial data that data stores have to manage. Traditional relational databases reveal limitations in handling such big geospatial data, mainly due to their rigid schema requirements and limited scalability. Numerous NoSQL databases have emerged and actively serve as alternative data stores for big spatial data.

This study presents a framework, called GeoYCSB, developed for benchmarking NoSQL databases with geospatial workloads. To develop GeoYCSB, we extend YCSB, a de facto benchmark framework for NoSQL systems, by integrating into its design architecture the new components necessary to support geospatial workloads. GeoYCSB supports both microbenchmarks and macrobenchmarks and facilitates the use of real datasets in both. It is extensible to evaluate any NoSQL database, provided they support spatial queries, using geospatial workloads performed on datasets of any geometric complexity. We use GeoYCSB to benchmark two leading document stores, MongoDB and Couchbase, and present the experimental results and analysis. Finally, we demonstrate the extensibility of GeoYCSB by including a new dataset consisting of complex geometries and using it to benchmark a system with a wide variety of geospatial queries: Apache Accumulo, a wide-column store, with the GeoMesa framework applied on top.

Colocation pattern mining finds a set of features whose instances frequently appear nearby in the same geographical space. Most of the existing algorithms for colocation patterns find nearby objects by a user-provided single-distance threshold. The value of the distance threshold is data specific and choosing a suitable distance for a user is not easy. In most real-world scenarios, it is rather meant to define spatial proximity by a distance range. It also provides flexibility to observe the change in the colocation patterns with distance and interprets the result better. Algorithms for mining colocations with a single distance threshold cannot be applied directly to the range of distances due to the computational overhead. We identify several structural properties of the collocation patterns and use them to propose an efficient single-pass colocation mining algorithm for distance range query, namely $Range-CoMine$. We compare the performance of the $Range-CoMine$ with adapted versions of the famous Join-less colocation mining approach using both real-world and synthetic data sets and show that $Range-CoMine$ outperforms the other algorithms.

Many database applications, such as OnLine Analytical Processing (OLAP), web-based information extraction or scientific computation, need to select a subset of fields based on several user-defined filters. Developers of these applications require effective assembly methods for on-demand filtering and aggregation, which raises new challenges in deploying parallel computing components on top of columnar storage.

To efficiently generate qualified records, an asynchronous skipping strategy is presented to speed up filtering and decoding in the column-based storage. Concentrating on filtering-pushdown in parallel analytical workloads, we offer in-depth analysis on record assembly. We highlight the bottleneck of traditional record-wise assembling methods in the cases of evaluating analytical tasks on a nested schema. With a concurrent queue structure, an asynchronous skipping strategy is presented to evaluate column scan separately by a software pipeline involving an optionally different set of threads. We show how to intensively read the sequential blocks of each column, and how to effectively eliminate invalid payloads by integrating filtering-pushdown in an asynchronous I/O stack.

We implement a columnar storage supporting filtering-pushdown in nested schema. Our experiments are conducted on a de-facto standard benchmark using both variant-selectivity scans and ad-hoc queries. The results revealed that in parallel I/O-intensive workloads, our implementation improved the I/O performance of the state-of-the arts by 1.3X∼2.7X. Coupling the asynchronous strategy with filtering-pushdown, our implementation remarkably outperforms its competitors with heavyweight coding workloads on both HDD and SSD.

In this paper, we investigate properties and performance of synthetic random graph models with a built-in community structure. Such models are important for evaluating and tuning community detection algorithms that are unsupervised by nature. We propose ABCDe—a multi-threaded implementation of the ABCD (Artificial Benchmark for Community Detection) graph generator. We discuss the implementation details of the algorithm and compare it with both the previously available sequential version of the ABCD model and with the parallel implementation of the standard and extensively used LFR (Lancichinetti–Fortunato–Radicchi) generator. We show that ABCDe is more than ten times faster and scales better than the parallel implementation of LFR provided in NetworKit. Moreover, the algorithm is not only faster but random graphs generated by ABCD have similar properties to the ones generated by the original LFR algorithm, while the parallelized NetworKit implementation of LFR produces graphs that have noticeably different characteristics.

Subgraph matching aims to find similar substructures in a single graph according to a given query graph and is known as a basic query for graph data management. There exist many categories of subgraph matching solutions. Subgraph isomorphism, which is thought of an NP-complete problem, is an initial solution for the subgraph matching task. To speed up the procedure, graph simulation has been presented to match subgraphs with a polynomial complexity of time. Unfortunately, graph simulation usually loses topologies of matched subgraphs because of its loose restrictions. In this paper, we propose an approximation approach named kSGM (top-k Subraph Graph Matching) for subgraph matching based on twig patterns. First, we transform query graphs into twig patterns and match candidate substructures in graph data. Second, we present an optimized join strategy along with top-k mechanism, including join order selection based on cost evaluation and optimized pruning based on maximum/minimum possible score. Finally, we design experiments on real-life and synthetic graph data to evaluate the performance of our work. The results show that our proposed kSGM obviously reduces the time complexity and guarantee the correctness for answering the queries of subgraph matching compared to existing algorithms.

Recent advances in machine learning (ML) bring more opportunities for greater implementation of smart government construction. However, there are many challenges in terms of government data application due to the previous nonstandard records and man-made errors. In this paper, we propose a practical intelligent government complaint prediction (IGCP) framework that helps governments quickly respond to citizens' consultations and complaints via ML technologies. In addition, we put forward an automatic label correction method and demonstrate its effectiveness on the performance improvement of intelligent government complaint prediction task. Specifically, the central server collects the interaction records from users and departments and automatically integrates them by the label correction approach which is designed to evaluate the similarity between different labels in data, and merge highly similar labels and corresponding samples into their most similar category. Based on those refined data, the central server quickly generates accurate solutions to complaints through text classification algorithms. The main innovation of our approach is that we turn the task of government complaint distribution into a text classification problem which is uniformly coordinated by the central server, and employ the label correction approach to correct redundant labels for training better models based on limited complaint records. To explore the influences of our approach, we evaluate its performance on real-world government service records provided by our collaborator. The experimental results demonstrate the prediction task which uses the label correction algorithm achieves significant improvements on almost all metrics of the classifier.