Big Data Research最新文献

To support the various analysis application of big data, big data processing frameworks are designed to be highly configurable. However, for common users, it is difficult to tailor the configurable frameworks to achieve optimal performance for every application. Recently, many automatic tuning methods are proposed to configure these frameworks. In detail, these methods firstly build a performance prediction model through sampling configurations randomly and measuring the corresponding performance. Then, they conduct heuristic search in the configuration space based on the performance prediction model. For most frameworks, it is too expensive to build the performance model since it needs to measure the performance of large amounts of configurations, which cause too much overhead on data collection. In this paper, we propose a novel data-efficient method to build the performance model with little impact on prediction accuracy. Compared to the traditional methods, the proposed method can reduce the overhead of data collection because it can train the performance model with much less training examples. Specifically, the proposed method can actively sample the important examples according to the dynamic requirement of the performance model during the iterative model updating. Hence, it can make full use of the collected informative data and train the performance model with much less training examples. To sample the important training examples, we employ several virtual performance model to estimate the importance of all candidate configurations efficiently. Experimental results show that our method needs less training examples than traditional methods with little impact on prediction accuracy.

Huge amounts of data are produced inside an industrial production plant every minute. This data is getting more accessible by higher network and computing capabilities. This poses an opportunity to apply methods in real time to support the reliability of production machines. In theory every time series, that is currently monitored by for a breach of thresholds, can be extended with a forecast method. Classical approaches, such as ARIMA and Exponential Smoothing can be used for forecasting. To describe the signal and boost the forecast results we use a clustering method to group each unknown data stream in a seasonality class. This seasonality classes can be used for insight into intra and inter group behaviour between machines and add causality to factory wide correlations. We collected 10000 multiple day segments of multiple identical and different machines. We manually hand labelled the data segments for their seasonality pattern to compare and explain the clustering results. Classes, obtained through clustering, are used to adapt each single forecast model for every machine. For the forecast method we could show improved results by selecting the correct seasonality for each data stream.

The emerging neural topic models make topic modeling more easily adaptable and extendable in unsupervised text mining. However, the existing neural topic models are difficult to retain representative information of the documents within the learnt topic representation. Fortunately, Deep Mutual Information Estimation (DMIE), which maximizes the mutual information between input data and the hidden representations to learn a good representation of the input data. DMIE provides a new paradigm for neural topic modeling. In this paper, we propose a neural topic model which incorporates deep mutual information estimation, i.e., Neural Topic Modeling with Deep Mutual Information Estimation (NTM-DMIE). NTM-DMIE is a neural network method for topic learning which maximizes the mutual information between the input documents and their latent topic representation. To learn robust topic representation, we incorporate the discriminator to discriminate negative examples and positive examples via adversarial learning. Moreover, we use both global and local mutual information to preserve the rich information of the input documents in the topic representation. We evaluate NTM-DMIE on several metrics, including accuracy of text clustering, with topic representation, topic uniqueness and topic coherence. Compared to the existing methods, the experimental results show that NTM-DMIE can outperform in all the metrics on the four datasets.

Over the past decade, the rapid increase in connected devices has enabled the emergence of new digital ecosystems to provide new opportunities for monitoring and managing systems to optimize overall performance. With these connected environments, data collection and management become increasingly challenging. A significant number of works in the literature have addressed data collection and management based on different contexts (e.g., mobile ad hoc, Peer-2-Peer, and IoT networks). Today, a wired network uses all of these protocols simultaneously, thus highlighting the need to build a standard data collection and management framework that considers all potential user preferences. For this purpose, multi-objective clustering has been utilized as a promising solution to ensure the stability of connected devices during the collection and management of data. In this paper, we introduce a new multi-objective clustering (MOC) technique based on various criteria for cluster construction and head selection in connected environments. More precisely, the proposed solution is based hypergraphs to represent the connected environment and clusters according to similarities between heterogeneous devices. Then, a cross-sectional hypergraph algorithm is applied to select the cluster heads. Experiments conducted show that our solution outperforms the pioneering literature methods in terms of performance and effectiveness.