Machine learning with applications最新文献

Anomaly Detection (AD) is an important area to reliably detect malicious behavior and attacks on computer systems. Log data is a rich source of information about systems and thus provides a suitable input for AD. With the sheer amount of log data available today, for years Machine Learning (ML) and more recently Deep Learning (DL) have been applied to create models for AD. Especially when processing complex log data, DL has shown some promising results in recent research to spot anomalies. It is necessary to group these log lines into log-event sequences, to detect anomalous patterns that span over multiple log lines. This work uses a centralized approach using a Long Short-Term Memory (LSTM) model for AD as its basis which is one of the most important approaches to represent long-range temporal dependencies in log-event sequences of arbitrary length. Therefore, we use past information to predict whether future events are normal or anomalous. For the LSTM model we adapt a state of the art open source implementation called LogDeep. For the evaluation, we use a Hadoop Distributed File System (HDFS) data set, which is well studied in current research. In this paper we show that without padding, which is a commonly used preprocessing step that strongly influences the AD process and artificially improves detection results and thus accuracy in lab testing, it is not possible to achieve the same high quality of results shown in literature. With the large quantity of log data, issues arise with the transfer of log data to a central entity where model computation can be done. Federated Learning (FL) tries to overcome this problem, by learning local models simultaneously on edge devices and overcome biases due to a lack of heterogeneity in training data through exchange of model parameters and finally arrive at a converging global model. Processing log data locally takes privacy and legal concerns into account, which could improve coordination and collaboration between researchers, cyber security companies, etc., in the future. Currently, there are only few scientific publications on log-based AD which use FL. Implementing FL gives the advantage of converging models even if the log data are heterogeneously distributed among participants as our results show. Furthermore, by varying individual LSTM model parameters, the results can be greatly improved. Further scientific research will be necessary to optimize FL approaches.

Freezing of gait (FoG) is a debilitating and serious motor system complication of Parkinson's disease (PD) that may expose patients to frequent falls and life-threating injuries. Several artificial and machine learning methods have been proposed for the prediction of FoG based upon a limited time-duration of sensory data, However, most of the related work has been insufficiently trained and tested on smaller datasets compromising the generalizability of the models. Further, the proposed models provided a prediction at a lower rate (e.g., every 7.8 s). In response to the above shortcomings, we propose a novel variational mode decomposition (VMD) based deep learning that is capable of efficiently inferring the occurrence of FoG at a higher time-resolution (i.e., every sampling period of 7.8 ms) and with a subject-independent accuracy up to 98.8 % outperforming the state-of-the-art architectures and the standard LSTM models. The proposed model will enable the prompt detection of FoG episodes and support PD sufferers reducing the likelihood of falls.

Machine Learning models have the ability to streamline the process by which Youtube video comments are filtered between legitimate comments (ham) and spam. In order to integrate machine learning models into regular usage on media-sharing platforms, recent approaches have aimed to develop models trained on Youtube comments, which have emerged as valuable tools for the classification and have enabled the identification of spam content and enhancing user experience. In this paper, eight machine learning approaches are applied to spam detection for YouTube comments. The eight machine learning models include Gaussian Naive Bayes, logistic regression, K-nearest neighbors (KNN) classifier, multi-layer perceptron (MLP), support vector machine (SVM) classifier, random forest classifier, decision tree classifier, and voting classifier. All eight models perform very well, specifically random forest approach can achieve almost perfect performance with average precision of 100% and AUC-ROC of 0.9841. The computational complexity of the eight machine learning approaches are compared.

The application of Long Short-Term Memory (LSTM) models for streamflow predictions has been an area of rapid development, supported by advancements in computing technology, increasing availability of spatiotemporal data, and availability of historical data that allows for training data-driven LSTM models. Several studies have focused on improving the performance of LSTM models; however, few studies have assessed the applicability of these LSTM models across different hydroclimate regions. This study investigated the single-basin trained local (one model for each basin), multi-basin trained regional (one model for one region), and grand (one model for several regions) models for predicting daily streamflow in water-limited Great Basin (18 basins) and energy-limited New England (27 basins) regions in the United States using the CAMELS (Catchment Attributes and Meteorology for Large-sample Studies) data set. The results show a general pattern of higher accuracy in daily streamflow predictions from the regional model when compared to local or grand models for most basins in the New England region. For the Great Basin region, local models provided smaller errors for most basins and substantially lower for those basins with relatively larger errors from the regional and grand models. The evaluation of one-layer and three-layer LSTM network architectures trained with 1-day lag information indicates that the addition of model complexity by increasing the number of layers may not necessarily increase the model skill for improving streamflow predictions. Findings from our study highlight the strengths and limitations of LSTM models across contrasting hydroclimate regions in the United States, which could be useful for local and regional scale decisions using standalone or potential integration of data-driven LSTM models with physics-based hydrological models.

In this study, we adapt three spatial-temporal graph neural network models to the unique characteristics of crude oil, gold, and silver markets for forecasting purposes. It aims to be the first to (i) explore the potential of spatial-temporal graph neural networks family for price forecasting of these markets, (ii) examine the role of attention mechanism in improving forecasting accuracy, and (iii) integrate various sources of predictors for better performance. Specifically, we present three distinct models: Multivariate Time Series Graph Neural Networks with Temporal Attention and Learnable Adjacency matrix (MTGNN-TAttLA), Spatial Attention Graph with Temporal Convolutional Networks (SAG-TCN), and Attention-based Spatial-Temporal Graph Convolutional Networks (ASTGCN), to capture the intricate interplay of spatial and temporal dependencies within crude oil and precious metals markets. Moreover, the effectiveness of the attention mechanism in improving models' accuracies is shown. Our empirical results reveal remarkable prediction accuracy, with all three models outperforming conventional deep learning methods such as Temporal Convolutional Networks (TCN), long short-term memory networks (LSTM) and convolutional neural networks (CNN). The MTGNN-TAttLA model, enriched with a temporal attention mechanism, exhibits exceptional performance in predicting the direction of price movement in the WTI, Brent, and silver markets, while ASTGCN is the best-performing model for the gold market. Additionally, we observed that incorporating technical indicators from the crude oil and precious metal markets into the graph structure has improved the classification accuracy of spatial-temporal graph neural networks.

Infrared (IR) spectroscopic imaging is of potentially wide use in medical imaging applications due to its ability to capture both chemical and spatial information. This complexity of the data both necessitates using machine intelligence as well as presents an opportunity to harness a high-dimensionality data set that offers far more information than today’s manually-interpreted images. While convolutional neural networks (CNNs), including the well-known U-Net model, have demonstrated impressive performance in image segmentation, the inherent locality of convolution limits the effectiveness of these models for encoding IR data, resulting in suboptimal performance. In this work, we propose an INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation (INSTRAS). This novel model leverages the strength of the transformer encoders to segment IR breast images effectively. Incorporating skip-connection and transformer encoders, INSTRAS overcomes the issue of pure convolution models, such as the difficulty of capturing long-range dependencies. To evaluate the performance of our model and existing convolutional models, we conducted training on various encoder–decoder models using a breast dataset of IR images. INSTRAS, utilizing 9 spectral bands for segmentation, achieved a remarkable AUC score of 0.9788, underscoring its superior capabilities compared to purely convolutional models. These experimental results attest to INSTRAS’s advanced and improved segmentation abilities for IR imaging.

The problem of malicious software (malware) detection and classification is a complex task, and there is no perfect approach. There is still a lot of work to be done. Unlike most other research areas, standard benchmarks are difficult to find for malware detection. This paper aims to investigate recent advances in malware detection on MacOS, Windows, iOS, Android, and Linux using deep learning (DL) by investigating DL in text and image classification, the use of pre-trained and multi-task learning models for malware detection approaches to obtain high accuracy and which the best approach if we have a standard benchmark dataset. We discuss the issues and the challenges in malware detection using DL classifiers by reviewing the effectiveness of these DL classifiers and their inability to explain their decisions and actions to DL developers presenting the need to use Explainable Machine Learning (XAI) or Interpretable Machine Learning (IML) programs. Additionally, we discuss the impact of adversarial attacks on deep learning models, negatively affecting their generalization capabilities and resulting in poor performance on unseen data. We believe there is a need to train and test the effectiveness and efficiency of the current state-of-the-art deep learning models on different malware datasets. We examine eight popular DL approaches on various datasets. This survey will help researchers develop a general understanding of malware recognition using deep learning.

University ranking is a technique of measuring the performance of Higher Education Institutions (HEIs) by evaluating them on various criteria like student satisfaction, expenditure, research and teaching quality, citation count, grants, and enrolment. Ranking has been determined as a vital factor that helps students decide which institution to attend. Hence, universities seek to increase their overall rank and use these measures of success in their marketing communications and prominently place their ranked status on their institution's websites. Despite decades of research on ranking methods, a limited number of studies have leveraged predictive analytics and machine learning to rank universities. In this article, we collected 49 Canadian universities’ data for 2017–2021 and divided them based on Maclean's categories into Primarily Undergraduate, Comprehensive, and Medical/Doctoral Universities. After identifying the input and output components, we leveraged various feature engineering and machine learning techniques to predict the universities’ ranks. We used Pearson Correlation, Feature Importance, and Chi-Square as the feature engineering methods, and the results show that “student to faculty ratio,” “total number of citations”, and “total number of Grants” are the most important factors in ranking Canadian universities. Also, the Random Forest machine learning model for the “primarily undergraduate category,” the Voting classifier model for the “comprehensive category” and the Gradient Boosting model for the “medical/doctoral category” performed the best. The selected machine learning models were evaluated based on accuracy, precision, F1 score, and recall.

Detection and estimation of pothole dimensions is an essential step in road maintenance. Aging, heavy rainfall, traffic, and weak underlying layers may cause pavement potholes. Potholes can cause accidents when drivers lose control after hitting or swerving to avoid them, which may lead to injuries or fatal crashes. Also, potholes may result in property damages, such as flat tires, scrapes, dents, and leaks. Additionally, potholes are costly; for example, in the United States, potholes cost drivers about $3 Billion annually. Traditional ways of attending to potholes involve field surveys carried out by skilled personnel to determine their sizes for quantity and cost estimates. This process is expensive, prone to errors, subjectivity, unsafe, and time-consuming. Some authorities use sensor vehicles to carry out the surveys, a method that is accurate, safer, and faster than the traditional approach but much more expensive; therefore, not all authorities can afford them. To avoid these challenges, a modern, real-time, cost-effective approach is proposed to ensure the efficient and fast process of pothole maintenance. This paper presents a Deep Learning model trained using the You Only Look Once (YOLO) algorithm to capture potholes and estimate their dimensions and locations using only built-in vehicle technologies. The model attained 93.0 % precision, 91.6 % recall, 87.0 % F1-score, and 96.3 % mAP. A statistical analysis of the on-site test results indicates that the results are significant at a 5 % level, with a p-value of 0.037. This approach provides an economical and faster way of monitoring road surface conditions.

Determining, understanding, and predicting the so-called structure–property relation is an important task in many scientific disciplines, such as chemistry, biology, meteorology, physics, engineering, and materials science. Structure refers to the spatial distribution of, e.g., substances, material, or matter in general, while property is a resulting characteristic that usually depends in a non-trivial way on spatial details of the structure. Traditionally, forward simulations models have been used for such tasks. Recently, several machine learning algorithms have been applied in these scientific fields to enhance and accelerate simulation models or as surrogate models. In this work, we develop and investigate the applications of six machine learning techniques based on two different datasets from the domain of materials science: data from a two-dimensional Ising model for predicting the formation of magnetic domains and data representing the evolution of dual-phase microstructures from the Cahn–Hilliard model. We analyze the accuracy and robustness of all models and elucidate the reasons for the differences in their performances. The impact of including domain knowledge through tailored features is studied, and general recommendations based on the availability and quality of training data are derived from this.