Annals of Data Science最新文献

With digitisation globally on the rise, corporates are compelled to better understand the usage of their websites. In doing so, corporates will be empowered to better understand consumers, and make necessary adjustments to ultimately improve the corporate’s stance in the competitive global landscape of this modern age. However, the online website visit data has proven to be highly complex, big in data volume, and highly transactional with users expressing unique behaviours. Thus, extracting insight can be a complex problem to solve. This study aimed to employ unsupervised machine learning models to identify the intentions behind the visits on the observed website. The data studied was sourced from the Google Analytics tracking tool that was deployed on a corporate informative website. The study employed a k-means, hierarchical and dbscan unsupervised machine learning models to understand the intents behind visitors on the studied website. All three models detected five major intents that were expressed within the observed data. The intents identified were labelled as “accidentals”, “drop-offs”, “engrossed”, “get-in-touch” and “seekers”. On the observed data, all three unsupervised machine learning methods have performed well. However, in the context of the study, which investigated the intents that drove online visits, the hierarchical clustering method yielded superior results by maintaining the best balance between cluster homogeneity (stronger silhouette coefficients) and cluster size.

In light of the rapid technological advancements witnessed in recent decades, numerous disciplines have been inundated with voluminous datasets characterized by multimodality, heavy-tailed distributions, and prevalent missing information. Consequently, the task of effectively modeling such intricate data poses a formidable yet indispensable challenge. This paper endeavors to address this challenge by introducing a novel finite mixture model predicated upon the generalized t distribution, tailored specifically to accommodate two-sided censored observations, thereby establishing a foundational framework for modeling this complex data structure. To facilitate parameter estimation within this model, we devise a variant of the EM-type algorithm, amalgamating the profile likelihood approach with the classical Expectation Conditional Maximization algorithm. Notably, this hybridized methodology affords analytical expressions in the E-step and a tractable M-step, thereby substantially enhancing computational expediency and efficiency. Furthermore, we furnish closed-form expressions delineating the observed information matrix, pivotal for approximating the asymptotic covariance matrix of the MLEs within this mixture model. To empirically evaluate the efficacy of the proposed algorithm, a series of simulation studies are conducted, demonstrating promising performance across various artificial datasets. Additionally, the practical applicability of the proposed methodology is elucidated through its deployment on two real-world datasets, thereby underscoring its feasibility and utility in practical settings.

Hyperspectral image classification involves assigning pixels or regions within a hyperspectral image to specific classes or categories based on the spectral information captured across multiple bands. Traditional method faces several challenges such as High Dimensionality, Scalability, Spectral Variability, as well as Limited Contextual Information. Hence to solve these issues a novel Gated Graph Attention-based Crossover Snake (GGA-CS) algorithm is proposed for classifying hyperspectral images. In this work, a Graph Neural Network (GNN) is employed to capture both spectral and spatial relationships between pixels, and a gated attention mechanism is utilized to enhance specific spectral bands. After the training process, a crossover-based snake optimization is applied that tuned the parameter and obtain classification output of GNN and adjust the pixels to enhance the performances of GGA-CS method. The study is validated on diverse datasets namely the Indian Pines dataset, the University of Pavia dataset, as well as Salinas dataset. The evaluation of the GGA-CS method’s performance includes assessing its effectiveness using key metrics. Comparisons with state-of-the-art methods are conducted to gauge its efficacy in hyperspectral image classification, as demonstrated by experimental results.

This paper presents a novel nonlinear binary classification method, namely the kernel-free reduced quadratic surface support vector machine with 0-1 loss function and L(_{p})-norm regularization (L(_p)-RQSSVM(_{0/1})). It uses kernel-free trick aimed at finding a reduced quadratic surface to separate samples, without considering the cross terms in quadratic form. This saves computational costs and provides better interpretability than methods using kernel functions. In addition, adding the 0-1 loss function and L(_p)-norm regularization to construct our L(_p)-RQSSVM(_{0/1}) enables sample sparsity and feature sparsity. The support vector (SV) of L(_p)-RQSSVM(_{0/1}) is defined, and it is derived that all SVs fall on the support hypersurfaces. Moreover, the optimality condition is explored theoretically, and a new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is used to solve our L(_p)-RQSSVM(_{0/1}) on the selected working set. The computational complexity and convergence of the algorithm are discussed. Furthermore, numerical experiments demonstrate that our L(_p)-RQSSVM(_{0/1}) achieves better classification accuracy, less SVs, and higher computational efficiency than other methods on most datasets. It also has feature sparsity under certain conditions.

Developing effective methodologies for territory design and relativity estimation is crucial in auto insurance rate filings and reviews. This study introduces a novel approach utilizing fuzzy clustering to enhance the design process of territories for auto insurance rate-making and regulation. By adopting a soft clustering method, we aim to overcome the limitations of traditional hard clustering techniques and improve the assessment of territory risk. Furthermore, we employ non-negative sparse matrix approximation techniques to refine the estimates of risk relativities for basic rating units. This method promotes sparsity in the fuzzy membership matrix by eliminating small membership values, leading to more robust and interpretable results. We also compare the outcomes with those obtained using non-negative sparse principal component analysis, a technique explored in our previous research. Integrating fuzzy clustering with non-negative sparse matrix decomposition offers a promising approach for auto insurance rate filings. The combined methodology enhances decision-making and provides sparse estimates, which can be advantageous in various data science applications where fuzzy clustering is relevant.

Partial label learning (PLL) is a particular problem setting within weakly supervised learning. In PLL, each sample corresponds to a candidate label set in which only one label is true. However, in some practical application scenarios, the emergence of label noise can make some candidate sets lose their true labels, leading to a decline in model performance. In this work, a robust training strategy for PLL, derived from the joint training with co-regularization (JoCoR), is proposed to address this issue in PLL. Specifically, the proposed approach constructs two separate PLL models and a joint loss. The joint loss consists of not only two PLL losses but also a co-regularization term measuring the disagreement of the two models. By automatically selecting samples with small joint loss and using them to update the two models, our proposed approach is able to filter more and more suspected samples with noise candidate label sets. Gradually, the robustness of the PLL models to label noise strengthens due to the reduced disagreement of the two models. Experiments are conducted on two state-of-the-art PLL models using benchmark datasets under various noise levels. The results show that the proposed method can effectively stabilize the training process and reduce the model's overfitting to noisy candidate label sets.

Agriculture is the primary source of food, fuel, and raw materials and is vital to any country’s economy. Farmers, the backbone of agriculture, primarily rely on instinct to determine what crops to plant in any given season. They are comfortable following customary farming practices and standards and are oblivious to the fact that crop yield is highly dependent on current environmental and soil conditions. Crop recommendations involve multifaceted factors such as weather, soil quality, crop production, market demand, and prices, making it crucial for farmers to make well-informed decisions. An improper or imprudent crop recommendation can affect them, their families, and the entire agricultural sector. Modern technologies like artificial intelligence, machine learning, and data science have emerged as efficient solutions to combat issues like declining crop production and lower profits. This research proposes a Smart Crop Recommendation framework that leverages machine learning to empower farmers to make informed decisions about optimal crop selection. The framework consists of two phases: crop filtration and yield prediction. Crops are filtered in the first phase using an artificial neural network based on local input parameters. The second phase estimates yield for filtered crops, considering the season, farm area, and location data. The final recommendation provides farmers with crops aimed at maximizing profit. The remarkable 99.10% accuracy of the framework is demonstrated through experimentation using artificial neural networks and the 0.99 (text {R}^{text {2}}) error metric for the random forest. The uniqueness of this framework lies in its distinctive focus on the farm level and its consideration of the challenges and various agricultural features that change over time. The experimental results affirm the effectiveness of the framework, and its lightweight nature enhances its practicality, making it an efficient real-time recommendation solution.

Observations indicate that the distributions of stock returns in financial markets usually do not conform to normal distributions, but rather exhibit characteristics of high peaks, fat tails and biases. In this work, we assume that the effects of events or information on prices obey normal distribution, while financial markets often overreact or underreact to events or information, resulting in non normal distributions of stock returns. Based on the above assumptions, we for the first time propose a reaction function for a financial market reacting to events or information, and a model based on it to describe the distribution of real stock returns. Our analysis of the returns of China Securities Index 300 (CSI 300), the Standard & Poor’s 500 Index (SPX or S &P 500) and the Nikkei 225 Index (N225) at different time scales shows that financial markets often underreact to events or information with minor impacts, overreact to events or information with relatively significant impacts, and react slightly stronger to positive events or information than to negative ones. In addition, differences in financial markets and time scales of returns can also affect the shapes of the reaction functions.