Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01217-8
Zizhu Fan, Yijing Huang, Chao Xi, Cheng Peng, Shitong Wang
Fuzzy broad learning system (FBLS) is a newly proposed fuzzy system, which introduces Takagi–Sugeno fuzzy model into broad learning system. It has shown that FBLS has better nonlinear fitting ability and faster calculation speed than the most of fuzzy neural networks proposed earlier. At the same time, compared to other fuzzy neural networks, FBLS has fewer rules and lower cost of training time. However, label errors or missing are prone to appear in large-scale dataset, which will greatly reduce the performance of FBLS. Therefore, how to use limited label information to train a powerful classifier is an important challenge. In order to address this problem, we introduce Mean-Teacher model for the fuzzy broad learning system. We use the Mean-Teacher model to rebuild the weights of the output layer of FBLS, and use the Teacher–Student model to train FBLS. The proposed model is an implementation of semi-supervised learning which integrates fuzzy logic and broad learning system in the Mean-Teacher-based knowledge distillation framework. Finally, we have proved the great performance of Mean-Teacher-based fuzzy broad learning system (MT-FBLS) through a large number of experiments.
{"title":"Semi-supervised fuzzy broad learning system based on mean-teacher model","authors":"Zizhu Fan, Yijing Huang, Chao Xi, Cheng Peng, Shitong Wang","doi":"10.1007/s10044-024-01217-8","DOIUrl":"https://doi.org/10.1007/s10044-024-01217-8","url":null,"abstract":"<p>Fuzzy broad learning system (FBLS) is a newly proposed fuzzy system, which introduces Takagi–Sugeno fuzzy model into broad learning system. It has shown that FBLS has better nonlinear fitting ability and faster calculation speed than the most of fuzzy neural networks proposed earlier. At the same time, compared to other fuzzy neural networks, FBLS has fewer rules and lower cost of training time. However, label errors or missing are prone to appear in large-scale dataset, which will greatly reduce the performance of FBLS. Therefore, how to use limited label information to train a powerful classifier is an important challenge. In order to address this problem, we introduce Mean-Teacher model for the fuzzy broad learning system. We use the Mean-Teacher model to rebuild the weights of the output layer of FBLS, and use the Teacher–Student model to train FBLS. The proposed model is an implementation of semi-supervised learning which integrates fuzzy logic and broad learning system in the Mean-Teacher-based knowledge distillation framework. Finally, we have proved the great performance of Mean-Teacher-based fuzzy broad learning system (MT-FBLS) through a large number of experiments.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01220-z
Ruchi Agarwal, Sara Ahmed, Manoj Kumar
In tandem with the fast-growing technology, the issue of secure data transmission over the Internet has achieved increasing importance. In digital media, enclosing data in images is one of the most common methods for communicating confidential information. A novel reversible data hiding in the encrypted images scheme based on selective bin models is proposed in this paper. The scheme focuses on enhancing the embedding capacity while ensuring the security of images with the help of encryption and the proposed data hiding process. For data embedding, lossless compression is utilized and the image is classified into three bins. Then, marker bits are assigned to these bins for distinguishing between embeddable and non-embeddable regions. The proposed method shows a satisfactory embedding rate for smooth images as well as complex ones due to its selective bin approach. Also, the method is separable in nature, i.e., data extraction and image recovery can be performed independently. Furthermore, the experimental results demonstrate the strategy’s effectiveness when compared with others.
随着技术的飞速发展,在互联网上安全传输数据的问题也变得越来越重要。在数字媒体中,将数据封装在图像中是传递机密信息最常用的方法之一。本文提出了一种基于选择性 bin 模型的新型加密图像可逆数据隐藏方案。该方案的重点是提高嵌入能力,同时借助加密和拟议的数据隐藏过程确保图像的安全性。为了进行数据嵌入,采用了无损压缩技术,并将图像分为三个分区。然后,为这些分区分配标记位,以区分可嵌入和不可嵌入区域。由于采用了选择性分仓方法,所提出的方法对于平滑图像和复杂图像都能显示出令人满意的嵌入率。此外,该方法还具有可分性,即数据提取和图像复原可以独立进行。此外,与其他方法相比,实验结果证明了该策略的有效性。
{"title":"Selective bin model for reversible data hiding in encrypted images","authors":"Ruchi Agarwal, Sara Ahmed, Manoj Kumar","doi":"10.1007/s10044-024-01220-z","DOIUrl":"https://doi.org/10.1007/s10044-024-01220-z","url":null,"abstract":"<p>In tandem with the fast-growing technology, the issue of secure data transmission over the Internet has achieved increasing importance. In digital media, enclosing data in images is one of the most common methods for communicating confidential information. A novel reversible data hiding in the encrypted images scheme based on selective bin models is proposed in this paper. The scheme focuses on enhancing the embedding capacity while ensuring the security of images with the help of encryption and the proposed data hiding process. For data embedding, lossless compression is utilized and the image is classified into three bins. Then, marker bits are assigned to these bins for distinguishing between embeddable and non-embeddable regions. The proposed method shows a satisfactory embedding rate for smooth images as well as complex ones due to its selective bin approach. Also, the method is separable in nature, i.e., data extraction and image recovery can be performed independently. Furthermore, the experimental results demonstrate the strategy’s effectiveness when compared with others.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140011526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01228-5
Zubair Khan, Jie Yang
{"title":"Nonparametric K-means clustering-based adaptive unsupervised colour image segmentation","authors":"Zubair Khan, Jie Yang","doi":"10.1007/s10044-024-01228-5","DOIUrl":"https://doi.org/10.1007/s10044-024-01228-5","url":null,"abstract":"","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140419388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Unsupervised domain adaptation (UDA) is a well-explored domain in transfer learning, finding applications across various real-world scenarios. The central challenge in UDA lies in addressing the domain shift between training (source) and testing (target) data distributions. This study focuses on image classification tasks within UDA, where label spaces are shared, but the target domain lacks labeled samples. Our primary objective revolves around mitigating the domain discrepancies between the source and target domains, ultimately facilitating robust generalization in the target domains. Domain adaptation techniques have traditionally concentrated on the global feature distribution to minimize disparities. However, these methods often need to pay more attention to crucial, domain-specific subdomain information within identical classification categories, challenging achieving the desired performance without fine-grained data. To tackle these challenges, we propose a unified framework, Subdomain Adaptation via Correlation Alignment with Entropy Minimization, for unsupervised domain adaptation. Our approach incorporates three advanced techniques: (1) Local Maximum Mean Discrepancy, which aligns the means of local feature subsets, capturing intrinsic subdomain alignments often missed by global alignment, (2) correlation alignment aimed at minimizing the correlation between domain distributions, and (3) entropy regularization applied to target domains to encourage low-density separation between categories. We validate our proposed methods through rigorous experimental evaluations and ablation studies on standard benchmark datasets. The results consistently demonstrate the superior performance of our approaches compared to existing state-of-the-art domain adaptation methods.
无监督领域适应(UDA)是迁移学习中一个被广泛探索的领域,在现实世界的各种场景中都有应用。UDA 的核心挑战在于解决训练(源)和测试(目标)数据分布之间的领域转换问题。本研究的重点是 UDA 中的图像分类任务,在这种任务中,标签空间是共享的,但目标域缺乏有标签的样本。我们的主要目标是减少源域和目标域之间的差异,最终促进目标域的稳健泛化。领域适应技术传统上集中在全局特征分布上,以尽量减少差异。然而,这些方法往往需要更多地关注相同分类类别中关键的、特定域的子域信息,这就对在没有细粒度数据的情况下实现理想性能提出了挑战。为了应对这些挑战,我们提出了一个统一的框架,即通过相关性对齐与熵最小化实现子域适应,用于无监督域适应。我们的方法融合了三种先进技术:(1) 局部最大均值差异(Local Maximum Mean Discrepancy),它对局部特征子集的均值进行对齐,捕捉全局对齐经常忽略的内在子域对齐;(2) 相关性对齐(Correlation Alignment),旨在最小化域分布之间的相关性;(3) 熵正则化(entropy regularization),应用于目标域,鼓励类别之间的低密度分离。我们在标准基准数据集上进行了严格的实验评估和消融研究,验证了我们提出的方法。结果一致表明,与现有的最先进的域适应方法相比,我们的方法具有卓越的性能。
{"title":"Subdomain adaptation via correlation alignment with entropy minimization for unsupervised domain adaptation","authors":"Obsa Gilo, Jimson Mathew, Samrat Mondal, Rakesh Kumar Sandoniya","doi":"10.1007/s10044-024-01232-9","DOIUrl":"https://doi.org/10.1007/s10044-024-01232-9","url":null,"abstract":"<p>Unsupervised domain adaptation (UDA) is a well-explored domain in transfer learning, finding applications across various real-world scenarios. The central challenge in UDA lies in addressing the domain shift between training (source) and testing (target) data distributions. This study focuses on image classification tasks within UDA, where label spaces are shared, but the target domain lacks labeled samples. Our primary objective revolves around mitigating the domain discrepancies between the source and target domains, ultimately facilitating robust generalization in the target domains. Domain adaptation techniques have traditionally concentrated on the global feature distribution to minimize disparities. However, these methods often need to pay more attention to crucial, domain-specific subdomain information within identical classification categories, challenging achieving the desired performance without fine-grained data. To tackle these challenges, we propose a unified framework, Subdomain Adaptation via Correlation Alignment with Entropy Minimization, for unsupervised domain adaptation. Our approach incorporates three advanced techniques: (1) Local Maximum Mean Discrepancy, which aligns the means of local feature subsets, capturing intrinsic subdomain alignments often missed by global alignment, (2) correlation alignment aimed at minimizing the correlation between domain distributions, and (3) entropy regularization applied to target domains to encourage low-density separation between categories. We validate our proposed methods through rigorous experimental evaluations and ablation studies on standard benchmark datasets. The results consistently demonstrate the superior performance of our approaches compared to existing state-of-the-art domain adaptation methods.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01210-1
Motoshi Abe, Yuichiro Nomura, Takio Kurita
In recent years, the dimensionality reduction has become more important as the number of dimensions of data used in various tasks such as regression and classification has increased. As popular nonlinear dimensionality reduction methods, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) have been proposed. However, the former outputs only one low-dimensional space determined by the t-distribution and the latter is difficult to control the distribution of distance between each pair of samples in low-dimensional space. To tackle these issues, we propose novel t-SNE and UMAP extended by q-Gaussian distribution, called q-Gaussian-distributed stochastic neighbor embedding (q-SNE) and q-Gaussian-distributed uniform manifold approximation and projection (q-UMAP). The q-Gaussian distribution is a probability distribution derived by maximizing the tsallis entropy by escort distribution with mean and variance, and a generalized version of Gaussian distribution with a hyperparameter q. Since the shape of the q-Gaussian distribution can be tuned smoothly by the hyperparameter q, q-SNE and q-UMAP can in- tuitively derive different embedding spaces. To show the quality of the proposed method, we compared the visualization of the low-dimensional embedding space and the classification accuracy by k-NN in the low-dimensional space. Empirical results on MNIST, COIL-20, OliverttiFaces and FashionMNIST demonstrate that the q-SNE and q-UMAP can derive better embedding spaces than t-SNE and UMAP.
{"title":"Nonlinear dimensionality reduction with q-Gaussian distribution","authors":"Motoshi Abe, Yuichiro Nomura, Takio Kurita","doi":"10.1007/s10044-024-01210-1","DOIUrl":"https://doi.org/10.1007/s10044-024-01210-1","url":null,"abstract":"<p>In recent years, the dimensionality reduction has become more important as the number of dimensions of data used in various tasks such as regression and classification has increased. As popular nonlinear dimensionality reduction methods, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) have been proposed. However, the former outputs only one low-dimensional space determined by the t-distribution and the latter is difficult to control the distribution of distance between each pair of samples in low-dimensional space. To tackle these issues, we propose novel t-SNE and UMAP extended by q-Gaussian distribution, called q-Gaussian-distributed stochastic neighbor embedding (q-SNE) and q-Gaussian-distributed uniform manifold approximation and projection (q-UMAP). The q-Gaussian distribution is a probability distribution derived by maximizing the tsallis entropy by escort distribution with mean and variance, and a generalized version of Gaussian distribution with a hyperparameter q. Since the shape of the q-Gaussian distribution can be tuned smoothly by the hyperparameter q, q-SNE and q-UMAP can in- tuitively derive different embedding spaces. To show the quality of the proposed method, we compared the visualization of the low-dimensional embedding space and the classification accuracy by k-NN in the low-dimensional space. Empirical results on MNIST, COIL-20, OliverttiFaces and FashionMNIST demonstrate that the q-SNE and q-UMAP can derive better embedding spaces than t-SNE and UMAP.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01215-w
Eduardo K. Nakao, Alexandre L. M. Levada
The metric learning area studies methodologies to find the most appropriate distance function for a given dataset. It was shown that dimensionality reduction algorithms are closely related to metric learning because, in addition to obtaining a more compact representation of the data, such methods also implicitly derive a distance function that best represents similarity between a pair of objects in the collection. Principal Component Analysis is a traditional linear dimensionality reduction algorithm that is still widely used by researchers. However, its procedure faithfully represents outliers in the generated space, which can be an undesirable characteristic in pattern recognition applications. With this is mind, it was proposed the replacement of the traditional punctual approach by a contextual one based on the data samples neighborhoods. This approach implements a mapping from the usual feature space to a parametric feature space, where the difference between two samples is defined by the vector whose scalar coordinates are given by the statistical divergence between two probability distributions. It was demonstrated for some divergences that the new approach outperforms several existing dimensionality reduction algorithms in a wide range of datasets. Although, it is important to investigate the framework divergence sensitivity. Experiments using Total Variation, Renyi, Sharma-Mittal and Tsallis divergences are exhibited in this paper and the results evidence the method robustness.
{"title":"Information theory divergences in principal component analysis","authors":"Eduardo K. Nakao, Alexandre L. M. Levada","doi":"10.1007/s10044-024-01215-w","DOIUrl":"https://doi.org/10.1007/s10044-024-01215-w","url":null,"abstract":"<p>The metric learning area studies methodologies to find the most appropriate distance function for a given dataset. It was shown that dimensionality reduction algorithms are closely related to metric learning because, in addition to obtaining a more compact representation of the data, such methods also implicitly derive a distance function that best represents similarity between a pair of objects in the collection. Principal Component Analysis is a traditional linear dimensionality reduction algorithm that is still widely used by researchers. However, its procedure faithfully represents outliers in the generated space, which can be an undesirable characteristic in pattern recognition applications. With this is mind, it was proposed the replacement of the traditional punctual approach by a contextual one based on the data samples neighborhoods. This approach implements a mapping from the usual feature space to a parametric feature space, where the difference between two samples is defined by the vector whose scalar coordinates are given by the statistical divergence between two probability distributions. It was demonstrated for some divergences that the new approach outperforms several existing dimensionality reduction algorithms in a wide range of datasets. Although, it is important to investigate the framework divergence sensitivity. Experiments using Total Variation, Renyi, Sharma-Mittal and Tsallis divergences are exhibited in this paper and the results evidence the method robustness.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01216-9
Vlad-Rareş Dănăilă, Cătălin Buiu
In censored regression, the outcomes are a mixture of known values (uncensored) and open intervals (censored), meaning that the outcome is either known with precision or is an unknown value above or below a known threshold. The use of censored data is widespread, and correctly modeling it is essential for many applications. Although the literature on censored regression is vast, deep learning approaches have been less frequently applied. This paper proposes three loss functions for training neural networks on censored data using gradient backpropagation: the tobit likelihood, the censored mean squared error, and the censored mean absolute error. We experimented with three variations in the tobit likelihood that arose from different ways of modeling the standard deviation variable: as a fixed value, a reparametrization, and an estimation using a separate neural network for heteroscedastic data. The tobit model yielded better results, but the other two losses are simpler to implement. Another central idea of our research was that data are often censored and truncated simultaneously. The proposed losses can handle simultaneous censoring and truncation at arbitrary values from above and below.
{"title":"A deep learning approach to censored regression","authors":"Vlad-Rareş Dănăilă, Cătălin Buiu","doi":"10.1007/s10044-024-01216-9","DOIUrl":"https://doi.org/10.1007/s10044-024-01216-9","url":null,"abstract":"<p>In censored regression, the outcomes are a mixture of known values (uncensored) and open intervals (censored), meaning that the outcome is either known with precision or is an unknown value above or below a known threshold. The use of censored data is widespread, and correctly modeling it is essential for many applications. Although the literature on censored regression is vast, deep learning approaches have been less frequently applied. This paper proposes three loss functions for training neural networks on censored data using gradient backpropagation: the tobit likelihood, the censored mean squared error, and the censored mean absolute error. We experimented with three variations in the tobit likelihood that arose from different ways of modeling the standard deviation variable: as a fixed value, a reparametrization, and an estimation using a separate neural network for heteroscedastic data. The tobit model yielded better results, but the other two losses are simpler to implement. Another central idea of our research was that data are often censored and truncated simultaneously. The proposed losses can handle simultaneous censoring and truncation at arbitrary values from above and below.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01230-x
Abstract
Texture is a visual attribute largely used in many problems of image analysis. Many methods that use learning techniques have been proposed for texture discrimination, achieving improved performance over previous handcrafted methods. In this paper, we present a new approach that combines a learning technique and the complex network (CN) theory for texture analysis. This method takes advantage of the representation capacity of CN to model a texture image as a directed network and then uses the topological information of vertices to train a randomized neural network. This neural network has a single hidden layer and uses a fast learning algorithm to learn local CN patterns for texture characterization. Thus, we use the weights of the trained neural network to compose a feature vector. These feature vectors are evaluated in a classification experiment in four widely used image databases. Experimental results show a high classification performance of the proposed method compared to other methods, indicating that our approach can be used in many image analysis problems.
{"title":"Local complex features learned by randomized neural networks for texture analysis","authors":"","doi":"10.1007/s10044-024-01230-x","DOIUrl":"https://doi.org/10.1007/s10044-024-01230-x","url":null,"abstract":"<h3>Abstract</h3> <p>Texture is a visual attribute largely used in many problems of image analysis. Many methods that use learning techniques have been proposed for texture discrimination, achieving improved performance over previous handcrafted methods. In this paper, we present a new approach that combines a learning technique and the complex network (CN) theory for texture analysis. This method takes advantage of the representation capacity of CN to model a texture image as a directed network and then uses the topological information of vertices to train a randomized neural network. This neural network has a single hidden layer and uses a fast learning algorithm to learn local CN patterns for texture characterization. Thus, we use the weights of the trained neural network to compose a feature vector. These feature vectors are evaluated in a classification experiment in four widely used image databases. Experimental results show a high classification performance of the proposed method compared to other methods, indicating that our approach can be used in many image analysis problems.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1007/s10044-024-01213-y
Koffi Eddy Ihou, Nizar Bouguila
As an extension to the standard symmetric latent Dirichlet allocation topic model, we implement asymmetric Beta-Liouville as a conjugate prior to the multinomial and therefore propose the maximum a posteriori for latent Beta-Liouville allocation as an alternative to maximum likelihood estimator for models such as probabilistic latent semantic indexing, unigrams, and mixture of unigrams. Since most Bayesian posteriors, for complex models, are intractable in general, we propose a point estimate (the mode) that offers a much tractable solution. The maximum a posteriori hypotheses using point estimates are much easier than full Bayesian analysis that integrates over the entire parameter space. We show that the proposed maximum a posteriori reduces the three-level hierarchical latent Beta-Liouville allocation to two-level topic mixture as we marginalize out the latent variables. In each document, the maximum a posteriori provides a soft assignment and constructs dense expectation–maximization probabilities over each word (responsibilities) for accurate estimates. For simplicity, we present a stochastic at word-level online expectation–maximization algorithm as an optimization method for maximum a posteriori latent Beta-Liouville allocation estimation whose unnormalized reparameterization is equivalent to a stochastic collapsed variational Bayes. This implicit connection between the collapsed space and expectation–maximization-based maximum a posteriori latent Beta-Liouville allocation shows its flexibility and helps in providing alternative to model selection. We characterize efficiency in the proposed approach for its ability to simultaneously stream both large-scale data and parameters seamlessly. The performance of the model using predictive perplexities as evaluation method shows the robustness of the proposed technique with text document datasets.
{"title":"Big topic modeling based on a two-level hierarchical latent Beta-Liouville allocation for large-scale data and parameter streaming","authors":"Koffi Eddy Ihou, Nizar Bouguila","doi":"10.1007/s10044-024-01213-y","DOIUrl":"https://doi.org/10.1007/s10044-024-01213-y","url":null,"abstract":"<p>As an extension to the standard symmetric latent Dirichlet allocation topic model, we implement asymmetric Beta-Liouville as a conjugate prior to the multinomial and therefore propose the maximum a posteriori for latent Beta-Liouville allocation as an alternative to maximum likelihood estimator for models such as probabilistic latent semantic indexing, unigrams, and mixture of unigrams. Since most Bayesian posteriors, for complex models, are intractable in general, we propose a point estimate (the mode) that offers a much tractable solution. The maximum a posteriori hypotheses using point estimates are much easier than full Bayesian analysis that integrates over the entire parameter space. We show that the proposed maximum a posteriori reduces the three-level hierarchical latent Beta-Liouville allocation to two-level topic mixture as we marginalize out the latent variables. In each document, the maximum a posteriori provides a soft assignment and constructs dense expectation–maximization probabilities over each word (responsibilities) for accurate estimates. For simplicity, we present a stochastic at word-level online expectation–maximization algorithm as an optimization method for maximum a posteriori latent Beta-Liouville allocation estimation whose unnormalized reparameterization is equivalent to a stochastic collapsed variational Bayes. This implicit connection between the collapsed space and expectation–maximization-based maximum a posteriori latent Beta-Liouville allocation shows its flexibility and helps in providing alternative to model selection. We characterize efficiency in the proposed approach for its ability to simultaneously stream both large-scale data and parameters seamlessly. The performance of the model using predictive perplexities as evaluation method shows the robustness of the proposed technique with text document datasets.</p>","PeriodicalId":54639,"journal":{"name":"Pattern Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":3.9,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140009371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}