Pub Date : 2023-09-08DOI: 10.1109/TBDATA.2023.3313031
Junfu Wang;Yuanfang Guo;Liang Yang;Yunhong Wang
Graph Neural Networks (GNNs) have been generalized to process the heterogeneous graphs by various approaches. Unfortunately, these approaches usually model the heterogeneity via various complicated modules. This article aims to propose a simple yet effective framework to assign adequate ability to the homogeneous GNNs to handle the heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Network (RE-GNN), which employs only one parameter per relation to embed the importance of distinct types of relations and node-type-specific self-loop connections. To optimize these relation embeddings and the model parameters simultaneously, a gradient scaling factor is proposed to constrain the embeddings to converge to suitable values. Besides, we interpret the proposed RE-GNN from two perspectives, and theoretically demonstrate that our RE-GCN possesses more expressive power than GTN (which is a typical heterogeneous GNN, and it can generate meta-paths adaptively). Extensive experiments demonstrate that our RE-GNN can effectively and efficiently handle the heterogeneous graphs and can be applied to various homogeneous GNNs.
{"title":"Enabling Homogeneous GNNs to Handle Heterogeneous Graphs via Relation Embedding","authors":"Junfu Wang;Yuanfang Guo;Liang Yang;Yunhong Wang","doi":"10.1109/TBDATA.2023.3313031","DOIUrl":"10.1109/TBDATA.2023.3313031","url":null,"abstract":"Graph Neural Networks (GNNs) have been generalized to process the heterogeneous graphs by various approaches. Unfortunately, these approaches usually model the heterogeneity via various complicated modules. This article aims to propose a simple yet effective framework to assign adequate ability to the homogeneous GNNs to handle the heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Network (RE-GNN), which employs only one parameter per relation to embed the importance of distinct types of relations and node-type-specific self-loop connections. To optimize these relation embeddings and the model parameters simultaneously, a gradient scaling factor is proposed to constrain the embeddings to converge to suitable values. Besides, we interpret the proposed RE-GNN from two perspectives, and theoretically demonstrate that our RE-GCN possesses more expressive power than GTN (which is a typical heterogeneous GNN, and it can generate meta-paths adaptively). Extensive experiments demonstrate that our RE-GNN can effectively and efficiently handle the heterogeneous graphs and can be applied to various homogeneous GNNs.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1697-1710"},"PeriodicalIF":7.2,"publicationDate":"2023-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44348282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graph classification, aiming at learning the graph-level representations for effective class assignments, has received outstanding achievements, which heavily relies on high-quality datasets that have balanced class distribution. In fact, most real-world graph data naturally presents a long-tailed form, where the head classes occupy much more samples than the tail classes, it thus is essential to study the graph-level classification over long-tailed data while still remaining largely unexplored. However, most existing long-tailed learning methods in visions fail to jointly optimize the representation learning and classifier training, as well as neglect the mining of the hard-to-classify classes. Directly applying existing methods to graphs may lead to sub-optimal performance, since the model trained on graphs would be more sensitive to the long-tailed distribution due to the complex topological characteristics. Hence, in this paper, we propose a novel long-tailed graph-level classification framework via Co llaborative M ulti- e xpert Learning (CoMe) to tackle the problem. To equilibrate the contributions of head and tail classes, we first develop balanced contrastive learning from the view of representation learning, and then design an individual-expert classifier training based on hard class mining. In addition, we execute gated fusion and disentangled knowledge distillation among the multiple experts to promote the collaboration in a multi-expert framework. Comprehensive experiments are performed on seven widely-used benchmark datasets to demonstrate the superiority of our method CoMe over state-of-the-art baselines.
{"title":"Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts","authors":"Si-Yu Yi;Zhengyang Mao;Wei Ju;Yong-Dao Zhou;Luchen Liu;Xiao Luo;Ming Zhang","doi":"10.1109/TBDATA.2023.3313029","DOIUrl":"https://doi.org/10.1109/TBDATA.2023.3313029","url":null,"abstract":"Graph classification, aiming at learning the graph-level representations for effective class assignments, has received outstanding achievements, which heavily relies on high-quality datasets that have balanced class distribution. In fact, most real-world graph data naturally presents a long-tailed form, where the head classes occupy much more samples than the tail classes, it thus is essential to study the graph-level classification over long-tailed data while still remaining largely unexplored. However, most existing long-tailed learning methods in visions fail to jointly optimize the representation learning and classifier training, as well as neglect the mining of the hard-to-classify classes. Directly applying existing methods to graphs may lead to sub-optimal performance, since the model trained on graphs would be more sensitive to the long-tailed distribution due to the complex topological characteristics. Hence, in this paper, we propose a novel long-tailed graph-level classification framework via \u0000<underline><b>Co</b></u>\u0000llaborative \u0000<underline><b>M</b></u>\u0000ulti-\u0000<underline><b>e</b></u>\u0000xpert Learning (CoMe) to tackle the problem. To equilibrate the contributions of head and tail classes, we first develop balanced contrastive learning from the view of representation learning, and then design an individual-expert classifier training based on hard class mining. In addition, we execute gated fusion and disentangled knowledge distillation among the multiple experts to promote the collaboration in a multi-expert framework. Comprehensive experiments are performed on seven widely-used benchmark datasets to demonstrate the superiority of our method CoMe over state-of-the-art baselines.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1683-1696"},"PeriodicalIF":7.2,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138138227","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-07DOI: 10.1109/TBDATA.2023.3313028
Chien-Tse Cheng;Yu-Hsun Lin;Chung-Shou Liao
Online users’ feedback has numerous text comments to enrich the review quality on mainstream platforms, such as Yelp and Google Maps. Reading through numerous review comments to speculate the important aspects is tedious and time-consuming. Apparently, there is a huge gap between the numerous commentary text and the crucial aspects for users’ preferences. In this study, we proposed a weakly supervised framework called Sequence to Commentary Aspect Score Estimation (Seq2CASE) to estimate the vital aspect scores from the review comments, since the ground truth of the aspect score is seldom available. The aspect score estimation from Seq2CASE is close to the actual aspect scoring; precisely, the average Mean Absolute Error (MAE) is less than 0.4 for a 5-point grading scale. The performance of Seq2CASE is comparable to or even better than the state-of-the-art supervised approaches in recommendation tasks. We expect this work to be a stepping stone that can inspire more unsupervised studies working on this important but relatively underexploited research.
{"title":"Seq2CASE: Weakly Supervised Sequence to Commentary Aspect Score Estimation for Recommendation","authors":"Chien-Tse Cheng;Yu-Hsun Lin;Chung-Shou Liao","doi":"10.1109/TBDATA.2023.3313028","DOIUrl":"10.1109/TBDATA.2023.3313028","url":null,"abstract":"Online users’ feedback has numerous text comments to enrich the review quality on mainstream platforms, such as Yelp and Google Maps. Reading through numerous review comments to speculate the important aspects is tedious and time-consuming. Apparently, there is a huge gap between the numerous commentary text and the crucial aspects for users’ preferences. In this study, we proposed a weakly supervised framework called Sequence to Commentary Aspect Score Estimation (Seq2CASE) to estimate the vital aspect scores from the review comments, since the ground truth of the aspect score is seldom available. The aspect score estimation from Seq2CASE is close to the actual aspect scoring; precisely, the average Mean Absolute Error (MAE) is less than 0.4 for a 5-point grading scale. The performance of Seq2CASE is comparable to or even better than the state-of-the-art supervised approaches in recommendation tasks. We expect this work to be a stepping stone that can inspire more unsupervised studies working on this important but relatively underexploited research.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1670-1682"},"PeriodicalIF":7.2,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-31DOI: 10.1109/TBDATA.2023.3310251
Meng Wang;Yanhao Yang;David Bindel;Kun He
Community is a universal structure in various complex networks, and community detection is a fundamental task for network analysis. With the rapid growth of network scale, networks are massive, changing rapidly, and could naturally be modeled as graph streams. Due to the limited memory and access constraint in graph streams, existing non-streaming community detection methods are no longer applicable. This raises an emerging need for online approaches. In this work, we consider the problem of uncovering the local community containing a few query nodes in graph streams, termed streaming local community detection. This new problem raised recently is more challenging for community detection, and only a few works address this online setting. Correspondingly, we design an online single-pass streaming local community detection approach. Inspired by the local property of communities, our method samples the local structure around the query nodes in graph streams and extracts the target community on the sampled subgraph using our proposed metric called approximate conductance. Comprehensive experiments show that our method remarkably outperforms the streaming baseline on both effectiveness and efficiency, and even achieves similar accuracy compared to the state-of-the-art non-streaming local community detection methods that use static and complete graphs.
{"title":"Streaming Local Community Detection Through Approximate Conductance","authors":"Meng Wang;Yanhao Yang;David Bindel;Kun He","doi":"10.1109/TBDATA.2023.3310251","DOIUrl":"10.1109/TBDATA.2023.3310251","url":null,"abstract":"Community is a universal structure in various complex networks, and community detection is a fundamental task for network analysis. With the rapid growth of network scale, networks are massive, changing rapidly, and could naturally be modeled as graph streams. Due to the limited memory and access constraint in graph streams, existing non-streaming community detection methods are no longer applicable. This raises an emerging need for online approaches. In this work, we consider the problem of uncovering the local community containing a few query nodes in graph streams, termed streaming local community detection. This new problem raised recently is more challenging for community detection, and only a few works address this online setting. Correspondingly, we design an online single-pass streaming local community detection approach. Inspired by the local property of communities, our method samples the local structure around the query nodes in graph streams and extracts the target community on the sampled subgraph using our proposed metric called approximate conductance. Comprehensive experiments show that our method remarkably outperforms the streaming baseline on both effectiveness and efficiency, and even achieves similar accuracy compared to the state-of-the-art non-streaming local community detection methods that use static and complete graphs.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"10 1","pages":"12-22"},"PeriodicalIF":7.2,"publicationDate":"2023-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89772759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-30DOI: 10.1109/TBDATA.2023.3310267
Xiaosai Huang;Jing Li;Jia Wu;Jun Chang;Donghua Liu
Aspect-level sentiment classification (ASC) seeks to reveal the emotional tendency of a designated aspect of a text. Some researchers have recently tried to exploit large amounts of document-level sentiment classification (DSC) data available to help improve the performance of ASC models through transfer learning. However, these studies often ignore the difference in sentiment distribution between document-level and aspect-level data without preprocessing the document-level knowledge. Our study provides a transfer learning with document-level data augmentation (TL-DDA) framework to transfer more accurate document-level knowledge to the ASC model by means of document-level data augmentation and attention fusion . First, we use document data selection and text concatenation to produce document-level data with various sentiment distributions. The augmented document data is then utilized for pre-training a well-designed DSC model. Finally, after attention adjustment , we fuse the word attention obtained from this DSC model into the ASC model. Results of experiments utilizing two publicly available datasets suggest that TL-DDA is reliable.
{"title":"Transfer Learning With Document-Level Data Augmentation for Aspect-Level Sentiment Classification","authors":"Xiaosai Huang;Jing Li;Jia Wu;Jun Chang;Donghua Liu","doi":"10.1109/TBDATA.2023.3310267","DOIUrl":"10.1109/TBDATA.2023.3310267","url":null,"abstract":"Aspect-level sentiment classification (ASC) seeks to reveal the emotional tendency of a designated aspect of a text. Some researchers have recently tried to exploit large amounts of document-level sentiment classification (DSC) data available to help improve the performance of ASC models through transfer learning. However, these studies often ignore the difference in sentiment distribution between document-level and aspect-level data without preprocessing the document-level knowledge. Our study provides a transfer learning with document-level data augmentation (TL-DDA) framework to transfer more accurate document-level knowledge to the ASC model by means of \u0000<italic>document-level data augmentation</i>\u0000 and \u0000<italic>attention fusion</i>\u0000. First, we use \u0000<italic>document data selection</i>\u0000 and \u0000<italic>text concatenation</i>\u0000 to produce document-level data with various sentiment distributions. The augmented document data is then utilized for pre-training a well-designed DSC model. Finally, after \u0000<italic>attention adjustment</i>\u0000, we \u0000<italic>fuse the word attention</i>\u0000 obtained from this DSC model into the ASC model. Results of experiments utilizing two publicly available datasets suggest that TL-DDA is reliable.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1643-1657"},"PeriodicalIF":7.2,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-30DOI: 10.1109/TBDATA.2023.3310254
Xingyu Cao;Xiangtao Zhang;Ce Zhu;Jiani Liu;Yipeng Liu
Tensor decomposition is widely used in feature extraction, data analysis, and other fields. As a means of tensor decomposition, the robust tensor power method based on tensor sketch (TS-RTPM) can quickly mine the potential features of tensor, but in some cases, its approximation performance is limited. In this paper, we propose a data-driven framework called TS-RTPM-Net, which improves the estimation accuracy of TS-RTPM by jointly training the TS value matrices with the RTPM initial matrices. It also uses two greedy initialization algorithms to optimize the TS location matrices. In addition, TS-RTPM-Net accelerates TS-RTPM by using fast power iteration modules. Comparative experiments on real-world datasets verify that TS-RTPM-Net outperforms TS-RTPM in terms of estimation accuracy, running speed, and memory consumption.
{"title":"TS-RTPM-Net: Data-Driven Tensor Sketching for Efficient CP Decomposition","authors":"Xingyu Cao;Xiangtao Zhang;Ce Zhu;Jiani Liu;Yipeng Liu","doi":"10.1109/TBDATA.2023.3310254","DOIUrl":"10.1109/TBDATA.2023.3310254","url":null,"abstract":"Tensor decomposition is widely used in feature extraction, data analysis, and other fields. As a means of tensor decomposition, the robust tensor power method based on tensor sketch (TS-RTPM) can quickly mine the potential features of tensor, but in some cases, its approximation performance is limited. In this paper, we propose a data-driven framework called TS-RTPM-Net, which improves the estimation accuracy of TS-RTPM by jointly training the TS value matrices with the RTPM initial matrices. It also uses two greedy initialization algorithms to optimize the TS location matrices. In addition, TS-RTPM-Net accelerates TS-RTPM by using fast power iteration modules. Comparative experiments on real-world datasets verify that TS-RTPM-Net outperforms TS-RTPM in terms of estimation accuracy, running speed, and memory consumption.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"10 1","pages":"1-11"},"PeriodicalIF":7.2,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Different from traditional vector-based fine-grained entity typing methods, the box-based method is more effective in capturing the complex relationships between entity mentions and entity types. The box-based fine-grained entity typing method projects entity types and entity mentions into high-dimensional box space, where entity types and entity mentions are embedded as d -dimensional hyperrectangles. However, the impacts of entity types are not considered during classification in high-dimensional box space, and the model cannot be optimized precisely when two boxes are completely separated or overlapped in high-dimensional box space. Based on the above shortcomings, an I mproved B ox E mbeddings (IBE) method for fine-grained entity typing is proposed in this work. The IBE not only introduces the impacts of entity types during classification in high-dimensional box space, but also proposes a distance based module to optimize the model precisely when two boxes are completely separated or overlapped in high-dimensional box space. Experimental results on four fine-grained entity typing datasets verify the effectiveness of the proposed IBE, demonstrating that IBE is a state-of-the-art method for fine-grained entity typing.
{"title":"Improved Box Embeddings for Fine-Grained Entity Typing","authors":"Yixiu Qin;Yizhao Wang;Jiawei Li;Shun Mao;He Wang;Yuncheng Jiang","doi":"10.1109/TBDATA.2023.3310239","DOIUrl":"10.1109/TBDATA.2023.3310239","url":null,"abstract":"Different from traditional vector-based fine-grained entity typing methods, the box-based method is more effective in capturing the complex relationships between entity mentions and entity types. The box-based fine-grained entity typing method projects entity types and entity mentions into high-dimensional box space, where entity types and entity mentions are embedded as \u0000<italic>d</i>\u0000-dimensional hyperrectangles. However, the impacts of entity types are not considered during classification in high-dimensional box space, and the model cannot be optimized precisely when two boxes are completely separated or overlapped in high-dimensional box space. Based on the above shortcomings, an \u0000<bold>I</b>\u0000mproved \u0000<bold>B</b>\u0000ox \u0000<bold>E</b>\u0000mbeddings (IBE) method for fine-grained entity typing is proposed in this work. The IBE not only introduces the impacts of entity types during classification in high-dimensional box space, but also proposes a distance based module to optimize the model precisely when two boxes are completely separated or overlapped in high-dimensional box space. Experimental results on four fine-grained entity typing datasets verify the effectiveness of the proposed IBE, demonstrating that IBE is a state-of-the-art method for fine-grained entity typing.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1631-1642"},"PeriodicalIF":7.2,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Activity pattern prediction is a critical part of urban computing, urban planning, intelligent transportation, and so on. Based on a dataset with more than 10 million GPS trajectory records collected by mobile sensors, this research proposed a CNN-BiLSTM-VAE-ATT-based encoder-decoder model for fine-grained individual activity sequence prediction. The model combines the long-term and short-term dependencies crosswise and also considers randomness, diversity, and uncertainty of individual activity patterns. The proposed results show higher accuracy compared to the ten baselines. The model can generate high diversity results while approximating the original activity patterns distribution. Moreover, the model also has interpretability in revealing the time dependency importance of the activity pattern prediction.
{"title":"PredLife: Predicting Fine-Grained Future Activity Patterns","authors":"Wenjing Li;Xiaodan Shi;Dou Huang;Xudong Shen;Jinyu Chen;Hill Hiroki Kobayashi;Haoran Zhang;Xuan Song;Ryosuke Shibasaki","doi":"10.1109/TBDATA.2023.3310241","DOIUrl":"10.1109/TBDATA.2023.3310241","url":null,"abstract":"Activity pattern prediction is a critical part of urban computing, urban planning, intelligent transportation, and so on. Based on a dataset with more than 10 million GPS trajectory records collected by mobile sensors, this research proposed a CNN-BiLSTM-VAE-ATT-based encoder-decoder model for fine-grained individual activity sequence prediction. The model combines the long-term and short-term dependencies crosswise and also considers randomness, diversity, and uncertainty of individual activity patterns. The proposed results show higher accuracy compared to the ten baselines. The model can generate high diversity results while approximating the original activity patterns distribution. Moreover, the model also has interpretability in revealing the time dependency importance of the activity pattern prediction.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1658-1669"},"PeriodicalIF":7.2,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Existing two-dimensional principal component analysis methods can only handle second-order tensors (i.e., matrices). However, with the advancement of technology, tensors of order three and higher are gradually increasing. This brings new challenges to dimensionality reduction. Thus, a multilinear method called MPCA was proposed. Although MPCA can be applied to all tensors, using the square of the F-norm makes it very sensitive to outliers. Several two-dimensional methods, such as Angle 2DPCA, have good robustness but cannot be applied to all tensors. We extend the robust Angle 2DPCA method to a multilinear method and propose Cosine Multilinear Principal Component Analysis (CosMPCA) for tensor representation. Our CosMPCA method considers the relationship between the reconstruction error and projection scatter and selects the cosine metric. In addition, our method naturally uses the F-norm to reduce the impact of outliers. We introduce an iterative algorithm to solve CosMPCA. We provide detailed theoretical analysis in both the proposed method and the analysis of the algorithm. Experiments show that our method is robust to outliers and is suitable for tensors of any order.
{"title":"Cosine Multilinear Principal Component Analysis for Recognition","authors":"Feng Han;Chengcai Leng;Bing Li;Anup Basu;Licheng Jiao","doi":"10.1109/TBDATA.2023.3301389","DOIUrl":"10.1109/TBDATA.2023.3301389","url":null,"abstract":"Existing two-dimensional principal component analysis methods can only handle second-order tensors (i.e., matrices). However, with the advancement of technology, tensors of order three and higher are gradually increasing. This brings new challenges to dimensionality reduction. Thus, a multilinear method called MPCA was proposed. Although MPCA can be applied to all tensors, using the square of the F-norm makes it very sensitive to outliers. Several two-dimensional methods, such as Angle 2DPCA, have good robustness but cannot be applied to all tensors. We extend the robust Angle 2DPCA method to a multilinear method and propose Cosine Multilinear Principal Component Analysis (CosMPCA) for tensor representation. Our CosMPCA method considers the relationship between the reconstruction error and projection scatter and selects the cosine metric. In addition, our method naturally uses the F-norm to reduce the impact of outliers. We introduce an iterative algorithm to solve CosMPCA. We provide detailed theoretical analysis in both the proposed method and the analysis of the algorithm. Experiments show that our method is robust to outliers and is suitable for tensors of any order.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1620-1630"},"PeriodicalIF":7.2,"publicationDate":"2023-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-01DOI: 10.1109/TBDATA.2023.3300546
Zhuang Yang
The extreme success of stochastic optimization (SO) in large-scale machine learning problems, information retrieval, bioinformatics, etc., has been widely reported, especially in recent years. As an effective tactic, conjugate gradient (CG) has been gaining its popularity in accelerating SO algorithms. This paper develops a novel type of stochastic conjugate gradient descent (SCG) algorithms from the perspective of the Powerball strategy and the hypergradient descent (HD) technique. The crucial idea behind the resulting methods is inspired by pursuing the equilibrium of ordinary differential equations (ODEs). We elucidate the effect of the Powerball strategy in SCG algorithms. The introduction of HD, on the other side, makes the resulting methods work with an online learning rate. Meanwhile, we provide a comprehension of the theoretical results for the resulting algorithms under non-convex assumptions. As a byproduct, we bridge the gap between the learning rate and powered stochastic optimization (PSO) algorithms, which is still an open problem. Resorting to numerical experiments on numerous benchmark datasets, we test the parameter sensitivity of the proposed methods and demonstrate the superior performance of our new algorithms over state-of-the-art algorithms.
{"title":"Adaptive Powerball Stochastic Conjugate Gradient for Large-Scale Learning","authors":"Zhuang Yang","doi":"10.1109/TBDATA.2023.3300546","DOIUrl":"10.1109/TBDATA.2023.3300546","url":null,"abstract":"The extreme success of stochastic optimization (SO) in large-scale machine learning problems, information retrieval, bioinformatics, etc., has been widely reported, especially in recent years. As an effective tactic, conjugate gradient (CG) has been gaining its popularity in accelerating SO algorithms. This paper develops a novel type of stochastic conjugate gradient descent (SCG) algorithms from the perspective of the Powerball strategy and the hypergradient descent (HD) technique. The crucial idea behind the resulting methods is inspired by pursuing the equilibrium of ordinary differential equations (ODEs). We elucidate the effect of the Powerball strategy in SCG algorithms. The introduction of HD, on the other side, makes the resulting methods work with an online learning rate. Meanwhile, we provide a comprehension of the theoretical results for the resulting algorithms under non-convex assumptions. As a byproduct, we bridge the gap between the learning rate and powered stochastic optimization (PSO) algorithms, which is still an open problem. Resorting to numerical experiments on numerous benchmark datasets, we test the parameter sensitivity of the proposed methods and demonstrate the superior performance of our new algorithms over state-of-the-art algorithms.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1598-1606"},"PeriodicalIF":7.2,"publicationDate":"2023-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62972533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：