Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280355
A. Coutant, Philippe Leray, H. L. Capitaine
Many machine learning algorithms aim at finding pattern in propositional data, where individuals are all supposed i.i.d. However, the massive usage of relational databases makes multi-relational datasets widespread, and the i.i.d. assumptions are often not reasonable in such data, thus requiring dedicated algorithms. Accurate and efficient learning in such datasets is an important challenge with multiples applications including collective classification and link prediction. Probabilistic Relational Models (PRM) are directed lifted graphical models which generalize Bayesian networks in the relational setting. In this paper, we propose a new PRM extension, named PRM with clustering uncertainty, which overcomes several limitations of PRM with reference uncertainty (PRM-RU) extension, such as the possibility to reason about some individual's cluster membership and use co-clustering to improve association variable dependencies. We also propose a structure learning algorithm for these models and show that these improvements allow: i) better prediction results compared to PRM-RU; ii) in less running time.
{"title":"Probabilistic Relational Models with clustering uncertainty","authors":"A. Coutant, Philippe Leray, H. L. Capitaine","doi":"10.1109/IJCNN.2015.7280355","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280355","url":null,"abstract":"Many machine learning algorithms aim at finding pattern in propositional data, where individuals are all supposed i.i.d. However, the massive usage of relational databases makes multi-relational datasets widespread, and the i.i.d. assumptions are often not reasonable in such data, thus requiring dedicated algorithms. Accurate and efficient learning in such datasets is an important challenge with multiples applications including collective classification and link prediction. Probabilistic Relational Models (PRM) are directed lifted graphical models which generalize Bayesian networks in the relational setting. In this paper, we propose a new PRM extension, named PRM with clustering uncertainty, which overcomes several limitations of PRM with reference uncertainty (PRM-RU) extension, such as the possibility to reason about some individual's cluster membership and use co-clustering to improve association variable dependencies. We also propose a structure learning algorithm for these models and show that these improvements allow: i) better prediction results compared to PRM-RU; ii) in less running time.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"1 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89862053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280824
C. Anderson, Minwoo Lee, D. Elliott
Deep learning algorithms have recently appeared that pretrain hidden layers of neural networks in unsupervised ways, leading to state-of-the-art performance on large classification problems. These methods can also pretrain networks used for reinforcement learning. However, this ignores the additional information that exists in a reinforcement learning paradigm via the ongoing sequence of state, action, new state tuples. This paper demonstrates that learning a predictive model of state dynamics can result in a pretrained hidden layer structure that reduces the time needed to solve reinforcement learning problems.
{"title":"Faster reinforcement learning after pretraining deep networks to predict state dynamics","authors":"C. Anderson, Minwoo Lee, D. Elliott","doi":"10.1109/IJCNN.2015.7280824","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280824","url":null,"abstract":"Deep learning algorithms have recently appeared that pretrain hidden layers of neural networks in unsupervised ways, leading to state-of-the-art performance on large classification problems. These methods can also pretrain networks used for reinforcement learning. However, this ignores the additional information that exists in a reinforcement learning paradigm via the ongoing sequence of state, action, new state tuples. This paper demonstrates that learning a predictive model of state dynamics can result in a pretrained hidden layer structure that reduces the time needed to solve reinforcement learning problems.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"50 1","pages":"1-7"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89874412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280507
Linlin Cao, Bao-Gang Hu
This paper describes a progress of the previous study on the generalized constraint neural networks (GCNN). The GCNN model aims to utilize any type of priors in an explicate form so that the model can achieve improved performance and better transparency. A specific type of priors, that is, equality function constraints, is investigated in this work. When the existing approaches impose the constrains in a discretized means on the given function, our approach, called GCNN-EF, is able to satisfy the constrain perfectly and completely on the equation. We realize GCNN-EF by a weighted combination of the output of the conventional radial basis function neural network (RBFNN) and the output expressed by the constraints. Numerical studies are conducted on three synthetic data sets in comparing with other existing approaches. Simulation results demonstrate the benefit and efficiency using GCNN-EF.
{"title":"Generalized constraint neural network regression model subject to equality function constraints","authors":"Linlin Cao, Bao-Gang Hu","doi":"10.1109/IJCNN.2015.7280507","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280507","url":null,"abstract":"This paper describes a progress of the previous study on the generalized constraint neural networks (GCNN). The GCNN model aims to utilize any type of priors in an explicate form so that the model can achieve improved performance and better transparency. A specific type of priors, that is, equality function constraints, is investigated in this work. When the existing approaches impose the constrains in a discretized means on the given function, our approach, called GCNN-EF, is able to satisfy the constrain perfectly and completely on the equation. We realize GCNN-EF by a weighted combination of the output of the conventional radial basis function neural network (RBFNN) and the output expressed by the constraints. Numerical studies are conducted on three synthetic data sets in comparing with other existing approaches. Simulation results demonstrate the benefit and efficiency using GCNN-EF.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"23 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86514627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280303
Dominik Schnitzer, A. Flexer
Unbalanced cluster solutions are affected by very different cluster sizes, with some clusters being very large while others contain almost no data. We demonstrate that this phenomenon is connected to `hubness', a recently discovered general problem of machine learning in high dimensional data spaces. Hub objects have a small distance to an exceptionally large number of data points, and anti-hubs are far from all other data points. In an empirical study of K-medoids clustering we show that hubness gives rise to very unbalanced cluster sizes resulting in impaired internal and external evaluation indices. We compare three methods which reduce hubness in the distance spaces and show that with the balancing of the clusters evaluation indices improve. This is done using artificial and real data sets from diverse domains.
{"title":"The unbalancing effect of hubs on K-medoids clustering in high-dimensional spaces","authors":"Dominik Schnitzer, A. Flexer","doi":"10.1109/IJCNN.2015.7280303","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280303","url":null,"abstract":"Unbalanced cluster solutions are affected by very different cluster sizes, with some clusters being very large while others contain almost no data. We demonstrate that this phenomenon is connected to `hubness', a recently discovered general problem of machine learning in high dimensional data spaces. Hub objects have a small distance to an exceptionally large number of data points, and anti-hubs are far from all other data points. In an empirical study of K-medoids clustering we show that hubness gives rise to very unbalanced cluster sizes resulting in impaired internal and external evaluation indices. We compare three methods which reduce hubness in the distance spaces and show that with the balancing of the clusters evaluation indices improve. This is done using artificial and real data sets from diverse domains.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"30 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86712445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280397
Nhat-Quang Doan, M. Ghesmoune, Hanene Azzag, M. Lebbah
Data stream clustering aims at studying large volumes of data that arrive continuously and the objective is to build a good clustering of the stream, using a small amount of memory and time. Visualization is still a big challenge for large data streams. In this paper we present a new approach using a hierarchical and topological structure (or network) for both clustering and visualization. The topological network is represented by a graph in which each neuron represents a set of similar data points and neighbor neurons are connected by edges. The hierarchical component consists of multiple tree-like hierarchic of clusters which allow to describe the evolution of data stream, and then analyze explicitly their similarity. This adaptive structure can be exploited by descending top-down from the topological level to any hierarchical level. The performance of the proposed algorithm is evaluated on both synthetic and real-world datasets.
{"title":"Growing Hierarchical Trees for Data Stream clustering and visualization","authors":"Nhat-Quang Doan, M. Ghesmoune, Hanene Azzag, M. Lebbah","doi":"10.1109/IJCNN.2015.7280397","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280397","url":null,"abstract":"Data stream clustering aims at studying large volumes of data that arrive continuously and the objective is to build a good clustering of the stream, using a small amount of memory and time. Visualization is still a big challenge for large data streams. In this paper we present a new approach using a hierarchical and topological structure (or network) for both clustering and visualization. The topological network is represented by a graph in which each neuron represents a set of similar data points and neighbor neurons are connected by edges. The hierarchical component consists of multiple tree-like hierarchic of clusters which allow to describe the evolution of data stream, and then analyze explicitly their similarity. This adaptive structure can be exploited by descending top-down from the topological level to any hierarchical level. The performance of the proposed algorithm is evaluated on both synthetic and real-world datasets.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"1296 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86485565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280335
Y. Qian, Tianxing He, Wei Deng, Kai Yu
Although deep neural networks (DNNs) have achieved great performance gain, the immense computational cost of DNN model training has become a major block to utilize massive speech data for DNN training. Previous research on DNN training acceleration mostly focussed on hardware-based parallelization. In this paper, node pruning and arc restructuring are proposed to explore model redundancy after a novel lightly discriminative pretraining process. With some measures of node/arc importance, model redundancies are automatically removed to form a much more compact DNN. This significantly accelerates the subsequent back-propagation (BP) training process. Model redundancy reduction can be combined with multiple GPU parallelization to achieve further acceleration. Experiments showed that the combined acceleration framework can achieve about 85% model size reduction and over 4.2 times speed-up factor for BP training on 2 GPUs, at no loss of recognition accuracy.
{"title":"Automatic model redundancy reduction for fast back-propagation for deep neural networks in speech recognition","authors":"Y. Qian, Tianxing He, Wei Deng, Kai Yu","doi":"10.1109/IJCNN.2015.7280335","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280335","url":null,"abstract":"Although deep neural networks (DNNs) have achieved great performance gain, the immense computational cost of DNN model training has become a major block to utilize massive speech data for DNN training. Previous research on DNN training acceleration mostly focussed on hardware-based parallelization. In this paper, node pruning and arc restructuring are proposed to explore model redundancy after a novel lightly discriminative pretraining process. With some measures of node/arc importance, model redundancies are automatically removed to form a much more compact DNN. This significantly accelerates the subsequent back-propagation (BP) training process. Model redundancy reduction can be combined with multiple GPU parallelization to achieve further acceleration. Experiments showed that the combined acceleration framework can achieve about 85% model size reduction and over 4.2 times speed-up factor for BP training on 2 GPUs, at no loss of recognition accuracy.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"1298 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86486291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280647
C. Ou
Immune memory of antigens are formed as limit behavior of cyclic idiotypic immune networks equipped with antibody dynamics. Immune memory mechanism is studied by combining network structure and dynamical systems. Moreover, associative memory can be explored by network dynamics determined by affinity index of antibody chain. Antibody chains with larger affinity indexes generate associative immune memory.
{"title":"Model of associative memory based on antibody chain with one-dimensional chaotic dynamical system","authors":"C. Ou","doi":"10.1109/IJCNN.2015.7280647","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280647","url":null,"abstract":"Immune memory of antigens are formed as limit behavior of cyclic idiotypic immune networks equipped with antibody dynamics. Immune memory mechanism is studied by combining network structure and dynamical systems. Moreover, associative memory can be explored by network dynamics determined by affinity index of antibody chain. Antibody chains with larger affinity indexes generate associative immune memory.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"1 1","pages":"1-7"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83664277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280782
L. Torres, C. Castro, A. Braga
This paper presents a geometrical approach for obtaining large margin classifiers. The method aims at exploring the geometrical properties of the dataset from the structure of a Gabriel graph, which represents pattern relations according to a given distance metric, such as the Euclidean distance. Once the graph is generated, geometric vectors, analogous to SVM's support vectors are obtained in order to yield the final large margin solution from a mixture model approach. A preliminary experimental study with five real-world benchmarks showed that the method is promising.
{"title":"A parameterless mixture model for large margin classification","authors":"L. Torres, C. Castro, A. Braga","doi":"10.1109/IJCNN.2015.7280782","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280782","url":null,"abstract":"This paper presents a geometrical approach for obtaining large margin classifiers. The method aims at exploring the geometrical properties of the dataset from the structure of a Gabriel graph, which represents pattern relations according to a given distance metric, such as the Euclidean distance. Once the graph is generated, geometric vectors, analogous to SVM's support vectors are obtained in order to yield the final large margin solution from a mixture model approach. A preliminary experimental study with five real-world benchmarks showed that the method is promising.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"45 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88167963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280574
Mashud Rana, I. Koprinska, V. Agelidis
Forecasting solar power generated from photovoltaic systems at different time intervals is necessary for ensuring reliable and economic operation of the electricity grid. In this paper, we study the application of neural networks for predicting the next day photovoltaic power outputs in 30 minutes intervals from the previous values, without using any exogenous data. We propose three different approaches based on ensembles of neural networks - two non-iterative and one iterative. We evaluate the performance of these approaches using four Australian solar datasets for one year. This includes assessing predictive accuracy, evaluating the benefit of using an ensemble, and comparing performance with two persistence models used as baselines and a prediction model based on support vector regression. The results show that among the three proposed approaches, the iterative approach was the most accurate and it also outperformed all other methods used for comparison.
{"title":"Forecasting solar power generated by grid connected PV systems using ensembles of neural networks","authors":"Mashud Rana, I. Koprinska, V. Agelidis","doi":"10.1109/IJCNN.2015.7280574","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280574","url":null,"abstract":"Forecasting solar power generated from photovoltaic systems at different time intervals is necessary for ensuring reliable and economic operation of the electricity grid. In this paper, we study the application of neural networks for predicting the next day photovoltaic power outputs in 30 minutes intervals from the previous values, without using any exogenous data. We propose three different approaches based on ensembles of neural networks - two non-iterative and one iterative. We evaluate the performance of these approaches using four Australian solar datasets for one year. This includes assessing predictive accuracy, evaluating the benefit of using an ensemble, and comparing performance with two persistence models used as baselines and a prediction model based on support vector regression. The results show that among the three proposed approaches, the iterative approach was the most accurate and it also outperformed all other methods used for comparison.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"70 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85798080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-07-12DOI: 10.1109/IJCNN.2015.7280450
Călin-Adrian Popa
This paper presents the full deduction of the quasi-Newton learning methods for complex-valued feedforward neural networks. Since these algorithms yielded better training results for the real-valued case, an extension to the complex-valued case is a natural option to enhance the performance of the complex backpropagation algorithm. The training methods are exemplified on various well-known synthetic and real-world applications. Experimental results show a significant improvement over the complex gradient descent algorithm.
{"title":"Quasi-Newton learning methods for complex-valued neural networks","authors":"Călin-Adrian Popa","doi":"10.1109/IJCNN.2015.7280450","DOIUrl":"https://doi.org/10.1109/IJCNN.2015.7280450","url":null,"abstract":"This paper presents the full deduction of the quasi-Newton learning methods for complex-valued feedforward neural networks. Since these algorithms yielded better training results for the real-valued case, an extension to the complex-valued case is a natural option to enhance the performance of the complex backpropagation algorithm. The training methods are exemplified on various well-known synthetic and real-world applications. Experimental results show a significant improvement over the complex gradient descent algorithm.","PeriodicalId":6539,"journal":{"name":"2015 International Joint Conference on Neural Networks (IJCNN)","volume":"44 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2015-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86873635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：