Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374299
S. Chakravarthy, Joydeep Ghosh
Adaptive learning dynamics of the radial basis function network (RBFN) are compared with a scale-based clustering technique and a relationship between the two is pointed out. Using this link, it is shown how scale-based clustering can be done using the RBFN, with the radial basis function (RBF) width as the scale parameter. The technique suggests the "right" scale at which the given data set must be clustered and obviates the need for knowing the number of clusters beforehand. We show how this method solves the problem of determining the number of RBF units and the widths required to get a good network solution.<>
{"title":"Scale-based clustering using the radial basis function network","authors":"S. Chakravarthy, Joydeep Ghosh","doi":"10.1109/ICNN.1994.374299","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374299","url":null,"abstract":"Adaptive learning dynamics of the radial basis function network (RBFN) are compared with a scale-based clustering technique and a relationship between the two is pointed out. Using this link, it is shown how scale-based clustering can be done using the RBFN, with the radial basis function (RBF) width as the scale parameter. The technique suggests the \"right\" scale at which the given data set must be clustered and obviates the need for knowing the number of clusters beforehand. We show how this method solves the problem of determining the number of RBF units and the widths required to get a good network solution.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128235052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374218
Yiwei Chen, F. Bastani
The paper examines the capability and performance of 1-hidden-layer feedforward neural networks with multi-activation product (MAP) units, through the application of drawing digital line segments. The MAP unit is a recently proposed multi-dendrite neuron model. The centroidal function is chosen as the MAP unit base activation function because it demonstrates a superior performance over the sigmoidal functions. The network with MAP units with more than one dendrite converges statistically faster during the learning phase with randomly selected training patterns. The generalization to the entire sample space is shown to be proportional to the size of the training patterns.<>
{"title":"Feedforward neural networks to learn drawing lines","authors":"Yiwei Chen, F. Bastani","doi":"10.1109/ICNN.1994.374218","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374218","url":null,"abstract":"The paper examines the capability and performance of 1-hidden-layer feedforward neural networks with multi-activation product (MAP) units, through the application of drawing digital line segments. The MAP unit is a recently proposed multi-dendrite neuron model. The centroidal function is chosen as the MAP unit base activation function because it demonstrates a superior performance over the sigmoidal functions. The network with MAP units with more than one dendrite converges statistically faster during the learning phase with randomly selected training patterns. The generalization to the entire sample space is shown to be proportional to the size of the training patterns.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130400298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374132
Xiao Liu, G. Wilcox
In this paper, we present a new, efficient implementation of the backpropagation algorithm (BP) on the CM-5 by fully taking advantage of its Control Network to avoid explicit message-passing. The nodes in the input and output layers are evenly distributed to all processors: all nodes in the hidden layer(s) are replicated in each processor, and all weights are distributed to all processors corresponding to the nodes. We have implemented this algorithm on the CM-5 in the MIMD mode using the C programming language. For a case study of protein tertiary structure prediction, we obtained performance of 76 million weight updates per second (WUPS) with the machine partitioned for 512 processors without vector units. Experiments using different sized partitions indicated an almost linear relationship between the computation time and the number of processors, indicating good parallelization. We have also implemented the backpropagation algorithm on the Cray machines using the C programming language. The Cray-2 implementation yields performance of 10 million WUPS; the Cray X-MP EA implementation yields 18 million WUPS; and the Cray Y-MP M92 implementation yields 40 million WUPS.<>
{"title":"Benchmarking of the CM-5 and the Cray machines with a very large backpropagation neural network","authors":"Xiao Liu, G. Wilcox","doi":"10.1109/ICNN.1994.374132","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374132","url":null,"abstract":"In this paper, we present a new, efficient implementation of the backpropagation algorithm (BP) on the CM-5 by fully taking advantage of its Control Network to avoid explicit message-passing. The nodes in the input and output layers are evenly distributed to all processors: all nodes in the hidden layer(s) are replicated in each processor, and all weights are distributed to all processors corresponding to the nodes. We have implemented this algorithm on the CM-5 in the MIMD mode using the C programming language. For a case study of protein tertiary structure prediction, we obtained performance of 76 million weight updates per second (WUPS) with the machine partitioned for 512 processors without vector units. Experiments using different sized partitions indicated an almost linear relationship between the computation time and the number of processors, indicating good parallelization. We have also implemented the backpropagation algorithm on the Cray machines using the C programming language. The Cray-2 implementation yields performance of 10 million WUPS; the Cray X-MP EA implementation yields 18 million WUPS; and the Cray Y-MP M92 implementation yields 40 million WUPS.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123563099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374200
T. Kwon, Ehsan H. Feroz, Hui Cheng
This paper introduces a data preprocessing algorithm that can improve the efficiency of the standard backpropagation (BP) algorithm. The basic approach is transforming input data to a range that associates high-slopes of sigmoid where relatively large modification of weights occurs. This helps escaping of early trapping from prematured saturation. However, a simple and uniform transformation to such desired range can lead to a slow learning if the data have a heavily skewed distribution. In order to improve the performance of BP algorithm on such distribution, the authors propose a modified histogram equalization technique which enhances the spacing between data points in the heavily concentrated regions of skewed distribution. The authors' simulation study shows that this modified histogram equalization can significantly speed up the BP training as well as improving the generalization capability of the trained network.<>
{"title":"Preprocessing of training set for backpropagation algorithm: histogram equalization","authors":"T. Kwon, Ehsan H. Feroz, Hui Cheng","doi":"10.1109/ICNN.1994.374200","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374200","url":null,"abstract":"This paper introduces a data preprocessing algorithm that can improve the efficiency of the standard backpropagation (BP) algorithm. The basic approach is transforming input data to a range that associates high-slopes of sigmoid where relatively large modification of weights occurs. This helps escaping of early trapping from prematured saturation. However, a simple and uniform transformation to such desired range can lead to a slow learning if the data have a heavily skewed distribution. In order to improve the performance of BP algorithm on such distribution, the authors propose a modified histogram equalization technique which enhances the spacing between data points in the heavily concentrated regions of skewed distribution. The authors' simulation study shows that this modified histogram equalization can significantly speed up the BP training as well as improving the generalization capability of the trained network.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"109 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115118767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374227
D. Alpsan, M. Towsey, O. Ozdamar, A. Tsoi, D. Ghista
A wide range of modifications and extensions to the backpropagation (BP) algorithm have been tested on a real world medical problem. Our results show that: 1) proper tuning of learning parameters of standard BP not only increases the speed of learning but also has a significant effect on generalisation; 2) parameter combinations and training options which lead to fast learning do not usually yield good generalisation and vice versa; 3) standard BP may be fast enough when its parameters are finely tuned; 4) modifications developed on artificial problems for faster learning do not necessarily give faster learning on real-world problems, and when they do, it may be at the expense of generalisation; and 5) even when modified BP algorithms perform well, they may require extensive fine-tuning to achieve this performance. For our problem, none of the modifications could justify the effort to implement them.<>
{"title":"Are modified back-propagation algorithms worth the effort?","authors":"D. Alpsan, M. Towsey, O. Ozdamar, A. Tsoi, D. Ghista","doi":"10.1109/ICNN.1994.374227","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374227","url":null,"abstract":"A wide range of modifications and extensions to the backpropagation (BP) algorithm have been tested on a real world medical problem. Our results show that: 1) proper tuning of learning parameters of standard BP not only increases the speed of learning but also has a significant effect on generalisation; 2) parameter combinations and training options which lead to fast learning do not usually yield good generalisation and vice versa; 3) standard BP may be fast enough when its parameters are finely tuned; 4) modifications developed on artificial problems for faster learning do not necessarily give faster learning on real-world problems, and when they do, it may be at the expense of generalisation; and 5) even when modified BP algorithms perform well, they may require extensive fine-tuning to achieve this performance. For our problem, none of the modifications could justify the effort to implement them.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"549 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123098449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374333
M. Hattori, M. Hagiwara, M. Nakagawa
Several important characteristics of Quick Learning for Bidirectional Associative Memory (QLBAM) are introduced. QLBAM uses two stage learning. In the first stage, the BAM is trained by Hebbian learning and then by Pseudo-Relaxation Learning Algorithm for BAM (PRLAB). The following features of the QLBAM are made clear: it is insensitive to correlation of training pairs; it is robust for noisy inputs; the minimum absolute value of net inputs indexes a noise margin; the memory capacity is greatly improved: the maximum capacity in our simulation is about 2.2N.<>
{"title":"New results of Quick Learning for Bidirectional Associative Memory having high capacity","authors":"M. Hattori, M. Hagiwara, M. Nakagawa","doi":"10.1109/ICNN.1994.374333","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374333","url":null,"abstract":"Several important characteristics of Quick Learning for Bidirectional Associative Memory (QLBAM) are introduced. QLBAM uses two stage learning. In the first stage, the BAM is trained by Hebbian learning and then by Pseudo-Relaxation Learning Algorithm for BAM (PRLAB). The following features of the QLBAM are made clear: it is insensitive to correlation of training pairs; it is robust for noisy inputs; the minimum absolute value of net inputs indexes a noise margin; the memory capacity is greatly improved: the maximum capacity in our simulation is about 2.2N.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"4 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133644627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374636
T. Yamada, T. Yabuta
Many studies such as Kawato's work (1987) have been undertaken in order to apply both the flexibility and learning ability of neural networks to dynamic system controllers. Most of them used a fixed shape sigmoid function. We have confirmed that it is useful to change the sigmoid function shape to improve the nonlinear mapping capability of neural network controllers. This paper introduces the a new concept for autotuning sigmoid function shapes of neural network servo controllers. Three types of tuning method are proposed in order to improve the nonlinear mapping capability. The first type uses a uniform sigmoid function shape. With the second type, the sigmoid function shapes within one layer are the same and the shapes are tuned layer by layer is tuned. With the third type, the sigmoid function shape of each neuron is different and is tuned individually. Their characteristics are confirmed by simulation.<>
{"title":"Remarks on neural network controller using different sigmoid functions","authors":"T. Yamada, T. Yabuta","doi":"10.1109/ICNN.1994.374636","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374636","url":null,"abstract":"Many studies such as Kawato's work (1987) have been undertaken in order to apply both the flexibility and learning ability of neural networks to dynamic system controllers. Most of them used a fixed shape sigmoid function. We have confirmed that it is useful to change the sigmoid function shape to improve the nonlinear mapping capability of neural network controllers. This paper introduces the a new concept for autotuning sigmoid function shapes of neural network servo controllers. Three types of tuning method are proposed in order to improve the nonlinear mapping capability. The first type uses a uniform sigmoid function shape. With the second type, the sigmoid function shapes within one layer are the same and the shapes are tuned layer by layer is tuned. With the third type, the sigmoid function shape of each neuron is different and is tuned individually. Their characteristics are confirmed by simulation.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131675108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-12-01DOI: 10.1109/ICNN.1994.374829
M. Krcmár, A. Dhawan
Genetic algorithms (GA) can be exploited for optimal graph matching. Graphs represent powerful method of a pattern formal description. Globally optimal graph matching is a NP-complete problem. Pattern distortions and noise increase an optimal search difficulty which could be tackled using GA. This paper describes results of simple GA applied on a graph matching problem. As a conclusion, the suitable GA for an optimal graph "isomorphism" and "monomorphism" is proposed. Used coding resembles the travelling salesman problem (TSP). Consequently, performance of ordering operators has been tested. In contrast to the TSP, the fitness function depends on chromosome value positioning not ordering. It results in differences between optimal GA configuration for graph matching and for TSP.<>
{"title":"Application of genetic algorithms in graph matching","authors":"M. Krcmár, A. Dhawan","doi":"10.1109/ICNN.1994.374829","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374829","url":null,"abstract":"Genetic algorithms (GA) can be exploited for optimal graph matching. Graphs represent powerful method of a pattern formal description. Globally optimal graph matching is a NP-complete problem. Pattern distortions and noise increase an optimal search difficulty which could be tackled using GA. This paper describes results of simple GA applied on a graph matching problem. As a conclusion, the suitable GA for an optimal graph \"isomorphism\" and \"monomorphism\" is proposed. Used coding resembles the travelling salesman problem (TSP). Consequently, performance of ordering operators has been tested. In contrast to the TSP, the fitness function depends on chromosome value positioning not ordering. It results in differences between optimal GA configuration for graph matching and for TSP.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127193074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-05DOI: 10.1109/ICNN.1994.375017
Mark Moll, R. Miikkulainen, Jonathan Abbey
Human episodic memory provides a seemingly unlimited storage for everyday experiences, and a retrieval system that allows us to access the experiences with partial activation of their components. This paper presents a computational model of episodic memory inspired by Damasio's idea of convergence zones. The model consists of a layer of perceptual feature maps and a binding layer. A perceptual feature pattern is coarse coded in the binding layer, and stored on the weights between layers. A partial activation of the stored features activates the binding pattern which in turn reactivates the entire stored pattern. A worst-case analysis shows that with realistic-size layers, the memory capacity of the model is several times larger than the number of units in the model, and could account for the large capacity of human episodic memory.<>
{"title":"The capacity of convergence-zone episodic memory","authors":"Mark Moll, R. Miikkulainen, Jonathan Abbey","doi":"10.1109/ICNN.1994.375017","DOIUrl":"https://doi.org/10.1109/ICNN.1994.375017","url":null,"abstract":"Human episodic memory provides a seemingly unlimited storage for everyday experiences, and a retrieval system that allows us to access the experiences with partial activation of their components. This paper presents a computational model of episodic memory inspired by Damasio's idea of convergence zones. The model consists of a layer of perceptual feature maps and a binding layer. A perceptual feature pattern is coarse coded in the binding layer, and stored on the weights between layers. A partial activation of the stored features activates the binding pattern which in turn reactivates the entire stored pattern. A worst-case analysis shows that with realistic-size layers, the memory capacity of the model is several times larger than the number of units in the model, and could account for the large capacity of human episodic memory.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124604249","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-01DOI: 10.1109/ICNN.1994.374880
P. Raveendran, S. Jegannathan, S. Omatu
This paper presents a technique to classify images that have been elongated or contracted. The problem is formulated using conventional regular moments. It is shown that the conventional regular moment-invariants remain no longer invariant when the image is scaled unequally in the x- and y-directions. A method is proposed to form moment-invariants that do not change under such unequal scaling. Results of computer simulations for images are also included verifying the validity of the method proposed.<>
{"title":"Classification of elongated and contracted images using new regular moments","authors":"P. Raveendran, S. Jegannathan, S. Omatu","doi":"10.1109/ICNN.1994.374880","DOIUrl":"https://doi.org/10.1109/ICNN.1994.374880","url":null,"abstract":"This paper presents a technique to classify images that have been elongated or contracted. The problem is formulated using conventional regular moments. It is shown that the conventional regular moment-invariants remain no longer invariant when the image is scaled unequally in the x- and y-directions. A method is proposed to form moment-invariants that do not change under such unequal scaling. Results of computer simulations for images are also included verifying the validity of the method proposed.<<ETX>>","PeriodicalId":209128,"journal":{"name":"Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124956483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：