Symbolic Regression Based Feature Extraction of Shallow Neural-Networks for Identification and Prediction

Abstract

This paper proposes a feature extraction method to improve the performance of shallow neural-network models with less number of parameters to apply especially on embedded system design at remote applications. Feature extraction method is designed using fuzzy c-means clustering based fuzzy system design cascaded a layer of symbolic operators and functions, respectively. During the training stage of neural-networks, symbolic operators and functions are selected using random-learning theory with the unity internal weights such that based on the prediction performance, optimal sequences are recorded for feature extraction to be utilized on testing phase. Extracted features are here used to empower the single-layer neural-network (SLNN) with sigmoid hyperbolic activation functions, functional-link neural- network (FLNN) with Chebyshev polynomials and Pi-Sigma higher-order neural-network (PSNN) with sigmoid activation functions, respectively. The internal and output parameters of the appended shallow neural-networks are optimized using batch optimization methods. Proposed regression models are first tested on identification of an artificial discrete-time dynamic system and real-time inverted pendulum then also for prediction of the sunspot time-series and traffic density estimation. As a result, the prediction performance of shallow neural networks is improved to be used in future applications.