Second-Order Structure Optimization of Fully Complex-Valued Neural Networks

Abstract

This paper focuses on proposing a second-order complex-valued incremental learning (CVIL) algorithm for the structure optimization of fully complex-valued neural networks (FCVNNs). The main purpose of this study is to integrate the structure optimization and parameter learning of FCVNNs into a unified framework such that good generalization is guaranteed. A hybrid training strategy is firstly developed for FCVNNs with fixed structure. By introducing complex-valued sparse matrices and generalized augmented hidden output matrix, nonlinear parameters between the hidden and input neurons are trained by complex-valued Levenberg-Marquardt (CLM) algorithm and linear parameters between the output and hidden neurons are obtained by complex-valued least squares (CLS) algorithm. Starting with an initial FCVNN, hidden neurons are added one by one once the training falls in the plateau. It is theoretically shown that the objective function is monotonously decreasing after adding hidden neuron and successive learning is immediately continuous with the latest training results. Repetition training is avoided and thus training efficiency is achieved. The experimental results on the channel modulation identification and real-valued pattern classification tasks are provided to demonstrate that the developed algorithm is superior to some existing ones for the training of FCVNNs.