IEEE transactions on neural networks最新文献

By following Zhang design method, a new type of recurrent neural network [i.e., Zhang neural network (ZNN)] is presented, investigated, and analyzed for online solution of time-varying linear inequalities. Theoretical analysis is given on convergence properties of the proposed ZNN model. For comparative purposes, the conventional gradient neural network is developed and exploited for solving online time-varying linear inequalities as well. Computer simulation results further verify and demonstrate the efficacy, novelty, and superiority of such a ZNN model and its method for solving time-varying linear inequalities.

This paper is concerned with the problem of observer design for switched recurrent neural networks with time-varying delay. The attention is focused on designing the full-order observers that guarantee the global exponential stability of the error dynamic system. Based on the average dwell time approach and the free-weighting matrix technique, delay-dependent sufficient conditions are developed for the solvability of such problem and formulated as linear matrix inequalities. The error-state decay estimate is also given. Then, the stability analysis problem for the switched recurrent neural networks can be covered as a special case of our results. Finally, four illustrative examples are provided to demonstrate the effectiveness and the superiority of the proposed methods.

The identification of nonlinear spatiotemporal systems is of significance to engineering practice, since it can always provide useful insight into the underlying nonlinear mechanism and physical characteristics under study. In this paper, nonlinear spatiotemporal system models are transformed into a class of multi-input-multi-output (MIMO) partially linear systems (PLSs), and an effective online identification algorithm is therefore proposed by using a pruning error minimization principle and least square support vector machines. It is shown that many benchmark physical and engineering systems can be transformed into MIMO-PLSs which keep some important physical spatiotemporal relationships and are very helpful in the identification and analysis of the underlying system. Compared with several existing methods, the advantages of the proposed method are that it can make full use of some prior structural information about system physical models, can realize online estimation of the system dynamics, and achieve accurate characterization of some important nonlinear physical characteristics of the system. This would provide an important basis for state estimation, control, optimal analysis, and design of nonlinear distributed parameter systems. The proposed algorithm can also be applied to identification problems of stochastic spatiotemporal dynamical systems. Numeral examples and comparisons are given to demonstrate our results.

The classical continuous time recurrent (Hopfield) network is considered and adapted to K -winner-take-all operation. The neurons are of sigmoidal type with a controllable gain G, an amplitude m and interconnected by the conductance p. The network is intended to process one by one a sequence of lists, each of them with N distinct elements, each of them squeezed to [0,I] admission interval, each of them having an imposed minimum separation between elements z(min). The network carries out: 1) a matching dynamic process between the order of list elements and the order of outputs, and 2) a binary type steady-state separation between K and K+1 outputs, the former surpassing a +ξ threshold and the later falling under the -ξ threshold. As a result, the machine will signal the ranks of the K largest elements of the list. To achieve 1), the initial condition of processing phase has to be placed in a computable θ -vicinity of zero-state. This requires a resetting procedure after each list. To achieve 2) the bias current M has to be within a certain interval computable from circuit parameters. In addition, the steady-state should be asymptotically stable. To these goals, we work with high gain and exploit the sigmoid properties and network symmetry. The various inequality type constraints between parameters are shown to be compatible and a neat synthesis procedure, simple and flexible, is given for the tanh sigmoid. It starts with the given parameters N, K, I, z(min), m and computes simple bounds of p, G, ξ, θ, and M. Numerical tests and comments reveal qualities and shortcomings of the method.

Local model networks approximate a nonlinear system through multiple local models fitted within a partition space. The main advantage of this approach is that the identification of complex nonlinear processes is alleviated by the integration of structured knowledge about the process. This paper extends these concepts by the integration of quantitative process knowledge into the identification procedure. Quantitative knowledge describes explicit dependences between inputs and outputs and is integrated in the parameter estimation process by means of equality constraints. For this purpose, a constrained generalized total least squares algorithm for local parameter estimation is presented. Furthermore, the problem of proper integration of constraints in the partitioning process is treated where an expectation-maximization procedure is combined with constrained parameter estimation. The benefits and the applicability of the proposed concepts are demonstrated by means of two illustrative examples and a practical application using real measurement data.

This paper evaluates the four leading techniques proposed in the literature for construction of prediction intervals (PIs) for neural network point forecasts. The delta, Bayesian, bootstrap, and mean-variance estimation (MVE) methods are reviewed and their performance for generating high-quality PIs is compared. PI-based measures are proposed and applied for the objective and quantitative assessment of each method's performance. A selection of 12 synthetic and real-world case studies is used to examine each method's performance for PI construction. The comparison is performed on the basis of the quality of generated PIs, the repeatability of the results, the computational requirements and the PIs variability with regard to the data uncertainty. The obtained results in this paper indicate that: 1) the delta and Bayesian methods are the best in terms of quality and repeatability, and 2) the MVE and bootstrap methods are the best in terms of low computational load and the width variability of PIs. This paper also introduces the concept of combinations of PIs, and proposes a new method for generating combined PIs using the traditional PIs. Genetic algorithm is applied for adjusting the combiner parameters through minimization of a PI-based cost function subject to two sets of restrictions. It is shown that the quality of PIs produced by the combiners is dramatically better than the quality of PIs obtained from each individual method.

In many applications, the available information is encoded in graph structures. This is a common problem in biological networks, social networks, web communities and document citations. We investigate the problem of classifying nodes' labels on a similarity graph given only a graph structure on the nodes. Conventional machine learning methods usually require data to reside in some Euclidean spaces or to have a kernel representation. Applying these methods to nodes on graphs would require embedding the graphs into these spaces. By embedding and then learning the nodes on graphs, most methods are either flexible with different learning objectives or efficient enough for large scale applications. We propose a method to embed a graph into a feature space for a discriminative purpose. Our idea is to include label information into the embedding process, making the space representation tailored to the task. We design embedding objective functions that the following learning formulations become spectral transforms. We then reformulate these spectral transforms into multiple kernel learning problems. Our method, while being tailored to the discriminative tasks, is efficient and can scale to massive data sets. We show the need of discriminative embedding on some simulations. Applying to biological network problems, our method is shown to outperform baselines.

A dynamic feedforward neural network (DFNN) is proposed for predictive control, whose adaptive parameters are adjusted by using Gaussian particle swarm optimization (GPSO) in the training process. Adaptive time-delay operators are added in the DFNN to improve its generalization for poorly known nonlinear dynamic systems with long time delays. Furthermore, GPSO adopts a chaotic map with Gaussian function to balance the exploration and exploitation capabilities of particles, which improves the computational efficiency without compromising the performance of the DFNN. The stability of the particle dynamics is analyzed, based on the robust stability theory, without any restrictive assumption. A stability condition for the GPSO+DFNN model is derived, which ensures a satisfactory global search and quick convergence, without the need for gradients. The particle velocity ranges could change adaptively during the optimization process. The results of a comparative study show that the performance of the proposed algorithm can compete with selected algorithms on benchmark problems. Additional simulation results demonstrate the effectiveness and accuracy of the proposed combination algorithm in identifying and controlling nonlinear systems with long time delays.

Reservoir computing (RC), a computational paradigm inspired on neural systems, has become increasingly popular in recent years for solving a variety of complex recognition and classification problems. Thus far, most implementations have been software-based, limiting their speed and power efficiency. Integrated photonics offers the potential for a fast, power efficient and massively parallel hardware implementation. We have previously proposed a network of coupled semiconductor optical amplifiers as an interesting test case for such a hardware implementation. In this paper, we investigate the important design parameters and the consequences of process variations through simulations. We use an isolated word recognition task with babble noise to evaluate the performance of the photonic reservoirs with respect to traditional software reservoir implementations, which are based on leaky hyperbolic tangent functions. Our results show that the use of coherent light in a well-tuned reservoir architecture offers significant performance benefits. The most important design parameters are the delay and the phase shift in the system's physical connections. With optimized values for these parameters, coherent semiconductor optical amplifier (SOA) reservoirs can achieve better results than traditional simulated reservoirs. We also show that process variations hardly degrade the performance, but amplifier noise can be detrimental. This effect must therefore be taken into account when designing SOA-based RC implementations.