Response Surfaces of Neural Networks Learned Using Bayesian Framework and Its Application to Optimization Problem

Abstract

We verified the generalization ability of the response surfaces of artificial neural networks (NNs), and that the surfaces could be applied to an engineering-design problem. A Bayesian framework to regularize NNs, which was proposed by Gull and Skilling, can be used to generate NN response surfaces with excellent generalization ability, i.e., to determine the regularizing constants in an objective function minimized during NN learning. This well-generalized NN might be useful to find an optimal solution in the process of response surface methodology (RSM). We, therefore, describe three rules based on the Bayesian framework to update the regularizing constants, utilizing these rules to generate NN response surfaces with noisy teacher data drawn from a typical unimodal or multimodal function. Good generalization ability was achieved with regularized NN response surfaces, even though an update rule including trace evaluation failed to determine the regularizing constants regardless of the response function. We, next, selected the most appropriate update rule, which included eigenvalue evaluation, and then the NN response surface regularized using the update rule was applied to finding the optimal solution to an illustrative engineering-design problem. The NN response surface did not fit the noise in the teacher data, and consequently, it could effectively be used to achieve a satisfactory solution. This may increase the opportunities for using NN in the process of RSM.