Time-Delay Neural Networks for robust speech recognition are usually trained using the back-propagation learning procedure. Since this learning procedure contains well known disadvantages like occasional instability or no guarantee of convergence, etc., we propose an algorithm which expresses the back-propagation in terms of solving a system of ordinary stiff differential equations. Tests have shown that the problem concerning the training of the TDNN is mathematically stiff, so that a special procedure suggested by Gear for solving stiff differential equations could be applied. Instead of optimizing the weights of the network by using fixed steps of small size, the stability of the differential equation solver allows computational steps of variable sizes that still follow the direction of the gradient's steepest descent. For the purpose of training speech recognizers, this procedure does not necessarily result in an acceleration of the computation time but it ensures the convergence without an adjustment of the learning rate and independent of the initial weight values.

Keywords: Time-Delay Neural Network, back-propagation, stiff differential equations, speech recognition