This paper presents a new connectionist architecture which incorporates the Forward-Backward (ap) alignment procedure. This generalises the Viterbi alignment procedure which exists in most "hybrid" connectionist systems used for speech recognition. Our aB-TDNN architecture extends Multi-State Time Delay Neural Networks (MS-TDNNs) [4] to make them theoretically more consistent with the Back-Propagation training procedure and experimentally more robust. With the ap-TDNN, simple modelling assumptions about time alignment suggest choices in the architecture and the objective function which leads to improvements in performance. In particular, we show the possibility to train the aB-TDNN with a global one-pass algorithm based on Maximum Mutual Information (MMI): this system was trained on two speaker independent word recognition tasks, without any bootstrapping, within a reasonable learning time and with good performances.

Keywords: Neural Networks, Forward-Backward Alignment, MMIE, TDNN