To build optimally effective word classifiers, one research direction in speech recognition is to train a connectionist architecture with a gradient back-propagation procedure that minimises the word error rate directly. The first step was the integration of the DTW alignment procedure into the architecture: the Multi-State Time Delay Neural Network (MS-TDNN[6]) architecture was successfully demonstrated on several large speech recognition tasks. In this paper, we provide an HMM probabilistic framework for the alignment procedure, with improved experimental results. Moreover, applying a unified HMM/connectionist formalation to global speech recognition systems suggests ways to exchange expertise between both fields.