Up to now, Hidden Markov modelling (HMM) has been the most important approach to model sub-lexical units in acoustic-phonetic decoding. In this framework, the Semicontinuous HMM (SCHMM) allows us to obtain better recognition rates than the discrete one. Moreover, computational complexity is less than that of continuous mixture HMM. Within the framework of the SCHMM, this paper deals with the multiple codebook formulation of different approaches to the Viterbi-based re-estimation procedure. We aimed to take advantage of the jointly optimization of the different codebooks together with the parameters of the model. From this formulation, new re-estimation equations are obtained. The above approaches allow the different codebooks to be updated in the training phase while keeping the computational cost quite low. Several series of decoding experiments were carried out over a corpus of 1,700 Spanish sentences uttered by 10 speakers. The decoding results present a strong improvement when multiple observations are considered

Keywords: Speech recognition, structural pattern recognition, hidden Markov models, Spanish acoustic-phonetic decoding.