In this work, we present a new strategy to combine neural networks with HMMs which tend to take advantage of the modelling abilities of two independent modules, the ANN and the HMM, to make the design of a hybrid system less complicated. This approach incorporates fuzzy probabilistic information in the HMM to decompose the training task of a hybrid system. Using this strategy the training system is optimized about 7 times without significant loss of information. Also, we describe different techniques to improve the performances of the system which reduce the word error rate by 40%. Using this methodology, the hybrid system is trained much faster and can now benefit from two distinct sources of improvements such as neural modelling and classical HMM modelling which is less costly to perform.