This paper describes a hybrid speech recognition system using MLPs and HMMs, Instead of the common use of MLPs as probability generators, we propose to use MLPs as labelers for discrete parameter HMMs, Compared to the probabilistic approach, this has the advantages of having much more flexibility in our system design (e.g. no limitation to phonetic models) and being able to use much smaller MLPs, Compared to Euclidean labeling, our approach has the advantage of needing fewer EMM parameters per state while achieving greater recognition accuracy. Our recent results show that the best configuration is to use four independent MLPs as labelers. The results can be improved by training the MLPs for subphoneme classification.