This paper proposed a maximum model distance (MMD) approach for HMM-based speech recognition. MMD uses the whole training set to estimate the parameters of each HMM while the traditional maximum likelihood (ML) uses only those data labeled for the model. Theoretical and practical issues concerning this approach in speech recognition are investigated. Both speakerdependent and multi-speaker experiments on confusable Chinese An-set showed that significant error reduction can be achieved through the proposed approach. In addition, the relationship between MMD and corrective training [5] was discussed and it is proved that the corrective training is a special case of MMD approach.