A new model matching approach for Mandarin syllable recognition based on the Chernoff distance is proposed in this paper. The recognition process is evaluated segment by segment instead of frame by frame based on the Chernoff distance such that the recognition time can be reduced significantly. Besides, since the measurement criterion, i.e. Chernoff distance, is also derived from the Bayes theorem, competitive recognition rates are expected to be achieved. Experimental results show that compared with the phone-based CHMM's, more than 25 times of recognition speed can be obtained with a 12.43% error rate reduction using less than | mixture numbers in the proposed Chernoff distance based segmental probability models (CD-SPM).