This paper proposes an utterance verification system for hidden Markov model (HMM) based automatic speech recognition systems. A verification objective function, based on a multi-layer-perceptron (MLP), is adopted which combines confidence measures from both the recognition and verification models. Discriminative minimum verification error training is applied for optimizing the parameters of the MLP and the verification models. Our proposed system provides a framework for combining different knowledge sources for utterance verification using an objective function that is consistently applied during both training and testing. Experimental results on telephone-based connected digits are presented.