In this paper, we propose a novel confidence scoring method that is applied to N-best hypotheses output from an HMM-based classifier. In the first pass of the proposed method, the HMM-based classifier with monophone models outputs N-best hypotheses (word candidates) and boundaries of all the monophones in the hypotheses. In the second pass, an SM (Sub-space Method)-based verifier tests the hypotheses by comparing confidence scores. We discuss how to convert a monophone similarity score of SM into a likelihood score, how to normalize the variations of acoustic quality in an utterance, how to combine an HMM-based likelihood of word level and an SM-based likelihood of monophone level, and also how to accept the correct words and reject OOV words. In the experiments performed on speaker- independent word recognition, the proposed confidence scoring method significantly reduced word error rate from 4.7% obtained by the standard HMM classifier to 2.0%, and it also reduced the equal error rate from 9.0% to 6.5% in an unknown word rejection task.