This paper describes two pragmatic criteria for ruling out speech recognition candidates that are correct syntactically and semantically, but incorrect contextually. The first criterion is whether the referents of the referring expressions in the candidates, especially zero pronouns in Japanese, can be properly identified or not. We have implemented an ellipsis resolution method that identifies zero pronouns designating the conversational participants based on the uses of honorific and deictic expressions, and employed the "fewest unidentified zero pronouns heuristics," in order to penalize the recognition candidates that have spurious zero pronouns whose referents cannot be identified. The second criterion is whether the candidate forms a natural local discourse in terms of the speech act sequence. We have classified the input utterances into nine Elocutionary Force Types (IFTs), such as REQUEST, INFORM, etc., and used its trigram to rule out erroneous candidates by predicting the next utterance type. The pilot experiment shows that the IFT trigram predicts the IFT of the next utterance with better than 70% accuracy when there is IFT ambiguity in the recognition results.