A limitation to current, HMM-based Speech Recognition approaches lies in the modeling of non-vocabulary utterances. Improved rejection is a key research direction in Interactive Voice Response Services (IVR), where field evaluations show that many users do not only utter the requested keywords. This paper compares several discriminant training criteria on this problem and applies a novel optimization technique which can be used to improve rejection, without seriously disturbing HMM modeling assumptions. A 23% reduction in the error rate is observed on field data recorded during the operation of an IVR service.