In this paper, we compare two powerful kernel-based learning machines, support vector machines (SVM) and relevance vector machines (RVM), within the framework of hidden Markov model-based speech recognition. Both machines provide nonlinear discriminative classification ability: the SVM by kernel-based margin maximization and the RVM using a Bayesian probabilistic framework. The hybrid systems are compared on a vowel classification task and on the continuous speech Alphadigits corpus. In both cases, the RVM system achieves better error rates with significantly fewer parameters.