In this paper we present results of two experiments on discriminative training of speech recognition systems based on the generalized probabilistic descent method. Firstly, we apply the approach as an extension of the traditional Viterbi training in order to estimate parameters of tied density hidden Markov models, namely the mean vectors of the densities and the state specific weights for the densities. Secondly, we discuss a scheme for the combination of scores obtained from models for different feature sets. In this case, the generalized probabilistic descent method is used in order to estimate the weights of the individual model scores.