In our previous work, we have presented an approach of minimum classification error linear regression (MCELR) for adaptation of Gaussian mixture continuous density HMM (CDHMM) parameters. It is shown that a stochastic approximation approach known as the GPD (generalized probabilistic descent) can be used to optimize the MCE objective function. However, it is relatively diffi- cult to set an appropriate value for the learning control parameter to achieve a fast yet stable GPD optimization process. In this paper, we study another batch-mode approximate second-order optimization approach, namely Quickprop, aiming at speeding up the convergence of the objective function of MCELR while making the learning more robust. It is demonstrated by a series of experiments for supervised speaker adaptation that Quickprop is a better alternative to GPD for MCELR.