This paper investigates the effectiveness of the DAEM (Deterministic Annealing EM) algorithm in acoustic modeling for speaker and speech recognition. Although the EM algorithm has been widely used to approximate the ML estimates, it has the problem of initialization dependence. To relax this problem, the DAEM algorithm has been proposed and confirmed the effectiveness in small tasks. In this paper, we applied the DAEM algorithm to speaker recognition based on GMMs and coutinuous speech recognition based on HMMs. Experimental results show that the DAEM algorithm can improve the recognition performance as compared to the ordinary EM algorithm with conventional initialization methods, especially in the flat start for continuous speech recognition.