We studied an unsupervised speaker adaptation method on the sequential training that used the theory of Maximum A Posteriori probability (MAP) estimation for continuous parameter hidden Markov model (HMM). In this method, we should only specify the syllable label sequence for the utterance. The label sequences were provided automatically by the recognizer using a speaker independent (or adapted) model in advance and the language model. The syllable recognition rate using a language model for a given task is expected to have an accuracy of more than 90%, even if we use a speaker independent (SI) model. The experimental results on continuous speech recognition, for sentence or syllable, showed that the better initial model gave a performance comparable to that of supervised adaptation.