In practical use of speaker adaptation, it is important to provide a framework that operates well for any amount of adaptation data since the amount of data available is often changed. We propose one such framework in which the number of free parameters for estimation is autonomously controlled according to the amount of data for adaptation. It has been applied to a speaker-independent speech recognition system using continuous density mixture Gaussian HMMs, and has proven to be effective through 5,000-word recognition experiments. For example, it achieved a 40% reduction in error rate over the speaker-independent recognition system when 50 words were used for adaptation.