This paper describes a new approach to dynamic speaker adaptation, which relies on switching between different methods of adaptation in order to gain maximum performance depending on the amount of speech data obtained through the speech recognition session. This adaptaion method has been successfully applied to a hidden Markov network (HMnet), which is an efficient representation of phoneme context-dependent HMMs. This speaker adaptation method has proven itself effective in improving the performance of a Japanese phrase recognition system.