In this paper, we propose a multi-stream approach that selectively uses Missing Frequency Band HMMs (MFB-HMM) that is trained on the band-eliminated speech. This makes the model insensitive to the noise in the missing frequency band. With multiple MFB-HMMs of different missing frequency bands, the proposed recognition system is robust in various types of noise conditions. Recognition experiments show that the selective use of the MFB-HMMs is very effective in narrow band noise condition even if the noise is unstationary, however, the improvements of the performance to general noisy conditions, e.g. in-car noise and music sound, are not as high as in the narrow band noise case. The results of the experiments also show that the optimal selection of the MFB-HMM significantly improves the performance regardless of the type of the noise; therefore, the model selection measure is the key issue in this method.