Many studies have shown the advantage of using ideal binary mask (IdBM) to improve the intelligibility of speech masked by interfering voices. The present work further investigated the segmental contribution to the intelligibility of the IdBM-processed sentences. Three types (i.e., vowel-only, consonant-only, and vowel-consonant transition) of Mandarin IdBM-processed stimuli were generated by using a noise-replacement paradigm to preserve the selected segments and replace the rest with noise. Normal-hearing subjects participated in listening experiments to recognize the IdBM-processed sentences. Experiment results showed that the recognition score of the vowel-only IdBM-processed sentences was significantly higher than that of the consonant-only IdBM-processed sentences, suggesting a greater contribution of vowels than consonants to the intelligibility of the IdBM-processed sentences. Vowel centers contained rich intelligibility information. While consonants and vowel-onset carried little intelligibility information, adding a small proportion of vowel-consonant transition significantly improved the recognition score of the consonant-only and vowel-onset IdBM-processed sentences.