In developing a text-to-speech system, it is well known that the accuracy of information extracted from a text is crucial to produce high quality synthesized speech. In this paper, by transferring probabilistic natural language processing techniques into TTS system field, we develop a more robust text analyzer with high accuracy for Korean TTS systems. The proposed system is composed of five modules: a preprocessor, a morphological analyzer, a part-of-speech tagger, a grapheme-to-phoneme module, and a parser. Among these modules, the part-of-speech tagger and the parser are designed under probabilistic framework, and trained automatically. Given a text, our system produces the structures of word phrases, word pronunciations, and governor-dependent relationships that represents the structure of the sentence. Experimental results showed that the tagger got 90.33% correctness for finding the structure of word phrases in the word level, and the parser, 80.87% for finding governor-dependent relationships of sentences respectively.