In this paper, a new approach of Latent Prosodic Modeling (LPM) for analyzing the prosody of speech is presented. Based on a set of newly defined prosodic characters, prosodic terms, documents, and the Probabilistic Latent Semantic Analysis (PLSA) framework, prosody can be modeled using a set of prosodic states representing various latent factors such as speakers, speaking rate, utterance modality, intonation behavior, etc. in terms of some probabilistic relationships with the observed prosodic features. Organizing the training data based on this new model may also produce more delicate classification models for various speech processing applications considering the prosody. In the initial application example, we showed the use of this model on the task of disfluency IP detection for spontaneous Mandarin speech recognition, and improved IP detection accuracy and speech recognition performance were obtained in the experiments.