In this paper, we present a new modeling approach for speaker recognition, which uses a kind of novel phonotactic information as the feature for SVM modeling. Gaussian mixture models (GMMs) have been proven extremely successful for textindependent speaker recognition. The GMM universal background model (UBM) is a speaker-independent model, each component of which can be considered to be modeling some underlying phonetic sounds. Thus, the UBM can be regarded to characterize a speaker-independent voice. We assume that the utterances from different speakers should get different average posterior probabilities on the same Gaussian component of the UBM, and the supervector composed of the average posterior probabilities on all components of the UBM for each utterance should be discriminative. We use these supervectors as the features for SVM based speaker recognition. Experiment results show that the proposed approach demonstrates comparable performance with the state-of-the-art systems on NIST 2006 SRE corpus. Fusion results are also presented. Index Terms— Speaker recognition, gaussian mixture model, universal background model, support vector machine