ISCA Archive ISCSLP 2008
ISCA Archive ISCSLP 2008

Subword Latent Semantic Analysis for TextTiling-based Automatic Story Segmentation of Chinese Broadcast News

Yu-Lian Yang, Lei Xie

This paper proposes to perform latent semantic analysis (LSA) on character/syllable n-gram sequences of automatic speech recognition (ASR) transcripts, namely subword LSA, as an extension of our previous work on subword TextTiling for automatic story segmentation of Chinese broadcast news. LSA represents the ‘meaning’ of a lexical term by a feature vector conveying the term’s relations with other terms. We apply subword LSA vectors to the measurement of inter-sentence lexical score in TextTiling-based story segmentation. Subword n-grams are robust to speech recognition errors, especially out-of-vocabulary (OOV) words, in lexical matching on Chinese ASR transcripts. This work combines the concept matching merit of LSA and the robustness of subwords. Experimental results on the TDT2 Mandarin corpus show that subwordLSA-based TextTiling can effectively improve the story segmentation performance. Character-bigram-LSA-based TextTiling achieves the best F1-measure of 0.6598 with relative improvement of 17.4% over the conventional word-based TextTiling and 6.5% over our previous syllable-bigram-based TextTiling. Index Terms— latent semantic analysis, TextTiling, story segmentation, topic segmentation, spoken document retrieval, subword