A severe sparse data problem is faced when building HMMs of syllable units due to the uneven distribution of syllables in natural speech. In this paper we present a novel approach to building syllable HMMs which attempts to overcome this problem. The method involves tying the states in syllable models using a bottom-up clustering algorithm and a state similarity measure employing phonetic information. Experiments using the new approach on the TIMIT database show that it improves the recognition accuracy of syllable HMMs. We present encouraging results which show that when the state-tying method is used in conjunction with a Multi- Model [11] approach to acoustic modeling a syllable identification accuracy of 53.4% can be achieved. This equates to a phoneme accuracy of 72.8% which is comparable with the best results achieved using triphone HMMs.