This paper describes a Chinese recognition and synthesis system based on syllables that is a part of research efforts recently toward man-machine communication in unlimited vocabulary or very large vocabulary. We believe that the first step of this effort is a Chinese recognition and synthesis system of all Chinese syllables, because of special structure of Chinese which is tone language and monosyllable based language. The high recognition accuracy of syllable ( 97% for 5 candidates ) and rather natural and intelligible synthesizing speech is achieved in real time.