Semi-continuous Hidden Markov Models (SCHMM) with gaussian distributions are often used in continuous speech or handwriting recognition systems. Our paper compares gaussian and tree-structured polynomial classiffiers which have been successfully used in pattern recognition since many years. In our system the binary classiffier tree is generated by clustering HMM states using an entropy measure. For handwriting recognition, gaussians are clearly outperformed by polynomial classiffication. However, for speech recognition, polynomial classiffication currently performs slightly worse because some system parameters are not yet optimized.