In this paper, we explore how 'phone transition' models can be efficiently integrated into a segment-based continuous speech recognizer using Multi Layer Perceptrons. In this transition-controlled recognition strategy, the acoustic continuum of speech is modeled as a sequence of transitions between phones, but the advantages of segment-based modeling are retained by performing a kind of non-linear interpolation with context-independent phone probabilities. Experiments show that the transition-based phone recognizer significantly outperforms the context-independent one.