We present a speech modeling methodolgy where no a priori assumption is made on the dependencies between the observed and hidden speech processes. Rather, dependencies are learned from data. This methodology guarantees improvement in modeling fidelity compared to HMMs. In addition, it gives the user a control on the trade-off between modeling accuracy and model complexity. Furthermore, the approach is technically very attractive because all the computational effort is made in the training phase.