A totally constraint-based computational architecture is applied to integration of speech and natural language processing. A major research issue in designing information processing systems based on constraint (level of description abstracting away from information flow) is how to guarantee global adequacy of computation. A probabilistic semantics of Horn clause programs is introduced, which is a generalization of hidden Markov models, stochastic context-free grammars, etc., and a computational method for maximum likelihood estimation is proposed. This computation deals efficiently with probabilistically dependent events, and is regarded as a sort of A* search in a general sense. Furthermore, this computational architecture supports omnidirectional information flow among heterogeneous knowledge sources, from acoustics to pragmatics, and naturally resolves problems in spoken language understanding with-out any domain/task dependent prescription of information flow.