In this paper, a robustness and discrimination oriented score function for integrating speech and language information is proposed. This unified approach uses probabilistic score functions to characterize different levels of knowledge in a uniform way. By jointly considering the knowledge from different levels, ranging from acoustics to syntax, the processing capability of speech and language are both enhanced with the information provided by the other module. To enhance the performance, the score function was modified to directly pursue correct rank ordering in the testing set, instead of pursuing maximal likelihood in the training set. This formulation has been applied to a Chinese phonetic typewriter task, and have considerably improved the performance of our system.