We propose a method of robust language modeling for a small amount of training text corpus. In this method, the word bigram and the class bigram are combined using a weighting function of preceding word frequency. We made experiments on speech recognition using JNAS speech corpus. As the results, it was proved that the performance of the class combined bigram is equivalent to that of the word bigram trained with 2.5 larger size of corpus. We also made experiments using sports news dialogue on TV. Recognition accuracy of the class-combined bigram was 83.3% that was 5.5 point higher than that of the word bigram.