The use of several n-gram and hybrid language models with and without cache is examined in the context of producing court transcripts. Language models with cache (in which words which have recently been uttered are preferred) have seen considerable use. The suitability of cache models (with fixed size cache) in the production of court transcripts is not clear. A decrease in perplexity and an improvement in the word error rate is observed with some of the models when using a cache, however, performance deteriorates with increasing cache size.