Standard statistical language modeling techniques suffer from sparse-data problems when applied to real tasks in speech recognition, where large amounts of domain-dependent text are not available. In this work, we introduce a modified representation of the standard word n-gram model using part-of-speech (POS) labels that compensates for word and POS usage differences across domains. Two different approaches are explored: (i) imposing an explicit transformation of the out-of-domain n-gram distributions before combining with an in-domain model, and (ii) POS smoothing of multi-domain n-gram components. Results are presented on a spontaneous speech recognition task (Switchboard), showing that the POS smoothing framework reduces word error rate and perplexity over a standard word n-gram model on in-domain data, with increased gains using multi-domain models.