This paper concerns the adaptation of automatic speech recognisers to new speakers. Existing recognisers, in their training and adaptation, treat between-speaker variation essentially as acoustic 'noise' and ignore structuring which originates at higher levels, caused for instance by accent differences. If a large-vocabulary recogniser is to cope efficiently with a realistic range of speakers it will have to incorporate linguistic knowledge about accents. A solution to the problem of disentangling accentual and personal characteristics of new voices is outlined, and the subsequent adaptation of different components of a recogniser is discussed.