In this paper, we investigate improving named speaker identification with the help of pretrained language models. First, we experiment with a supervised approach where the content of each speaker's utterances in training data is used to finetune an encoder-based BERT-style language model. Next, we experiment with large generative language models, demonstrating their ability to perform zero-shot named speaker recognition using text transcripts. In both scenarios, we experiment with two languages, including VoxCeleb1 speaker identification dataset and three Estonian broadcast news and conversational datasets. We show that large language models can provide dramatic improvements to named speaker identification performance on conversational speech where speakers are introduced by their name. Furthermore, the OpenAI GPT-4 model sometimes surpasses human performance in recalling Estonian speaker names from public debate transcripts.