Speaker identification has traditionally been performed by pattern matching an utterance from an unknown speaker to all the speaker models in the database. By clustering speakers into a fixed number of groups and performing speaker group identification instead, the computation required may be much less when the population of enrolled speakers becomes very large in size. This paper introduces the idea of speaker set identification and ways to efficiently accomplish speaker identification over a large population of speakers through speaker set identification and speaker clustering.