ISCA Archive Odyssey 2001
ISCA Archive Odyssey 2001

Speaker Recognition Evaluation -- a challenge and an opportunity

George Doddington

As those who have played the game know, there are many ways to define the task of identifying speakers according to their voice characteristics. My personal favorite is a simple speaker detection task, with the requirement that the speaker recognition system output a Yes/No response to the target speaker hypothesis. I will argue for this task as the best task to support speaker recognition research. There are also many different technical and application parameters that affect the choice of technical approach and the performance that is achievable. I will survey these parameters. Finally, there are many distinct dimensions of variability that affect performance and that often, unfortunately, are not obvious, such as the dependency of performance error statistics on speaker identity (sometimes referred to as the sheep/goat/lamb/wolf menagerie). I will discuss some of these dependencies and argue that effort directed toward task definition and evaluation analysis is a good way to stimulate progress.