This paper briefly reviews current-practice usability evaluation methods and criteria for spoken dialogue systems. We then describe how two commercial and one research system were evaluated with respect to usability and discuss similarities and differences. Finally, we discuss the industrial need for cheaper ways of evaluating usability and the need to pursue research on usability in a field in which the technological capabilities of systems continue to improve and diversify at a rapid pace.