This paper examines the difficulty subjects experience in communicating with natural language (NL) speech recognizers in real telephone services. We show that providing an NL interface is not sufficient for creating a comfortable experience for untrained users, and that customer confusion is a key challenge in building NL-based services. We examine three recent NL trials from both the technology perspective and the human perspective and show where improvements must be made before NL services can be broadly deployed.