This paper presents a case study analyzing the results of an on-going trial of a prototype mixed-initiative spoken dialog system for telephony control and messaging. System usage and performance data were captured at three points in time. Information from multiple data sources, including spoken utterances, system call logs, speech recognizer output, and subjective surveys was evaluated to determine the relationship between aspects of system performance and user perceptions of the system. This report provides several examples using these data sources in combination to identify key areas to focus on in modifying the system, application, and/or user interface in order to significantly improve system usability and user satisfaction.