Evaluation of human-computer spoken dialogues is often based on analyses of objective metrics such as task completion rate, turn-taking, time consumption, etc. While these data are easily obtained and processed from log files, they offer or very little information about the actual usability of the given service as perceived by the test subjects.

This study combines the evaluation based on objective data obtained from log files with subjective data, where the test subjects express their attitudes to a number issues directly related to the usability of the service. It is shown how the joint analysis can be used to support, but also question findings from either source.