In the ARISE project we developed an experimental spoken dialogue system for access to train timetable information of the Dutch Railways. It is based on an existing system (VIOS), which became operational during the course of the project. This paper discusses a number of issues that came to light during evaluations of the ARISE system. We are able to shorten the dialogue with the system considerably, by using confidence measures, short prompts and a zooming exceptions handling strategy. The problems that we observed are all related to the fact that both the dialogue system and the user are unable to infer each other’s intentions and capabilities. Also, evaluation of a dialogue system is difficult since the evaluator has little means to infer the real intentions of the user. Working with predefined scenarios is not the optimal solution for this problem.