The basic technologies for speech input and output are ready for large-scale implementation, though improvements in several areas will certainly be made. What is still missing is an integrated framework for evaluation and design, in which the existing know-how about speech input and output ergonomics can be embedded. In this paper we describe the components of such a framework and a strategy for design-oriented system evaluation. We will focus on formative evaluation, i. e. a comprehensive scheme of assessment which can be used during speech system design. The proposed approach has been accepted within the ESPRIT project SUNSTAR as the common framework for assessment. Keywords: speech input and output assessment, dialogue evaluation, application evaluation, evaluation strategy, evaluation layers, evaluation criteria, evaluation methods