ISCA Archive Eurospeech 1993
ISCA Archive Eurospeech 1993

Reliable assessment of speech recognisers for telephone environment

A. Riccio, F. Ceglie, A. Brancaccio

The need for a standardised assessment of Automatic Speech Recognisers has been already emphasised in the framework of several Research Projects and in particular in Europe in the context of ESPRIT (SAM, SUNSTAR). In general the procedures proposed and the audio material available are fully suitable for a laboratory assessment of the performances of an ASR, but as soon as the application field is concerned, the limitations of such methodologies becomes evident. In particular the adoption of speech databases recorded in quiet conditions (clean speech) and the total absence of any kind of impairment on the communication channel are the main reasons of unreliability. Therefore on the basis of the work already initiated inside the SUNSTAR Project and as an enhancement of the performances of the standard SESAM workstation, the paper firstly concentrates on the generation of telephone grade speech corpora and a methodology to collect them; secondly it describes the realisation of an assessment experiment in the telephone environment aiming at an evaluation of the robustness of the DIVA recogniser (and in particular of one of its component, the end-point detector). The assessment has been performed in according to the procedures developed in the ESPRIT 2589 (SAM) project.