The goal of this paper is to compare two different methodologies for automatic testing of isolated word speech recognisers. They are usually tested by playing single speech tokens and collecting the device response. The second approach, allows to output speech files containing lists of words and simultaneously to collect recogniser answers. It ensures a more realistic testing of the performance of any kind of isolated word speech recogniser. The results obtained are significantly lower than the ones got with isolated word testing.

Furthermore many recognisers, designed to work in noisy environment, cannot be correctly tested in isolation as they need to be continuously fed by background noise.