This paper reports on the investigation of diagnostic and predictive assessment techniques in testing a recogniser on the EUROM.1 speech database, produced within the ESPRIT Project 2589 SAM.
Based on test results obtained on the full set of CVC-type (Consonant-Vowel-Consonant) words contained within the Danish Few Talker Part of the database, three different assessment techniques are investigated in this paper. In the first, the substitution errors encountered are related to an analysis of the Danish phoneme inventory in terms of a representation of the phonetic distinctive features and the results show that many frequent substitutions found in a con-f fusion matrix may be predicted as seen from the distinctive-feature representation.
In the second, a statistical analysis is carried out in which the asymmetric confusion matrices are transformed into symmetrical similarity matrices and applied in an individual differences multi-dimensional scaling analysis. This data-driven characterisation of the phoneme-inventory is illustrated in two-dimensional plots, and a high degree of correspondence with the distinctive feature map of phonemes is observed. This indicates possibilities of calibrating the performance of a recogniser against a given language by specifying the limitations of the recogniser in terms of phonetic dimensions specific to the language.
In the third, the estimated phoneme distances from testing a target recogniser on the EUROM.1 CVC material are used in a recogniser response model in order to aim at predicting the performance of the target recogniser on a different vocabulary. A limited experiment concludes that the observed performance of the target recogniser on the different vocabulary does not vary significantly from the predicted.