ISCA Archive Eurospeech 1993
ISCA Archive Eurospeech 1993

An experiment involving the consistency and reliability of voice quality ratings for different types of speech fragments

Guus de Krom

An experiment was performed in which 6 listeners were asked to rate voice fragments obtained from a variety of speakers on a number of (pathological) voice quality aspects, including the overall degree of deviance (grade), breathiness, and roughness. Four different types of stimuli were presented to each listener; one was based on connected speech fragments, the other 3 were based on different segments of a sustained vowel (the onset, a mid-vowel segment and the whole vowel). Analyses were focussed on measures of listener consistency, and rating reliability. Considering the higher physiological complexity of the connected speech fragments as compared to the vowel-type fragments, we hypothesized that deviant aspects of the voice would be more prominent in the former, resulting in more consistent and more reliable voice quality ratings for connected speech stimuli. Within the three vowel-type stimuli, we expected lowest consistency and reliability for the ratings of the postonset stimuli, for similar reasons. Results indicated that stimulus type had little effect on either within-, or between- listener consistency of the voice quality ratings. Rating reliability was also hardly influenced by stimulus type. When determined as a function of the overall degree of deviance of a voice, the reliability of breathiness and roughness ratings was slightly higher for the whole vowel and vowel onset than for the connected speech and post-onset stimuli. We therefore conclude that connected speech stimuli are not necessarily to be preferred over vowel-type stimuli for a perceptual evaluation of pathological voice quality, and that the onset part of a vowel may contain voice quality cues that are less salient in the most stable part of a vowel.

Keywords: voice quality perception, listener consistency, rating reliability