This paper reports a study of the perception of affective information in conversational speech utterances and shows that there are consistent differences in the acoustic features of same-word utterances that are perceived as having different discourse effects or displaying different affective states. We propose that rather than selecting one label to describe each utterance, a vector of activations across a range of features may be more appropriate. This finding complicates the representation of speech elements, but offers a more appropriate description of their attributes.