In this paper we examine some of the issues involved in analyzing the expression of emotions, mental states and attitudes in speech. We investigate timing issues that may affect the structure of future automated inference systems. These timing issues include tracking transitions and changes over time within expressions, existence of thresholds between expressions, and the necessity of tracking time-frequency features within an utterance. We also suggest an outline for a system that includes these features.