During production of emotional speech there are deviations in the components of speech production mechanism when compared to normal speech. The objective of this study is to capture the deviations in features related to the excitation source component of speech, and to develop a system for automatic recognition of emotions based on these deviations. The emotions considered for this study are: anger, happy, neutral and sad. The study shows that there are useful features in the deviations of the excitation source features at subsegmental level, and they can be exploited to develop an emotion recognition system. A hierarchical binary decision tree approach is used for classification.