Recognition of personality traits is a well studied problem in psychology while only recently it has been addressed by speech and language technology research. This paper describes annotation and experiments towards automatically inferring speakers personality traits in spontaneous conversations. In the first part, the work describes the annotation framework based on the Big-Five personality traits model (Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness) applied to 128 speakers from the AMI corpus. As the corpus contains rich annotations, those data can generalize previous studies based on enacted speech or dialogues. In the second part, the paper describes experiments based on various features including prosody, words n-gram, dialog acts and speech activity. Results reveal that high/low extraversion, consciousness and neuroticism traits can be automatically recognized with accuracy rate of 74.5%, 67.6% and 68.7%, respectively, while agreeableness and openness classification error rates are not statistically better than chance. Non-linguistic features (prosody, speech activity, overlaps and interruptions) outperform linguistic features (words n-gram and dialog acts) in this setup.