In this paper, prosody-based attitude recognition and its application to a spoken dialog system are discussed. Para-linguistic information plays a important role in the human communication. We aimed to recognize the user's attitude by prosody, and apply it to a spoken dialog system as para-linguistic information. In order to find important features to recognize the attitude from automatically extracted features, we applied some feature selection methods. Experimental results show the stepwise method, a combination of the forward selection method and the backward selection method, achieved the best recognition rate and some important features effective for attitude recognition are revealed. Finally, the dialog system using the speaker's attitude as para-linguistic information is developed.