This paper describes the semantic interpretation of user utterance in the IMAR project (multimodal spoken dialogue system). In a human-computer dialogue, a good sentence understanding is a prime necessity to have a good progression in the exchange. The heaviness of an interaction is due to the misunderstanding and the faltering speak of the user. The aim of this paper is to propose a robust spoken understanding system capable to cover effects of spontaneous speech (filled pauses, non grammatical structure), the misrecognition (unknow words) and the ambiguity of the language. The method is based on the use of several linguistic knowledges (syntactic, semantic and pragmatic) and also the context of the interaction (user and task model).