In this paper we present a system to translate speech into its semantic representation. The system consists of a semantic model, a representation of each semantic unit and phone HMM. All the models of the basic system are infered automatically from samples. However, some improvements are achieved if a keyword list is provided to the inference algorithm of the semantic units. The system decodes correctly 85.5% of the semantic units. 60.0% of the sentences are transcribed exactly as the reference labels.