Experiments with Automatic Speech Understanding (ASU) problems were presented in [1], in which the underlying acoustic and syntactic-semantic structure of the task was tried to be automatically learned into a global model. Results indicated that, while a Simple Recurrent Network (SRN) was actually able to capture most of the acoustic features of the ASU task, the syntactic-semantic structure was difficult to learn. We propose here appropriate modifications to this SRN which significantly improve the understanding rates.