This paper describes a multimodal spoken language system, which allows users to furnish a virtual 3D living room. Speech and touch modalities are tightly integrated in a single user input turn. An advanced dialogue manager allows users to have a natural flexible style of interaction with the "virtual assistant" (a talking head). The system is mixed initiative, with a system intelligence which allows the assistant to make expert suggestions on room design. This paper concentrates on the architecture and components which were needed to achieve these results.