In this paper, we describe an environment for supporting collaborative problem solving that uses dialogue agents both for creating a collaborative attitude between students as well as for offering instruction. We evaluated the effect of the social dialogue agents on student collaboration by contrasting a condition that included the social agents with a condition that did not include them. Both conditions involved dialogue agents for offering math instruction. Our finding is that the social agents changed the attitude students displayed towards one another as well as their perceptions of how much help they gave a received. There was some weak evidence suggestive of a positive learning effect.