This paper gives an overview of the research that we have carried out at Linkoping University on cognitive aspects of visual speech processing. First, a cognitive, individual difference perspective on speechreading is introduced, with a specific emphasis on the cognitive architecture of speechreading skill. Second, common cognitive characteristics of exceptionally skilled speechreaders is described. In the final part, a working memory model for visual speechreading is outlined. The model includes three parts; one with amodal cognitive constraints, one with poorly specified language input, and a bottleneck between the modal and amodal parts that is constituted by a semi-abstract phonological processor.