Recent advances in Audio-Visual Speech Recognition (AVSR) have led to unprecedented results in the field, significantly enhancing its robustness in noisy environments. However, the current state of the art mostly relies on models comprising vast amounts of parameters and resource-intensive pre-training procedures. To address this, emerging research is focusing on developing more portable, lightweight alternatives, aiming to maintain high performance while reducing computational demands. This paper explores the Sim-T architecture for the design of parameter-efficient and non-autoregressive AVSR systems for continuous Spanish in the wild. Experimental results across varied data conditions and domains demonstrate the method’s effectiveness, reducing model complexity by up to 70% without significant performance losses. Additional assessments under noise settings highlight, however, the need for further research to improve the robustness of lightweight AVSR in these adverse scenarios.