ISCA Archive Interspeech 2024
ISCA Archive Interspeech 2024

Investigation of look-ahead techniques to improve response time in spoken dialogue system

Masaya Ohagi, Tomoya Mizumoto, Katsumasa Yoshikawa

This paper reports a new method that improves the response speed in spoken dialogue systems that use large language models. In existing systems, the start of the chatbot’s response after the user utterance is delayed by the time required to generate that response. In contrast, our system predicts what the user may say next and pre-generates the bot’s response before the user finishes speaking. This look-ahead technique allows the response to be returned by simply matching the predicted user utterance with the actual user utterance. Evaluation results show that our method has high look-ahead accuracy in task-oriented dialogue, contributing to improved response speeds.