ISCA Archive Interspeech 2014
ISCA Archive Interspeech 2014

Evaluating coherence in open domain conversational systems

Ryuichiro Higashinaka, Toyomi Meguro, Kenji Imamura, Hiroaki Sugiyama, Toshiro Makino, Yoshihiro Matsuo

We propose a method for evaluating coherence between user utterances and those generated from open domain conversational systems. Our aim is to make it possible for such systems to ascertain whether utterances generated from them are appropriate to the context before generation so that possible breakdown in conversation arising from inappropriate utterances can be avoided. In our method, we train a classifier that distinguishes a pair of a user utterance and that generated from a system as coherent or incoherent by using various pieces of information related to dialogue exchange, such as dialogue acts, question types, and predicate-argument structures. Experimental results show that our method significantly outperforms the baseline, confirming its effectiveness.

doi: 10.21437/Interspeech.2014-38

Cite as: Higashinaka, R., Meguro, T., Imamura, K., Sugiyama, H., Makino, T., Matsuo, Y. (2014) Evaluating coherence in open domain conversational systems. Proc. Interspeech 2014, 130-134, doi: 10.21437/Interspeech.2014-38

  author={Ryuichiro Higashinaka and Toyomi Meguro and Kenji Imamura and Hiroaki Sugiyama and Toshiro Makino and Yoshihiro Matsuo},
  title={{Evaluating coherence in open domain conversational systems}},
  booktitle={Proc. Interspeech 2014},