ISCA Archive Interspeech 2014
ISCA Archive Interspeech 2014

Enhancing multimodal silent speech interfaces with feature selection

João Freitas, Artur Ferreira, Mário Figueiredo, António Teixeira, Miguel Sales Dias

In research on Silent Speech Interfaces (SSI), different sources of information (modalities) have been combined, aiming at obtaining better performance than the individual modalities. However, when combining these modalities, the dimensionality of the feature space rapidly increases, yielding the well-known “curse of dimensionality”. As a consequence, in order to extract useful information from this data, one has to resort to feature selection (FS) techniques to lower the dimensionality of the learning space. In this paper, we assess the impact of FS techniques for silent speech data, in a dataset with 4 non-invasive and promising modalities, namely: video, depth, ultrasonic Doppler sensing, and surface electromyography. We consider two supervised (mutual information and Fisher's ratio) and two unsupervised (mean-median and arithmetic mean geometric mean) FS filters. The evaluation was made by assessing the classification accuracy (word recognition error) of three well-known classifiers (k-nearest neighbors, support vector machines, and dynamic time warping). The key results of this study show that both unsupervised and supervised FS techniques improve on the classification accuracy on both individual and combined modalities. For instance, on the video component, we attain relative performance gains of 36.2% in error rates. FS is also useful as pre-processing for feature fusion.

doi: 10.21437/Interspeech.2014-297

Cite as: Freitas, J., Ferreira, A., Figueiredo, M., Teixeira, A., Dias, M.S. (2014) Enhancing multimodal silent speech interfaces with feature selection. Proc. Interspeech 2014, 1169-1173, doi: 10.21437/Interspeech.2014-297

  author={João Freitas and Artur Ferreira and Mário Figueiredo and António Teixeira and Miguel Sales Dias},
  title={{Enhancing multimodal silent speech interfaces with feature selection}},
  booktitle={Proc. Interspeech 2014},