ISCA Archive Interspeech 2014
ISCA Archive Interspeech 2014

Manifold regularized deep neural networks

Vikrant Singh Tomar, Richard C. Rose

Deep neural networks (DNNs) have been successfully applied to a variety of automatic speech recognition (ASR) tasks, both in discriminative feature extraction and hybrid acoustic modeling scenarios. The development of improved loss functions and regularization approaches have resulted in consistent reductions in ASR word error rates (WERs). This paper presents a manifold learning based regularization framework for DNN training. The associated techniques attempt to preserve the underlying low dimensional manifold based relationships amongst speech feature vectors as part of the optimization procedure for estimating network parameters. This is achieved by imposing manifold based locality preserving constraints on the outputs of the network. The techniques are presented in the context of a bottleneck DNN architecture for feature extraction in a tandem configuration. The ASR WER obtained using these networks is evaluated on a speech-in-noise task and compared to that obtained using DNN-bottleneck networks trained without manifold constraints.


doi: 10.21437/Interspeech.2014-82

Cite as: Tomar, V.S., Rose, R.C. (2014) Manifold regularized deep neural networks. Proc. Interspeech 2014, 348-352, doi: 10.21437/Interspeech.2014-82

@inproceedings{tomar14_interspeech,
  author={Vikrant Singh Tomar and Richard C. Rose},
  title={{Manifold regularized deep neural networks}},
  year=2014,
  booktitle={Proc. Interspeech 2014},
  pages={348--352},
  doi={10.21437/Interspeech.2014-82},
  issn={2308-457X}
}