Conventional MLP classifiers used in phonetic recognition and speech recognition may encounter local minima during training, and they often lack an intuitive and flexible adaptation approach. This paper presents a hybrid MLP-SVM classifier and its associated adaptation strategy, where the last layer of a conventional MLP is learned and adapted in the maximum separation margin sense. This structure also provides a support vector based adaptation mechanism which better interpolates between a speaker-independent model and speaker-dependent adaptation data. Preliminary experiments on vowel classification have shown promising results for both MLP learning and adaptation problems.