An analysis of a feed-forward neural network model performing a vowel recognition task is reported. Two experiments analysed in detail involved training on speech processed using linear prediction and the discrete FFT. A phonetic analysis of the resultant weight matrices is offered, additionally we analyse the network structure using pairwise correlations and principal components analysis. We suggest methods of network initialisation obtained by inverting these analyses.