In this paper a method to decompose a conventional feature space (LPC-cepstrum) into subspaces which carry information about the linguistic and speaker variability is presented. Principal component analysis is used to study the correlation between these sub-spaces. Oriented principal component analysis (OPCA) is then used to estimate a sub-space which is relatively speaker- independent. A method to estimate the dimensionality of the speaker independent sub- space is also presented. Original features can now be projected into the speaker independent sub-space to make them less sensitive to speaker variations. Finally the effectiveness of the proposed method in suppressing the speaker dependence is studied by experiments conducted on two different databases.