This paper reports on the development of a foreign speaker accent classification system based on phoneme class specific accent discrimination models. This new approach to the problem of automatic accent classification allows fast and reliable prediction of the speaker accents for continuous speech through exploitation of the accent specific information at the phoneme level. The system was trained and evaluated on a corpus representing three speaker groups with native Australian English (AuE), Lebanese Arabic (LA) and South Vietnamese (SV) accents. The speaker accent classification rates achieved by our system come close to the benchmarks set by human listeners.