This paper describes a new vector-quantization-based phoneme recognition method which uses the hierarchical time spectrum pattern (TSP). The phonetic feature is discussed by a mutual information and a posteriori probability between vector quantization codes and phoneme label codes. The TSP of Mel-scaled LPC cepstra and the power-change pattern (PCP) are used as acoustic parameters. Input speech is firstly vector-quantized by the PCP codebook. Secondly it is vector-quantized by the TSP of which the codebooks are classified by the PCP-VQ code. Hierarchical TSP-VQ improves performance of phoneme classification compared with only the TSP-VQ. A frame-label matching experiment on a speaker-independent condition with the JEIDA Japanese speech database of connected 4-digit uttered by 16 males and 16 females, shows 79.4% of recognition accuracy using the method. The experimental result indicates that the proposed method is highly effective.