In order to solve the problem of the performance decrease when state-of-art automatic speech recognition (ASR) system facing accent speech, we propose the Pronunciation Variation Model (PVM). Two approaches are proposed to construct the PVM in this paper. 6.38% and 7.78% relative error rate reduction is achieved for Shanghai and Wuhan accent mandarin, respectively. The experiment on these two typical accent mandarin shows it is a possible way to deal with accent speech.