Neural vocoder has been studied for years, aiming at modeling speech signals and enabling speech signal reconstruction from acoustic features. Unfortunately, the existing end-to-end neural vocoders lack revealing the intrinsic structure of speech due to their black-box nature, thus losing the ability to flexibly synthesize or modify the speech with high quality. Moreover, they usually require complicated networks to generate speech with substantial time consumption. In this paper, we are inspired by the quasi-harmonic model (QHM) and propose a neural vocoder incorporating QHM for network architectures. In this way, speech signals can be parameterized into quasi-harmonic components and be arbitrarily resynthesized with a high quality where the time consumption and network size prominently decrease. The experiments indicate that the proposed method combines the advantages of QHM and neural vocoders and outperforms other methods, such as HiFi-GAN, in terms of generation speed and quality.