In this paper, a hybrid network based on the combination of Radial Basis Function Networks (RBFNs) and Gaussian Mixture Models (GMMs) is proposed and used for speaker recognition. The hybrid network is a hierarchical one, where a GMM is built for each speaker and an RBFN is built for each group of speakers. The GMMs and RBFNs are trained independently. The RBFNs are used as a first stage coarse classifier and the GMMs are used as the final classifier. For each RBFN, only the first several candidates are chosen to take part in the final classification. The hybrid system is used for the SPIDRE database speaker recognition. Some experiments were carried out to choose the proper structure and parameters of RBFNs and GMMs. After using RBFNs, about 40% speakers were excluded without decreasing the performance. If the most confusable speaker sets in GMMs are grouped into RBFNs, the performance of GMMs can be increased more by using RBFNs.