This work explores the use of DNN/RNN for extracting Baum-Welch sufficient statistics in place of the conventional GMM-UBM in speaker recognition. In this framework, the DNN/RNN is trained for automatic speech recognition (ASR) and each of the output unit corresponds to a component of GMM-UBM. Then the outputs of network are combined with acoustic features to calculate sufficient statistics for speaker recognition. We evaluate and analyze the performance of networks with different configurations and training corpuses in this paper. Experimental results on text-independent SRE NIST 2008 and text-dependent RSR2015 speaker verification tasks show the robustness of DNN/RNN for extracting statistics in mismatched evaluation conditions compared with GMM-UBM system. Particularly, Long Short-Term Memory (LSTM) RNN realized in this work outperforms traditional DNN and GMM-UBM in most mismatched conditions.