The soft thresholding function has been used in wavelet packet structure to efficiently remove background noises. However, it suffers from both serious residual noise and speech distortion. The classical method that estimates the threshold of wavelet coefficients assumes the corrupted noise to be white Gaussian noise. In fact, the noise is generally not white in practical environments. This paper proposes a method to adapt wavelet coefficient thresholds (WCTs) of each wavelet subband using both the segmental SNR (SegSNR) and the noise masking thresholds (NMTs). The experiments shows that integrating SegSNR and NMTs to improve the wavelet-based speech enhancement method can efficiently remove the background noise and suppress the residual noise. Removing the infecting noise from the corrupted speech, with a little impact on speech, can be obtained by this proposed scheme.