Adversarial examples pose a security threat to Autopilot's speech command recognition module, which attracted widespread attention from researchers. Previous works purify the malicious adversarial perturbations through pre-processing data from the time and frequency domain information. However, these methods either have a weak purification capacity or require a significant purification cost. To tackle this problem, we propose a real-time and efficient purification-based defense method DualPure, which combines the two defense aspects in the time and frequency domain for copurification. Specifically, we first disrupt the potential malicious perturbation in the sample at the waveform level and then apply an unconditional diffusion model to purify the feature at the frequency level. Numerous experiments show that the proposed method can effectively purify and achieve good adversarial robustness in white-box attacks (+ 6.3%) and black-box attacks (+ 1.08%).