In this paper, we address the threat of data poisoning attacks by proposing a novel method for detecting and isolating poisoned samples. Our approach uses a variational autoencoder (VAE) trained in an unsupervised fashion on the manipulated dataset. By performing per-class clustering and statistical analysis of the latent vectors, we can identify poisoned classes and separate clean and poisoned samples. We evaluate our method on an audio dataset and demonstrate that we outperform two popular baseline defenses. Furthermore, we show the generalizability of a single trained VAE model in exposing a variety of different poisoning attacks against the same dataset.