Huntington's disease (HD) is an autosomal-dominant neurodegenerative disorder that leads to the devastating loss of motor control - including severe speech impairment. Current models are insufficient to predict the onset or progression of manifest symptoms and early signs of the disease remain challenging to detect and monitor. Therefore, we propose a purely speech-based, non-invasive approach to discriminate Huntington's Disease patients who are exhibiting early signs of disease from those who are not. We study various features derived from speech and machine learning models to classify HD patients. Our results show that Random Forest classifiers leveraging language features perform very well with an unweighted accuracy of 0.95. In addition, we analyze the statistical significance of features, the importance of different questions asked to the patients, and other classification problems in Huntington's disease to provide a strong foundation for this field of research.