Automatic prediction of amyotrophic lateral sclerosis (ALS) disease progression provides a more efficient and objective alternative than manual approaches. We propose ALS longitudinal speech transformer (ALST), a neural network-based automatic predictor of ALS disease progression from longitudinal speech recordings of ALS patients. By taking advantage of high-quality pretrained speech features and longitudinal information in the recordings, our best model achieves 91.0%AUC, improving upon the previous best model by 5.6% relative on the ALS TDI dataset. Careful analysis reveals that ALST is capable of fine-grained, interpretable predictions of ALS progression.