Early diagnosis of reading difficulties requires assessment of children’s oral reading fluency. Since obtaining reliable assessments through subjective rating rubrics has proven to be difficult and time consuming, the current study presents a novel procedure incorporating Automatic Speech Recognition (ASR) to compute 15 measures related to the aspects phrasing, smoothness and pacing of oral reading fluency. We investigate the validity of these measures by comparing them to the same measures computed on human transcripts. This comparison was performed on a dataset of 244 recordings of texts read by 131 Dutch primary school children (aged 6-13 years). We found strong correlations for 12 out of 15 measures, which emphasizes the great potential of these measures for more reliable, less time-consuming and sustainable reading assessment.