This study evaluates the reliability of an extended reading miscue annotation protocol applied to a Dutch child read-aloud speech corpus. Two annotators reviewed existing annotations to identify confusing or inconsistent cases. The results indicate that while the protocol allows for fine-grained analysis, several challenges remain. These include insufficient guidelines for labeling disfluent attempts, overlapping or ambiguously defined labels, difficulties in judging reduced pronunciations, and overly detailed grapheme/phoneme distinctions. These issues often led to annotator confusion and inconsistent labeling, underscoring the need for protocol refinement. Refining this protocol can lead to more accurate automated reading assessment tools and a deeper understanding of reading development in children, ultimately improving diagnostic precision.