In this paper, an approach is presented to improve the existing performance of the Recognizer Output Voting Error Reduction (ROVER) procedure used for speech decoders' combination in automatic speech transcription. A contextual analysis is injected within the ROVER process to detect and eliminate erroneous words. This filtering is carried out through the combination of automatic error detection techniques. Experiments showed it is possible to outperform the ROVER baseline, and that combining it with error detection methods leads to an even lower Word Error Rate (WER) in the final ROVER composite output.