Additive bias compensation is a simple and effective technique to overcome the performance degradation caused by acoustic mismatch in speech recognition systems. Bias is usually estimated in a batch mode, assuming that its parameters are constant for the whole utterance. This paper develops a new sequential algorithm for additive bias estimation, which can potentially track time varying mismatch effects within a test utterance. Relation to recursive Kullback-Leibler technique is pointed out, and the method is tested using computer simulations and speech recognition experiments. Significant performance improvements in the recognition rate are obtained for supervised adaptation.