The work presented in this paper describes a novel on-line adaptation approach for extremely low adaptation data scenario. The proposed approach extends a similar redundant dictionary based approach reported recently in literature. In this work, the orthogonal matching pursuit (OMP) algorithm is used for bases selection instead of the matching pursuit (MP). This helps in avoiding the selection of an atom more than once. Furthermore, this work also explores the use of cluster-specific eigenvoices to capture local acoustic details unlike the conventional eigenvoices technique. These approaches are then combined to reduce the number of weight parameters being estimated for deriving adapted model. Towards this purpose, separate sparse coding of the test data is performed over a set of dictionaries. Those sparse coded supervectors are then scaled and used as the Gaussian mean parameter in the adapted model. Consequently, only a few scaling factors are needed to be estimated. Such a reduction in number of parameters is highly desirable for on-line applications where the latency is a major factor.