This paper describes the systems SRIB has proposed for task-1 of the inaugural MERLion CCS challenge in the closed domain and open domain. Our system for the closed task is based on an end-to-end conformer architecture trained for the task of automatic speech recognition using RNN-T loss, which is then transfer learned for the task of language classification. We train the ASR model initially to ease the task of learning the right features for the classification task. This system achieves a 13.9% Equal Error Rate (EER) and 81.7% Balanced Accuracy (BAC) on the evaluation set. For the open track, we use an ensemble of Open AI's Whisper model and one of the ASR models used our closed track. This system achieves 9.5% EER and 78.9% BAC on the evaluation set. Compared to the challenge baseline we observe relative improvements for EER of 35.9% in the closed track and 56.2% in the open track. We achieve 1st position on both the closed and the open track leaderboards.