We propose an incremental unsupervised adaptation method based on reinforcement learning in order to achieve robust speech recognition in various noisy environments. Reinforcement learning is a training method based on rewards that represents correctness of outputs instead of supervised data. Training progresses gradually based on rewards given. Our method is able to perform environmental adaptation without priori knowledge about such things as speakers and noises in noisy environments. We conducted speech recognition experiments using a connected digit recognition database. We demonstrate that our method has higher recognition performance than the conventional adaptation method.