There have been various approaches to separate mixed voices. In the real world, input voices contain many different kinds of background sounds but existing methods have not considered the background sounds in model architectures. These approaches are difficult to control the background sounds directly and the voice separation results include the background sounds randomly. In this paper, we propose an extended voice separation framework, background-sound controllable voice source separation that can control the degrees of background sounds of voice separation outputs using a control parameter that ranges from 0 to 1 without additional mixing procedures. Several experiments show the controllability of background sounds on various real world datasets with preserving voice separation performances.