Following Self-Determination Theory, speakers can communicate either in more controlling or more autonomy-supportive ways. Whereas most previous studies focused on the content of both communication styles, the current research examined whether experimentally induced controlling versus autonomy-supportive tone of voice differentially predicts listeners’ experienced autonomy frustration, closeness, intention for collaboration, and feelings of sadness and anger, even when listeners are exposed to these communications only briefly. In three studies with adults (Study 1, N = 61; Mage = 31.51), parents (Study 2, N = 111; Mage = 44.73), and toddlers (Study 3, N = 189; Mage = 4.93), multilevel analyses indicated that sentences spoken with a harsher (i.e., with increased vocal energy), relative to a softer tone of voice were perceived as more pressuring, leading to higher levels of experienced autonomy frustration (Study 1, 2 and 3). Listening to such harsh voices explained why listeners felt less close to and were less inclined to collaborate with controlling speakers (Study 2) and reported higher levels of anger and fear (Study 3). Results for the first time show the impact of speakers’ tone of voice on listeners across ages, with adults and toddlers alike reporting more maladaptive effects following controlling tone of voice.