Most of what we know about speech perception has been gleaned from tests in which listeners respond to stimuli chosen by an experimenter. This paper presents SpeechAdjuster, an open source tool that reverses the roles of listener and experimenter by allowing listeners direct control of speech characteristics in real-time. This change of paradigm enables listener preferences — reflecting factors such as cognitive effort, naturalness or distortion — to be measured directly, without recourse to rating scales. Incorporation of a test phase in which listener preferences are frozen also enables intelligibility to be estimated within the same trial. Offline computation and smooth online interpolation within the tool permits the impact of changes in practically any target speech feature (e.g. fundamental frequency or spectral slope) or background characteristic (e.g. noise spectrum), regardless of complexity, to be measured. The paper describes the tool’s capabilities, presents a range of visualisations, and notes some potential applications and limitations.