This exploratory study addresses the relation between pitch encoding and the processing of phonetic details with an individual approach, from the bottom up to higher-level phonological processing. It reports on individual-based analyses from two experiments as a proof of concept for a larger project: (1) a preliminary report on the relation between JND (just-noticeable-differences) in pitch and duration, and the use of spectral and temporal cues in New Zealand English vowel perception, and (2) a re-analysis of the results from a perception study consisting of the use of VOT and pitch in the perception of [±voice] in French. By exploring individual differences in these two experiments, we advocate that it is important to consider individual variations on multiple levels of auditory and speech processing in an integrated manner. We further discuss a conceptual framework to dig into causal relations between auditory processing of spectral and temporal cues, language processing, and general cognitive mechanisms, based on large-scale participative research from typical and atypical listeners.