When people engage in conversations, their interactions can reveal potentially privacy-sensitive information that might not be apparent from isolated speech. However, current attribute-based privacy research tends to focus on a single person speaking, overlooking attributes that can be inferred from spoken interaction data. This paper presents an overview of inferable attributes in conversational speech, grounded in both existing literature and current applications. The paper also explores properties of spoken interaction that make it possible to infer these attributes, highlighting how interaction itself enables new forms of inference. By broadening the scope of speech privacy to include conversational dynamics, this work aims to support future research and systems that more fully account for privacy risks in real-world spoken interactions.