Towards an assessment method for multi-party audio conferencing systems, we investigated in a pilot study the influence of the number of interlocutors and the audio reproduction method on the quality of experience. Despite some room for improving the sensitivity of our experimental method, the results show that the number of interlocutors and the audio reproduction method influence at least partially the cognitive effort required in conferencing situations and the perceived speech transmission quality.