Anomalous Sound Detection requires constructing a distribution using only normal sounds. However, collecting sufficient normal samples across diverse conditions is challenging, leading to sample imbalance within subclasses. Existing subcenter angular margin loss methods use multiple subcenters to capture intra-class diversity but still suffer from under-representation or overfitting. To address this issue, we propose Adaptive Across-Subcenter Representation Learning (AASRL). Unlike existing methods that use either a single or all subcenters, AASRL adaptively selects subcenters based on the representation quality of samples and optimizes their representation across the most relevant subcenters. This ensures efficient representation of each sample and prevents the majority subclass from dominating the representation space. Experiments on the DCASE2023 Challenge Task2 dataset and a constructed imbalanced dataset demonstrate the effectiveness of AASRL.