Recently plenty of attention-based encoders have been proposed for end-to-end (E2E) automatic speech recognition (ASR). Despite the impressive performance, these encoders usually have a large model size and suffer from expensive memory and computation costs. To obtain more compact encoders for E2E ASR, we propose searching compact attention-based encoders using neural architecture search (NAS) in this paper, named NAS-SCAE. NAS-SCAE consists of one search space that contains a set of candidate encoders and one search algorithm responsible for searching the optimal encoder from the search space. On one hand, NAS-SCAE designs a topology-fused search space to integrate different architecture topologies of existing encoders (e.g. Transformer, Conformer) and explore more brand-new architectures. On the other hand, combined with the training pipeline of E2E ASR, NAS-SCAE develops a resource-aware differentiable search algorithm to search compact encoders efficiently and proposes an adjustable search scheme to alleviate the joint optimization problem of the differentiable search algorithm. On four Mandarin and English datasets, NAS-SCAE can effectively reduce the encoder resource consumption with negligible performance drop and achieve at least 2.13x/2.09x parameters/FLOPs reduction than the human-designed baselines.