Most speech quality assessment methods require a perfect reference signal to evaluate the damaged speech's quality. However, it is challenging to obtain clean reference signals due to various types and levels of noise in reality. Meanwhile, no-reference speech quality assessment is less accurate than full-reference method. To address these issues, we propose a novel no-reference speech quality assessment model that improves evaluation accuracy with lower complexity. The model is primarily composed of three densely connected convolutional (DCC) modules and a bidirectional long short-term memory (BLSTM) module. Experiment results demonstrate that our method outperforms the baselines, achieving state-of-the-art on the no-reference speech quality assessment task. When using PESQ as optimization targets, the MSE, PLCC and SRCC reach 0.0389, 0.9695 and 0.9715, whereas when using STOI, these metrics reach 0.0019, 0.9608, and 0.9630, respectively.