In this work, we propose a promising neural method for automated essay scoring (AES), wherein BERT is taken as the backbone model optimized with an effective metric-based learning approach. We further seek to investigate the complementary role of handcrafted readability features in neural modeling of AES by infusing them into our neural model, while there still has been little work on comprehensively analyzing the effect and utility of such an endeavor. Our findings reveal the essential role of each model component in achieving better performance, especially for the inclusion of readability-aware features as auxiliary information sources. Extensive experiments on a benchmark dataset underscore the promising potential of leveraging metric-based learning and readability-aware features in the development of AES methods. This also lays the foundation for future work managing to optimize these model components for better performance.