Vehicle detection in aerial imagery is of paramount importance for various intelligent transportation applications, including vehicle tracking, traffic control, and traffic behavior analysis. However, this task presents significant challenges due to factors such as scale variance, the presence of small vehicles, and complex scenes conditions. The demand for high-quality and precise vehicle detection exacerbates these challenges, particularly in complex environments characterized by low light and occlusion. To address these issues, we propose a balanced feature fusion and attention mechanism-based vehicle detection network, termed BaAM-YOLO, specifically designed for aerial images. Firstly, to improve the vehicle detection accuracy, we didn't directly fuse deep and shallow features. Instead, we focused on achieving a balanced contribution from different feature layers. Secondly, to enhance the feature extraction capabilities of our model, we incorporated a simple parameter-free attention mechanism (SimAM), which aims to improve detection accuracy. Additionally, to effectively model the target confidence loss and classification loss, we emplogyed focal loss to derive a loss function that facilitates dynamic adjustment for multi-class classification. The proposed vehicle detection method was evaluated using the VisDrone2019 dataset. Comparative analyses with state-of-the-art research indicate that the proposed method can obtain superior and competitive results in vehicle detection.