Target detection for UAV aerial images is significant in the applications of real-time monitoring and anomaly detection. However, the detection process is complicated due to challenges such as complex backgrounds, large variations in target scales, and a high proportion of small targets. To address these issues, this study proposes an improved algorithm for small target detection, AMD-YOLOv8. First, the algorithm replaces the large-target detection head with a micro-target detection head to effectively preserve small target features. A dynamic target detection head (Dyhead) is introduced, which applies fine grained attention adjustments across the horizontal, spatial, and channel dimensions, significantly enhancing the model’s ability to extract detailed features of the target. Second, a multi-scale global attention (MSGA) module is designed, utilizing various convolutional kernel operations and global context layouts to enhance the detection capability for distant small targets. Third, a bidirectional dense extended feature pyramid network (BDEFPN) is proposed to efficiently integrate multi-scale information through scale expansion and dense connections. Finally, the LAMP pruning strategy is applied for model lightweight by adaptively assessing and removing redundant connections between layers, effectively reducing the computational load and accelerating inference speed. Experimental results demonstrate that the improved AMD-YOLOv8 reduces parameters by 46.0% compared to YOLOv8n on the VisDrone2019 dataset while achieving an 8.6% increase in mAP50 and an FPS of 98.3 f/s. Additionally, test results of the UAVDT dataset further confirm the algorithm’s superior generalization ability, validating its effectiveness in the detection of UAV aerial images.