The transmission line channels feature complex environments, and various hidden external damage targets exhibit significant scale differences due to factors such as shooting angles and observation distances, thereby resulting in the model’s low precision and prominent problems of false and missed detection in diversified risk target recognition. To this end, this study proposes a transmission line external damage detection method based on hierarchical feature fusion. The method is based on the RT-DETR model and introduces a lightweight C2f_MambaOut module to optimize the backbone structure and effectively reduce model parameters. Additionally, a PA_CGLU module integrating polarity-aware attention and gating mechanisms is established to replace the original AIFI module, thereby enhancing the query vectors’ directional perception of image features and salient modeling capabilities, as well as improving adaptive semantic matching efficiency. Furthermore, a hierarchical attention fusion block (HAFB) is designed to realize multi-scale hierarchical fusion and enhancement of input features by employing local and global attention branches, thus boosting the comprehensive recognition ability of multi-category and multi-scale targets. Additionally, a transmission line external damage detection dataset that covers various real-world scenarios and features a balanced distribution of samples is constructed. Experimental results on this dataset demonstrate that the improved model achieves a 1.5% increase in mean average precision (mAP) and a 20.7% reduction in the parameter count. The results demonstrate that the effectiveness of the proposed method in mitigating the challenges posed by target scale variation and enhancing the overall detection performance for diverse external damage risks, thereby achieving a better balance between model efficiency and accuracy.