To address the issues of low recognition accuracy for small traffic sign targets and frequent false negatives and false positives in complex road scenarios with existing algorithms, this study proposes a traffic sign detection method based on improved YOLO11n, termed as TSP-YOLO. First, the traffic sign adaptive detection module (TSDAM) is designed, which enhances the feature expression of small targets through multi-scale feature enhancement. Second, the SPPF module in the backbone network is improved to construct the IMSPPF module, which integrates max pooling, average pooling, and the EMA attention mechanism, effectively suppressing background noise interference and enhancing the model’s ability to model multi-scale contextual information. Finally, partial convolution is introduced into the C3k2 structure to design the C3k2_PConv module, achieving a reduction in the number of parameters while maintaining model accuracy. Experimental results on the TT100K dataset show that this method increases the mAP50 metric from 68.7% to 74.5%, and the mAP50:95 metric from 53.2% to 58.1%, with a 19.3% reduction in parameters, significantly improving the detection performance of small traffic signs. Additionally, experimental results on the CCTSDB 2021 dataset further confirm the excellent generalization ability of the proposed algorithm.