As a typical example of China’s invasion of alien species, Eupatorium adenophorum causes serious damage to the ecological environment and affects the development of agro-forestry economy. Eupatorium adenophorum detection as the initial stage and monitoring stage of prevention and control, its detection accuracy will affect the control results. Aiming at the target detection problem of the complex background leaf image of Eupatorium adenophorum, this study proposes a migration learning method based on YOLOv3 to detect Eupatorium adenophorum. The deep learning model YOLOv3 was migrated to the E. adenophorum data set, and the K-means algorithm was used to perform dimensional clustering to determine the target frame parameters. The weight of various losses is changed in the loss function during training, and the adaptability of the model is increased to the data set. The experimental results show that Average Precision (AP) is 17% higher than that of the original YOLOv3 in the detection task of Eupatorium adenophorum, which can meet the detection task of Eupatorium adenophorum under complex background.