To address the problems of low contrast between target and background and lack of target feature information facing satellite video images, this study proposes a target segmentation and tracking method combining target motion information, spatio-temporal background, and appearance model. After the target area is obtained by positioning in the first frame, the histogram of oriented gradient method is employed to extract the features of the target, and the kernel correlation filter (KCF) is utilized to obtain the target tracking area 1. Subsequently, color and spatial features are used to build a spatial model of the context information about the target and its surrounding area and thereby obtain the target tracking area 2. Then, the visual background extraction algorithm is applied to detect the moving target in the target area in pixels and further obtain the segmentation area 3 of the single target. Finally, the correlation of the three areas is calculated, respectively, to obtain the optimal area as the final target tracking position and the template update sample. The experimental results show that compared with the KCF algorithm, the proposed algorithm obtains a significantly higher tracking success rate and accuracy and also achieves single target segmentation.