Heart rate is an important basis for reflecting the status of the cardiovascular system in humans. Video-based non-contact heart rate detection has been widely applied due to its advantages of strong scene adaptability and low cost. However, this method is susceptible to noise interference such as illumination change and target movement. To solve this problem, this study proposes a method based on the decomposition of vibration phase signals to extract the video heart rate on the wrist skin. Its core idea is to find the band range of pulse signals by designing a direction-selective complex steerable pyramid. The signal-to-noise ratio is used to select the pulse signals of interest and the robust principal component analysis is employed to isolate pulse signals from the mixed signals. Finally, the heart rate of noise-resistant pulse signals is detected. In this study, the data set of heart rate detection is collected, and the output from a pulse detector is taken as the true values to verify the method. The accuracy is 97.80% in the interference scene, which is over 5% higher than that of the three mainstream methods.