For lack of stability in dynamic hand gesture tracking and low recognition efficiency, a hand gesture tracking and recognition framework is proposed based on TLD and DTW algorithm. First, the hand gesture area is got with a Haar features based classifier. Then the gesture trajectory is obtained by using the TLD tracking algorithm initialized by the hand area. Finally, features are extracted from the trajectory and the improved DTW algorithm is used to recognize the dynamic gesture. Experiments show that the framework can track the hand gesture stably for a long time and be able to improve the recognition speed greatly under the premise of ensuring high recognition accuracy.