Feature point extraction and dimension measurement for human bodies has always been the key content of virtual garment fitting. Based on the human body image, this study realizes the extraction of human feature points and the size measurement by improving the ASM algorithm. Firstly, it calculates the distance between the two central points of face and body in the image, and matches them to the corresponding template, while changes the local template matching pattern in the traditional ASM algorithm. So the accuracy and efficiency of the initial model matching are improved. Then, it sets the feature point as the center and selects the less effective neighborhood points around the feature point for the object searching in the gray scale training model, which can solve the problem that the traditional ASM method takes long time and the feature points are easy mismatching. To solve that the unilateral fitting effect is better for the lower part of the human crotch, it uses the Mahalanobis distance formula, compares the gray scale of the specific matrix size neighborhood with the gray scale model, and combines with the human body shape distribution and symmetry feature to implement the feature point fitting process. Experimental results show that this method can adapt to the feature point extraction and size measurement of human body image in a complex background, and improve the extraction of human feature points and the accuracy of dimension measurement.