Collaborative Filtering(CF) is one of the most successful approaches for building recommender system,it uses the known preferences of a group of users to make predictions of unknown preferences of other users. The matrix factorization models which can profile both users and items latent factors directly,and the neighborhood models which can analyze similarities between users and items are current research focuses.A method of merging both matrix factorization models and neighborhood models is proposed, which can make further accuracy improvements. The experiment results show that this method is correct and feasible.