Working diary is an important way of monitoring the progress of software outsourcing project. They are committed by project staff as a report of daily work. The quality of working diary can reflect procedure execution of project to some extent, but because the content of working diary is large and trivial, it is hard to check and evaluate rely only on manual work. Existing researches pay little attention to the quality of logs, so we proposed an automated method to evaluate the quality of working diary. Firstly, this method uses the lexical analysis, interdependence syntactic analysis and LDA model to mine and analysis the historical data. Secondly, we extract quality features from aspects such as structure, content, subject relevance, then collect training samples by artificial tagging. Lastly, we establish evaluation model using classification algorithm, through which we get the final evaluation of the work diary. This paper made a case study based on a national project and achieved a highly accurate evaluation model. The result shows that the method can effectively evaluate the quality of working diary, which helps to make decision for outsourcing department.