To address the problems of missed detection of faces, the insufficient computing power of mobile platforms, and the limited hardware resources of face recognition applications under epidemic prevention and control, this study proposes an improved lightweight detection model for faces with masks based on YOLOv5. In this model, the C3 module in the original network is replaced with a lightweight C3Ghost module to compress the computations of the convolution process and the size of the model. Moreover, an attention mechanism is added to the backbone network to improve the feature extraction capability of the network, and the border regression loss function is improved to improve the speed and accuracy of detection. The experimental results indicate that the amount of calculation and parameters of the improved model are decreased by 29.79% and 33.33%, respectively, with the weight file size of only 2.8 M. The improved model reduces the dependence on the hardware environment, and its detection rate reaches 96.6%. Compared with the existing models, it has outstanding advantages and can be effectively applied to face recognition.