To tackle the problem that traditional container vector detection is limited to manual detection, this study designs a visual search system for container vectors based on machine vision. The system collects real-time video and captures the activity of vectors through a smart car under remote control. Then, it recognizes the vectors in the video returned by the car through deep learning and inter-frame detection. The system takes the YOLOv5 model as the training core and adopts a modular structure to realize the visual detection of container vectors. Machine vision helps improve detection efficiency and lays the foundation for the further use of robots to detect vectors.