To address the challenges of underwater target detection, including complex backgrounds, low illumination, and frequent occlusion and overlap of targets, this study proposes an improved real-time underwater target detection algorithm based on YOLOv8n. A feature fusion module, SEHAP, is first designed to fuse layers P2 and P3. This fused output is then processed by EHAPOKM to learn features ranging from global to local, thus improving the detection accuracy for small targets and low-resolution images. Moreover, a lightweight detection head, SLDH, is introduced. By employing shared convolution and integrating it with the ASL feature scaling module, the proposed method reduces both parameter count and computational load with minimal accuracy loss. The C2f-EGMSC module is further incorporated, utilizing group convolution to enhance multi-level feature extraction. To improve the model’s capacity to capture spatial and channel-wise information, the bottleneck attention module (BAM) is applied after selected C2f-EGMSC modules. The improved model, named ESE-YOLOv8, is evaluated on the RUOD dataset. Experimental results demonstrate a detection accuracy of 85.2%, representing a 1.2 percentage point improvement over the baseline, with a 36.7% reduction in parameters. The improved model takes into account both lightweight and precision, providing a feasible solution for deployment in underwater environments.