Cracks are a key indicator for assessing the health of concrete structures. To address the missed detection of fine cracks and structural discontinuities caused by class imbalance and the elongated structural characteristics of crack images, this study proposes an improved SU-Mamba segmentation network based on the SegMAN architecture. The network adopts the encoder of SegMAN, which integrates neighborhood attention and a two-dimensional selective scan mechanism, as its feature extraction backbone. On top of this backbone, a symmetric U-shaped decoder is constructed to progressively recover high-resolution spatial details through multi-layer upsampling and skip connections. During the feature fusion stage, a channel attention mechanism is incorporated to enhance the representation of crack-related features. Furthermore, a gradient standard deviation (GSD)-based penalty term is introduced to promote directionally biased gradient distributions in low-confidence crack regions, thus guiding the network to learn the geometric characteristics of cracks and enhancing its ability to segment fine cracks while reducing missed detections. Experimental results on a concrete crack dataset demonstrate that the proposed method effectively identifies fine cracks in complex backgrounds, achieving improvements of 7.04% in IoU and 5.83% in Dice coefficient over the baseline model.