In complex, cluttered environments, introducing pushing actions to assist robotic grasping can improve the success rate. However, existing push-grasp collaboration methods face issues such as insufficient feature extraction capability and inefficient pushing strategies. To address these problems, this study proposes an improved push-grasp collaborative algorithm based on deep Q-network (DQN). The proposed method integrates an efficient multi-scale attention (EMA) mechanism into the perception-action policy network. The EMA module enhances the extraction of key task features, including object edges and surfaces, through channel grouping and cross-spatial modeling. In addition, a push effectiveness evaluation mechanism is designed based on changes in image frequency energy and the displacement of the energy centroid, which contributes to constructing a more discriminative reward function. This function guides the agent to learn an effective push-grasp collaborative strategy. Experiments conducted on the CoppeliaSim simulation platform show that the proposed method significantly outperforms baseline methods such as METOVPG in terms of grasp success rate and action efficiency. Specifically, it achieves a 21.2% relative improvement in the grasp success rate in simulation tests, validating the effectiveness and collaborative advantages of the proposed attention mechanism and reward design in complex scenarios.