Cancer diagnosis and subtype classification based on whole slide image (WSI) has emerged as a critical research focus in the field of pathology. However, WSIs are characterized by extremely large sizes and complex structures, making it challenging for existing methods to capture the complex relationships among morphologically diverse image patches, particularly when these patches are spatially distant. To address this issue, a deep learning framework based on graph neural network (GNN), termed Bi-DDRGNN, is proposed for accurate cancer diagnosis and subtype classification. The architecture of the proposed Bi-DDRGNN comprises two branches: a directed dynamic graph attention network (DD-GAT) and a neighborhood fully connected residual graph convolutional network (ResGCN). By constructing graph structures from both global and local perspectives, the proposed framework effectively captures long-range dependencies and local fine-grained features among image patches, enabling improved modeling of the complex structures inherent in WSIs. Specifically, the DD-GAT branch builds directed edges and dynamically adjusts attention mechanisms to propagate information across distant regions, facilitating the modeling of long-range dependencies. The ResGCN branch, on the other hand, connects spatially adjacent patches to extract local structural features and enhances the representation of fine details through residual connections. In addition, a graph feature fusion (GF) module is introduced to integrate the features from both branches, thus improving classification performance and the overall representational capability of the model. Extensive experiments on three public datasets—TCGA-NSCLC, TCGA-BRAC, and CAMELYON16—demonstrate that the proposed Bi-DDRGNN outperforms existing state-of-the-art methods, validating the effectiveness of this study.