本文已被:浏览 475次 下载 1493次
Received:September 06, 2024 Revised:October 10, 2024
Received:September 06, 2024 Revised:October 10, 2024
中文摘要: TensorGCN模型是图神经网络应用在文本分类领域的SOTA模型之一. 然而在处理文本语义信息方面, 该模型使用的LSTM难以完全地提取短文本语义特征, 且对复杂的语义处理效果不佳; 同时, 由于长文本中包含的语义及句法特征较多, 在进行图间异构信息共享时特征共享不完全, 影响文本分类的准确性. 针对这两个问题, 对TensorGCN模型进行改进, 提出融合BERT和自注意力机制的张量图卷积网络 (BTSGCN)文本分类方法. 首先, 使用BERT代替TensorGCN架构中的LSTM模块进行语义特征提取, 通过考虑给定单词两侧的周围单词来捕获单词之间的依赖关系, 更准确地提取短文本语义特征; 然后, 在图间传播时加入自注意力机制, 帮助模型更好地捕捉不同图之间的特征, 完成特征融合. 在MR、R8、R52和20NG这4个数据集上的实验结果表明BTSGCN相比于其他对比方法的分类准确度更高.
Abstract:TensorGCN model is one of the state-of-the-art (SOTA) models applied by graph neural networks in the field of text classification. However, in terms of processing text semantic information, the long short-term memory (LSTM) used by the model has difficulty in completely extracting the semantic features of short text and performs poorly in handling complex semantic information. At the same time, due to the large number of semantic and syntactic features contained in long texts, feature sharing is incomplete when heterogeneous information is shared among graphs, which affects the accuracy of text classification. To solve these two problems, the TensorGCN model is improved, and a text classification method based on the tensor graph convolutional network fusing BERT and the self-attention mechanism (BTSGCN) is proposed. Firstly, BERT is used to replace the LSTM module in the TensorGCN architecture for semantic feature extraction. It captures the dependencies between words by considering the surrounding words on both sides of a given word, thus extracting the semantic features of short texts more accurately. Then, the self-attention mechanism is added during the propagation among graphs to help the model better capture the features among different graphs and complete the feature fusion. Experimental results on MR, R8, R52, and 20NG datasets show that BTSGCN has higher classification accuracy than other comparison methods.
keywords: text classification graph neural network (GNN) tensor graph convolutional network fusing BERT and the self-attention mechanism (BTSGCN) BERT self-attention mechanism
文章编号: 中图分类号: 文献标志码:
基金项目:陕西省重点研发计划 (2024GX-YBXM-548)
引用文本:
史文艺,朱欣娟.融合BERT和自注意力机制的张量图卷积网络文本分类.计算机系统应用,2025,34(3):152-160
SHI Wen-Yi,ZHU Xin-Juan.Text Classification with Tensor Graph Convolutional Network Fusing BERT and Self-attention Mechanism.COMPUTER SYSTEMS APPLICATIONS,2025,34(3):152-160
史文艺,朱欣娟.融合BERT和自注意力机制的张量图卷积网络文本分类.计算机系统应用,2025,34(3):152-160
SHI Wen-Yi,ZHU Xin-Juan.Text Classification with Tensor Graph Convolutional Network Fusing BERT and Self-attention Mechanism.COMPUTER SYSTEMS APPLICATIONS,2025,34(3):152-160

