基于对比学习和焦点损失的持续关系抽取
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

陕西省自然科学基础研究计划(2023-JC-YB-568); 陕西省教育厅科研计划(22JP028); 陕西省计算机学会&翔腾公司基金(XT-QC-202309-119287)


Continual Relation Extraction Based on Contrastive Learning and Focal Loss
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 增强出版
  • |
  • 文章评论
    摘要:

    持续关系抽取旨在训练模型从不断变化的数据流中学习新关系, 同时保持对旧关系的准确分类. 然而, 由于神经网络的灾难性遗忘问题, 模型在学习完新关系之后, 对旧关系的识别能力往往会大幅度降低. 为了缓解灾难性遗忘对模型性能的影响, 本文提出了一种基于对比学习和焦点损失的持续关系抽取方法. 首先, 在训练集与其增强样本集的并集上训练模型, 以学习新任务; 其次, 从训练集中, 为每个新关系选取并存储记忆样本; 然后, 将激活集中的示例与所有已知关系原型进行对比, 以学习新旧关系; 最后, 利用关系原型进行记忆再巩固, 并引入焦点损失提高模型对相似关系的区分. 在TACRED数据集上进行实验, 结果表明本文方法能够进一步缓解灾难性遗忘问题, 提升模型的分类能力.

    Abstract:

    Continual relation extraction aims to train models to learn new relations from evolving data streams while maintaining accurate classification of previously learned relations. However, due to the catastrophic forgetting problem of neural networks, the model’s ability to recognize old relations tends to decrease drastically after being trained on new relations. To mitigate the impact of catastrophic forgetting on model performance, this study proposes a continual relation extraction method based on contrastive learning and focal loss. First, the model is trained on a concatenated set of the original training set and its augmented samples to learn a new task. Second, from the training set, memory samples are selected and stored for each new relation. Then, instances from the activation set are contrasted with all known relation prototypes to learn the old and the new relations. Finally, memory reconsolidation is performed using the relation prototypes and focal loss is introduced to improve the model’s distinction between similar relations. Experiments are conducted on the TACRED dataset, and the results show that the method proposed can further alleviate catastrophic forgetting and improve the model’s classification ability.

    参考文献
    相似文献
    引证文献
引用本文

王苏越,马丽丽,陈金广.基于对比学习和焦点损失的持续关系抽取.计算机系统应用,2024,33(7):180-187

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-01-03
  • 最后修改日期:2024-02-04
  • 录用日期:
  • 在线发布日期: 2024-05-31
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号