基于指针生成网络和扩展Transformer的多属性可控文本摘要模型
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:


Multi-attribute Controllable Text Summary Model Based on Pointer Generator Network and Extended Transformer
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 增强出版
  • |
  • 文章评论
    摘要:

    可控文本摘要模型可以生成符合用户偏好的摘要. 之前的摘要模型侧重于单独控制某个属性, 而不是多个属性的组合. 传统的Seq2Seq多属性可控文本摘要模型在满足多个控制属性时, 存在无法整合所有控制属性、无法准确再现文本中关键信息和无法处理单词表外单词等问题. 为此, 本文提出了一种基于扩展Transformer和指针生成网络(pointer generator network, PGN)的模型. 模型中的扩展Transformer将Transformer单编码器-单解码器的模型形式扩展成具有双重文本语义信息提取的双编码器和单个可融合指导信号特征的解码器形式. 然后利用指针生成网络模型选择从源文本中复制单词或利用词汇表生成新的摘要信息, 以解决摘要任务中常出现的OOV (out of vocabulary)问题. 此外, 为高效完成位置信息编码, 模型在注意力层中使用相对位置表示来引入文本的序列信息. 模型可以用于控制摘要的许多重要属性, 包括长度、主题和具体性等. 通过在公开数据集MACSum上的实验表明, 相较以往方法, 本文提出的模型在确保摘要质量的同时, 更加符合用户给定的属性要求.

    Abstract:

    The controllable text summary models can generate summaries that conform to user preferences. Previous summary models focus on controlling a certain attribute alone, rather than the combination of multiple attributes. When multiple control attributes are satisfied, the traditional Seq2Seq multi-attribute controllable text summary model cannot integrate all control attributes, accurately reproduce key information in the texts, and handle words outside the word lists. Therefore, this study proposes a model based on the extended Transformer and pointer generator network (PGN). The extended Transformer in the model extends the Transformer single encoder-single decoder model form into a dual encoder with dual text semantic information extraction and a single decoder form that can fuse guidance signal features. Then the PGN model is employed to select the source from the source copy words in the text or adopt vocabulary to generate new summary information to solve the OOV (out of vocabulary) problem that often occurs in summary tasks. Additionally, to efficiently complete position information encoding, the model utilizes relative position representation in the attention layer to introduce sequence information of the texts. The model can be leveraged to control many important summary attributes, including lengths, topics, and specificity. Experiments on the public dataset MACSum show that compared with previous methods, the proposed model performs better at ensuring the summary quality. At the same time, it is more in line with the attribute requirements given by users.

    参考文献
    相似文献
    引证文献
引用本文

冼广铭,李凡龙,郑兆明.基于指针生成网络和扩展Transformer的多属性可控文本摘要模型.计算机系统应用,2024,33(4):246-253

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-09-28
  • 最后修改日期:2023-11-09
  • 录用日期:
  • 在线发布日期: 2024-01-30
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号