细粒度对齐与优势模态增强的多模态假新闻检测
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:


Multimodal Fake News Detection via Fine-grained Alignment and Dominant Modality Enhancement
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    现有的多模态假新闻检测方法仍存在以下不足: 在跨模态语义对齐过程中, 仅对全局特征进行对齐, 难以建立图像局部区域与对应文本片段之间的细粒度语义对齐; 在模态融合阶段通常简单采用等权融合策略, 未能充分发挥信息更丰富的优势模态的作用, 从而限制了模型性能. 鉴于此, 提出了一种细粒度对齐与优势模态增强的多模态假新闻检测模型. 所提模型中的细粒度对齐模块利用FG-CLIP模型的细粒度对齐能力, 引导新闻的图像与文本的深层语义特征建立精确对应, 有效抑制无关区域的干扰. 提出用置信度来判定优势模态, 该置信度根据单模态特征与其类别原型之间的距离计算得出. 同时, 引入原型交叉熵损失以增强优势模态的表征能力, 使其在融合过程中发挥主导作用. 在Weibo和GossipCop数据集上的实验结果表明, 该模型在多数评估指标上优于基线模型, 验证了其在虚假新闻检测任务中的有效性与鲁棒性.

    Abstract:

    Existing multimodal fake news detection methods still suffer from the following limitations. During cross-modal semantic alignment, only global features are typically aligned, failing to establish fine-grained semantic correspondences between local image regions and their relevant text fragments. In the modality fusion stage, an equal-weight combination strategy is usually adopted, which prevents the more informative dominant modality from being fully utilized, thereby limiting model performance. To address these issues, this study proposes a multimodal fake news detection model integrating fine-grained alignment and dominant modality enhancement. The fine-grained alignment module leverages the capability of the FG-CLIP model to guide precise correspondence between deep semantic features of news images and texts, effectively suppressing interference from irrelevant regions. Moreover, the dominant modality is determined by a confidence score, which is computed based on the distance between single-modality features and their corresponding class prototypes. A prototype cross-entropy loss is introduced to enhance the representational capacity of the dominant modality, enabling it to play a leading role in the fusion process. Experimental results on the Weibo and GossipCop datasets demonstrate that the proposed model outperforms baseline methods on most evaluation metrics, verifying its effectiveness and robustness in fake news detection tasks.

    参考文献
    相似文献
    引证文献
引用本文

白书铭,曹霑懋,刘聪,李俊斌.细粒度对齐与优势模态增强的多模态假新闻检测.计算机系统应用,,():1-11

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2025-09-10
  • 最后修改日期:2025-10-10
  • 录用日期:
  • 在线发布日期: 2026-01-19
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62661041 传真: Email:csa@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号