基于H-GEM模型的多模态情感分析
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

国家自然科学基金(622711491)


Multimodal Sentiment Analysis with H-GEM Model
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    传统多模态情感分析方法在特征拼接和融合中易产生信息冗余, 难以捕捉细粒度复杂情感特征, 在模态缺失和跨域迁移场景下鲁棒性不足. 同时, 现有混合专家(MoE)方法大多为单层结构, 专家分工不明确, 存在功能重叠和泛化性欠佳的问题. 本文提出一种分层自适应混合专家模型H-GEM (hierarchical gated expert mixture). 通过构建3层分级专家体系: 模态专家层提炼模态特征; 融合与抽象专家层自适应选择融合策略; 情感极性专家层进行细粒度建模. 同时引入信息论与判别性约束提升专家选择的语义区分性和稀疏性. 通过分层门控实现逐级决策, 保证专家差异化分工与跨任务建模. 在CMU-MOSI和CMU-MOSEI数据集上实验表明, H-GEM在一系列指标上均优于基线模型. 与单层MoE架构相比, 显著降低的路由熵表明其有效缓解专家冗余问题. 该模型在低资源和模态缺失复杂任务中表现出更高的鲁棒性, 展现出良好的应用潜力.

    Abstract:

    Traditional multimodal sentiment analysis methods often suffer from information redundancy during feature concatenation and fusion, making it difficult to capture fine-grained and complex emotional features, while also exhibiting limited robustness in modality-missing and cross-domain transfer scenarios. Meanwhile, most existing mixture of experts (MoE) methods adopt a single-layered structure with ambiguous expert specialization, leading to functional overlap and suboptimal generalization. To address these issues, this study proposes a hierarchical gated expert mixture (H-GEM) model. A three-layer hierarchical expert architecture is constructed: a modality expert layer extracts modal features, a fusion and abstraction expert layer adaptively selects fusion strategies, and a sentiment polarity expert layer performs fine-grained modeling. In addition, information-theoretic and discriminative constraints are incorporated to enhance the semantic discriminability and sparsity of expert selection. By leveraging hierarchical gating for progressive decision-making, H-GEM ensures differentiated expert specialization and cross-task modeling. Experiments on CMU-MOSI and CMU-MOSEI datasets demonstrate that H-GEM outperforms baseline models across a series of metrics. Compared with single-layer MoE architectures, the significantly reduced routing entropy indicates effective mitigation of expert redundancy. Moreover, the proposed model demonstrates higher robustness in low-resource and modality-missing scenarios, highlighting its strong practical applicability.

    参考文献
    相似文献
    引证文献
引用本文

杨新航,王晶晶,陈思宇,田宏.基于H-GEM模型的多模态情感分析.计算机系统应用,,():1-10

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2025-08-11
  • 最后修改日期:2025-09-10
  • 录用日期:
  • 在线发布日期: 2026-01-19
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62661041 传真: Email:csa@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号