###
计算机系统应用英文版:2026,35(3):59-68
本文二维码信息
码上扫一扫!
基于H-GEM模型的多模态情感分析
(大连交通大学 轨道智能工程学院, 大连 116052)
Multimodal Sentiment Analysis with H-GEM Model
(School of Intelligent Railway Engineering, Dalian Jiaotong University, Dalian 116052, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 752次   下载 385
Received:August 11, 2025    Revised:September 10, 2025
中文摘要: 传统多模态情感分析方法在特征拼接和融合中易产生信息冗余, 难以捕捉细粒度复杂情感特征, 在模态缺失和跨域迁移场景下鲁棒性不足. 同时, 现有混合专家(MoE)方法大多为单层结构, 专家分工不明确, 存在功能重叠和泛化性欠佳的问题. 本文提出一种分层自适应混合专家模型H-GEM (hierarchical gated expert mixture). 通过构建3层分级专家体系: 模态专家层提炼模态特征; 融合与抽象专家层自适应选择融合策略; 情感极性专家层进行细粒度建模. 同时引入信息论与判别性约束提升专家选择的语义区分性和稀疏性. 通过分层门控实现逐级决策, 保证专家差异化分工与跨任务建模. 在CMU-MOSI和CMU-MOSEI数据集上的实验结果表明, H-GEM在一系列指标上均优于基线模型. 与单层MoE架构相比, 显著降低的路由熵表明其能够有效缓解专家冗余问题. 该模型在低资源和模态缺失复杂任务中表现出更高的鲁棒性, 展现出良好的应用潜力.
Abstract:Traditional multimodal sentiment analysis methods often suffer from information redundancy during feature concatenation and fusion, making it difficult to capture fine-grained and complex emotional features, while also exhibiting limited robustness in modality-missing and cross-domain transfer scenarios. Meanwhile, most existing mixture of experts (MoE) methods adopt a single-layered structure with ambiguous expert specialization, leading to functional overlap and suboptimal generalization. To address these issues, this study proposes a hierarchical gated expert mixture (H-GEM) model. A three-layer hierarchical expert architecture is constructed: a modality expert layer extracts modal features, a fusion and abstraction expert layer adaptively selects fusion strategies, and a sentiment polarity expert layer performs fine-grained modeling. In addition, information-theoretic and discriminative constraints are incorporated to enhance the semantic discriminability and sparsity of expert selection. By leveraging hierarchical gating for progressive decision-making, H-GEM ensures differentiated expert specialization and cross-task modeling. Experimental results on CMU-MOSI and CMU-MOSEI datasets demonstrate that H-GEM outperforms baseline models across a series of metrics. Compared with single-layer MoE architectures, the significantly reduced routing entropy indicates effective mitigation of expert redundancy. Moreover, the proposed model demonstrates higher robustness in low-resource and modality-missing scenarios, highlighting its strong practical applicability.
文章编号:     中图分类号:    文献标志码:
基金项目:国家自然科学基金(622711491)
引用文本:
杨新航,王晶晶,陈思宇,田宏.基于H-GEM模型的多模态情感分析.计算机系统应用,2026,35(3):59-68
YANG Xin-Hang,WANG Jing-Jing,CHEN Si-Yu,TIAN Hong.Multimodal Sentiment Analysis with H-GEM Model.COMPUTER SYSTEMS APPLICATIONS,2026,35(3):59-68