本文已被:浏览 752次 下载 385次
Received:August 11, 2025 Revised:September 10, 2025
Received:August 11, 2025 Revised:September 10, 2025
中文摘要: 传统多模态情感分析方法在特征拼接和融合中易产生信息冗余, 难以捕捉细粒度复杂情感特征, 在模态缺失和跨域迁移场景下鲁棒性不足. 同时, 现有混合专家(MoE)方法大多为单层结构, 专家分工不明确, 存在功能重叠和泛化性欠佳的问题. 本文提出一种分层自适应混合专家模型H-GEM (hierarchical gated expert mixture). 通过构建3层分级专家体系: 模态专家层提炼模态特征; 融合与抽象专家层自适应选择融合策略; 情感极性专家层进行细粒度建模. 同时引入信息论与判别性约束提升专家选择的语义区分性和稀疏性. 通过分层门控实现逐级决策, 保证专家差异化分工与跨任务建模. 在CMU-MOSI和CMU-MOSEI数据集上的实验结果表明, H-GEM在一系列指标上均优于基线模型. 与单层MoE架构相比, 显著降低的路由熵表明其能够有效缓解专家冗余问题. 该模型在低资源和模态缺失复杂任务中表现出更高的鲁棒性, 展现出良好的应用潜力.
Abstract:Traditional multimodal sentiment analysis methods often suffer from information redundancy during feature concatenation and fusion, making it difficult to capture fine-grained and complex emotional features, while also exhibiting limited robustness in modality-missing and cross-domain transfer scenarios. Meanwhile, most existing mixture of experts (MoE) methods adopt a single-layered structure with ambiguous expert specialization, leading to functional overlap and suboptimal generalization. To address these issues, this study proposes a hierarchical gated expert mixture (H-GEM) model. A three-layer hierarchical expert architecture is constructed: a modality expert layer extracts modal features, a fusion and abstraction expert layer adaptively selects fusion strategies, and a sentiment polarity expert layer performs fine-grained modeling. In addition, information-theoretic and discriminative constraints are incorporated to enhance the semantic discriminability and sparsity of expert selection. By leveraging hierarchical gating for progressive decision-making, H-GEM ensures differentiated expert specialization and cross-task modeling. Experimental results on CMU-MOSI and CMU-MOSEI datasets demonstrate that H-GEM outperforms baseline models across a series of metrics. Compared with single-layer MoE architectures, the significantly reduced routing entropy indicates effective mitigation of expert redundancy. Moreover, the proposed model demonstrates higher robustness in low-resource and modality-missing scenarios, highlighting its strong practical applicability.
keywords: multimodal sentiment analysis hierarchical gating mechanism mixture of experts (MoE) mutual information constraint robustness
文章编号: 中图分类号: 文献标志码:
基金项目:国家自然科学基金(622711491)
引用文本:
杨新航,王晶晶,陈思宇,田宏.基于H-GEM模型的多模态情感分析.计算机系统应用,2026,35(3):59-68
YANG Xin-Hang,WANG Jing-Jing,CHEN Si-Yu,TIAN Hong.Multimodal Sentiment Analysis with H-GEM Model.COMPUTER SYSTEMS APPLICATIONS,2026,35(3):59-68
杨新航,王晶晶,陈思宇,田宏.基于H-GEM模型的多模态情感分析.计算机系统应用,2026,35(3):59-68
YANG Xin-Hang,WANG Jing-Jing,CHEN Si-Yu,TIAN Hong.Multimodal Sentiment Analysis with H-GEM Model.COMPUTER SYSTEMS APPLICATIONS,2026,35(3):59-68

