###
计算机系统应用英文版:2026,35(4):41-51
本文二维码信息
码上扫一扫!
基于多模态增强融合与多分支蒸馏的内窥镜异常检测模型
(1.福建理工大学 电子电气与物理学院, 福州 350118;2.福建理工大学 数智系统与装置研究院, 福州 350118;3.复旦大学 生物医学工程与技术创新学院, 上海 200433;4.闽江学院 计算机与大数据学院, 福州350121;5.闽江学院 福建省信息处理与智能控制重点实验室, 福州350121)
Multimodal Enhanced Fusion and Multi-branch Distillation Based Model for Endoscopic Anomaly Detection
(1.School of Electronic, Electrical Engineering and Physics, Fujian University of Technology, Fuzhou 350118, China;2.Institute for Digital Intelligence Systems and Devices, Fujian University of Technology, Fuzhou 350118, China;3.College of Biomedical Engineering, Fudan University, Shanghai 200433, China;4.School of Computer and Data Science, Minjiang University, Fuzhou 350121, China;5.Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University, Fuzhou 350121, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 226次   下载 178
Received:September 18, 2025    Revised:October 09, 2025
中文摘要: 内窥镜影像为胃癌的筛查与诊断提供重要依据. 然而, 传统内窥镜检查准确率有限. 为此, 多模态融合异常检测方法被引入内窥镜影像分析, 但仍面临模态偏差与配对数据稀缺等问题. 针对这些问题, 本文提出一种基于多模态增强融合与多分支蒸馏的内窥镜异常检测模型. 首先, 设计交叉掩码注意力跨模态融合模块, 通过局部特征重建与交叉注意力机制挖掘模态间的潜在关系. 其次, 提出一种多分支跨模态蒸馏架构, 由多模态教师网络和两个独立学生分支组成. 该架构仅教师网络需配对数据训练, 学生分支则完全无需配对数据. 这一设计降低模型对配对数据的依赖并有效缓解模态偏差. 最后, 引入全局余弦相似度损失以增强多模态特征的一致性表示. 在真实公开数据集上进行的大量实验表明, 本文方法在多模态内窥镜异常检测任务中取得领先的性能. 本文的源码已公开在: https://github.com/LuoYifei-xs/CEMD.
Abstract:Endoscopic images provide a critical foundation for the screening and diagnosis of gastric cancer. However, the accuracy of traditional endoscopic examinations remains limited. To address this issue, multimodal fusion-based anomaly detection techniques have been applied to endoscopic image analysis. However, they still suffer from modality bias and the scarcity of paired data. To overcome these limitations, this study proposes an endoscopic anomaly detection model integrating multimodal enhanced fusion and multi-branch knowledge distillation. The framework incorporates a cross-masked attention cross-modal fusion module that explores latent inter-modal relationships through local feature reconstruction and cross-attention mechanisms. Furthermore, a multi-branch cross-modal distillation architecture is established, comprising a multimodal teacher network and two independent student branches. This design requires only the teacher network to be trained on paired data while enabling the student branches to learn without any paired data, thus significantly reducing dependency on paired samples and effectively mitigating modality bias. Additionally, a global cosine similarity loss is introduced to enhance consistency in multimodal feature representation. Extensive experiments on public real-world datasets demonstrate that the proposed method achieves competitive performance in multimodal endoscopic anomaly detection tasks. Code will be released at: https://github.com/LuoYifei-xs/CEMD.
文章编号:     中图分类号:    文献标志码:
基金项目:国家自然科学基金 (62471207, 61972187); 福建省自然科学基金 (2024J02029, 2023R1050, 2023J011390, 2020J02024); 福建省卫生健康委员会科技重大专项 (2021ZD01004); 福建省医疗大数据工程重点实验室开放项目 (KLKF202301); 福建中医药大学高层次人才研究创业基金 (NX2020005-Talent)
引用文本:
罗逸飞,林清华,陈健,郑文斌,李佐勇.基于多模态增强融合与多分支蒸馏的内窥镜异常检测模型.计算机系统应用,2026,35(4):41-51
LUO Yi-Fei,LIN Qing-Hua,CHEN Jian,ZHENG Wen-Bin,LI Zuo-Yong.Multimodal Enhanced Fusion and Multi-branch Distillation Based Model for Endoscopic Anomaly Detection.COMPUTER SYSTEMS APPLICATIONS,2026,35(4):41-51