面向分布式文件系统的大目录动态拆分机制
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

国家自然科学基金面上项目 (62472392); 国家自然科学基金青年科学基金 (C类) (62502189)


Large Directory Dynamic Splitting Mechanism for Distributed File Systems
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    在分布式文件系统中, 为提高元数据访问性能往往将同一目录下的文件元数据存储于一台服务器. 在大目录工作负载中, 单个目录需要管理海量文件, 这会使单台服务器的负载过高从而限制系统的扩展性. 同时, 高并发地创建和删除文件也会引起严重的锁争用. 为了解决这一挑战, 本文提出面向大目录场景的动态拆分机制. 随着目录规模增长, 持续地将大目录拆分为多个逻辑目录并迁移部分元数据到其他服务器, 从而缓解原服务器的高负载和高并发下的锁争用. 同时, 通过迁移开销优化和一致性维护机制减少迁移对系统的影响. 实验结果表明, 在大目录场景中, 动态拆分机制提高了元数据操作性能, 系统在并发度提高和目录规模增长时呈现更好的扩展性. 相较于不做动态拆分的场景, 文件创建和删除操作性能至多提升了4.09倍, 查询操作性能提升了2.05倍.

    Abstract:

    In distributed file systems, file metadata within the same directory is typically located on a single server to optimize access performance. However, in workloads involving large directories, a single directory may contain a massive number of files, which places excessive load on one server and limits system scalability. Moreover, highly concurrent operations such as file creation and deletion can lead to severe lock contention. To address these challenges, this study proposes a dynamic splitting mechanism for large directory scenarios. As a directory grows, the mechanism continuously splits it into multiple logical directories and migrates portions of metadata to other servers, thus reducing the load on the original server and alleviating lock contention under high concurrency. Meanwhile, the impact of migration on the system is reduced through migration overhead optimization and consistency maintenance mechanisms. Experimental results show that in large directory scenarios, the dynamic splitting mechanism improves the performance of metadata operations, and the system exhibits better scalability as concurrency increases and directory size grows. Compared to the baseline without splitting, the performance of file creation and deletion operations is improved by up to 4.09×, and the performance of lookup operations is improved by 2.05×.

    参考文献
    相似文献
    引证文献
引用本文

任德铭,郭帆,曹宇昂,李雨航,朱文喆,李永坤,许胤龙.面向分布式文件系统的大目录动态拆分机制.计算机系统应用,2026,35(7):120-129

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2025-11-25
  • 最后修改日期:2025-12-16
  • 录用日期:
  • 在线发布日期: 2026-05-22
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62661041 传真: Email:csa@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号