面向任务调度优化的分布式系统信息管理框架
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:


System Information Management Framework of Distributed System for Task Scheduling Optimization
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 增强出版
  • |
  • 文章评论
    摘要:

    近年来深度学习作为学术界与工业界共同关注的热点,取得了飞跃式的发展,在计算机视觉、语音识别等领域取得了令人瞩目的成果.深度学习分训练与推理两个阶段,在实际应用中主要关注的是推理阶段.深度学习推理过程中伴随着巨大的计算量,通过分布式系统提高其计算速度也得到了越来越多的关注.然而,构建分布式深度学习推理系统面临着深度学习加速设备更新迭代快速、上层应用及计算任务复杂多样等挑战.本文设计并实现的系统信息管理框架,用于收集并处理系统中的各类信息,收集及处理的规则具有高度的可扩展性和灵活性,并提供通用的RESTful API数据访问接口,以支持分布式深度学习推理系统对各类硬件加速器的灵活兼容性以及对任务调度策略的动态调整能力.最后,本文通过一个应用实例对该框架的功能进行验证并对实验结果进行分析.

    Abstract:

    In recent years, deep learning, as a hotspot of common concern in academia and industry, has made great progress and achieved remarkable achievements in computer vision, speech recognition and other fields. It is divided into two stages:training and inferencing. In practical application, the main concern is the inferencing stage. The process of deep learning inferecing is accompanied by a huge amount of computation, and more and more attention has been paid to using distributed system to improve its computing speed. However, the construction of distributed deep learning inferencing system is faced with the challenges such as rapid updating and iteration of deep learning accelerators, complex of applications and computing tasks. The information management mechanism proposed in this study is used to collect and process all kinds of information in the distributed system, and the rules of collection and processing are highly customizable and flexible. It also provides a universal RESTful API data access interface to support the flexible compatibility of various hardware and the dynamic adjustment ability of task scheduling strategy in the deep learning inferencing system. Finally, we verified the function of the mechanism through an example and analysed the experimental results.

    参考文献
    相似文献
    引证文献
引用本文

胡亚辉,朱宗卫,刘黄河,王超.面向任务调度优化的分布式系统信息管理框架.计算机系统应用,2019,28(11):54-62

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2019-04-26
  • 最后修改日期:2019-05-23
  • 录用日期:
  • 在线发布日期: 2019-11-08
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号