本文已被：浏览 2254次 下载 3319次
Received:June 19, 2017 Revised:June 30, 2017
Received:June 19, 2017 Revised:June 30, 2017
中文摘要: 针对传统车联网平台在处理海量数据时存在吞吐量小，实时性差的问题，设计了一种基于大数据流处理技术的实时分析系统.系统分为数据采集、 数据转发、实时分析、数据存储和可视化展示5层.为了满足系统高并发接入以及实时性的需求，引入Storm实时计算系统进行数据的实时分析.同时，利用Kafka消息队列的异步通信机制将各层之间解耦，采用Hbase进行海量数据存储，从而提高车联网非机构化数据存储效率.另外，针对访问数据库开销大的问题，采用Redis缓存策略，进一步提高查询效率.实验证明，较传统的多线程处理平台，该系统具有低延迟，高吞吐，可拓展等特点，能够满足车联网大数据流处理要求.
Abstract:To address the large data processing problem on the vehicle networking platform in which the data throughput is small, and its poor real-time feature, this paper proposes a new real-time analysis system based on the big data stream processing technology. The proposed system consists of 5 layers including data acquisition, data forwarding, real-time data analysis, data cache and storage, and visual display. Specifically, it introduces Storm real-time computing system to real-time data processing, which is beneficial to the high concurrent access and can meet real-time requirements of the system. Furthermore, aiming at the problem that the access to the database is expensive, Redis cache strategy is used to improve the query efficiency. Experiments show that the system has low latency, high throughput, and scalability compared with the conventional multithreaded processing platform, which is able to satisfy the requirements of vehicle network data stream processing.
文章编号： 中图分类号： 文献标志码：
基金项目:国家自然科学基金（61273323）
引用文本：
张春风,申飞,张俊,陈杰,刘静.基于Storm的车联网数据实时分析系统.计算机系统应用,2018,27(3):44-50
ZHANG Chun-Feng,SHEN Fei,ZHANG Jun,CHEN Jie,LIU Jing.Real-Time Analysis System of Vehicle Network Data Based on Storm.COMPUTER SYSTEMS APPLICATIONS,2018,27(3):44-50
张春风,申飞,张俊,陈杰,刘静.基于Storm的车联网数据实时分析系统.计算机系统应用,2018,27(3):44-50
ZHANG Chun-Feng,SHEN Fei,ZHANG Jun,CHEN Jie,LIU Jing.Real-Time Analysis System of Vehicle Network Data Based on Storm.COMPUTER SYSTEMS APPLICATIONS,2018,27(3):44-50
