本文已被：浏览 1774次 下载 2593次
Received:April 21, 2014 Revised:May 19, 2014
Received:April 21, 2014 Revised:May 19, 2014
中文摘要: 在“辽河流域水环境管理技术综合示范”项目中, 随着时间的累积, 环境监测数据处理系统采集到的数据量越来越大.然而目前辽宁省环境监测数据处理系统无法有效处理日益增长的海量数据.研究运用大数据技术, 改进环境监测数据处理系统中的数据中心.利用HDFS强大的数据存储、管理功能, 以应对数据量的增长, 利用MapReduce及Hadoop其他相关工具, 快速处理海量数据, 降低数据规模, 最后将数据存储到数据库中.
Abstract:With the accumulation of time, data collected by environmental monitoring data processing system is more in the project of "Comprehensive Demonstration of Water Environment Management about Liao He River". However, Liaoning province environmental monitoring data processing system cannot effectively deal with massive data growing. In this paper, we study and use big data technology. After that, we improve the data center of the environmental monitoring data processing system. We can use HDFS powerful function of data storage and management to cope with the growth of the massive data and take advantage of MapReduce and other Hadoop related tools, rapidly processing massive data and reducing the data size. Finally, the data will be stored in the database.
keywords: big data MapReduce Hadoop parallel computing distributed
文章编号： 中图分类号： 文献标志码：
基金项目:国家水体污染控制与治理科技重大专项(2012ZX07505003)
引用文本：
李安增,王宁,王常权,邱燕.大数据技术在环境信息中的应用.计算机系统应用,2015,24(1):60-64
LI An-Zeng,WANG Ning,WANG Chang-Quan,QIU Yan.Application of Big Data Technology to Environment Information.COMPUTER SYSTEMS APPLICATIONS,2015,24(1):60-64
李安增,王宁,王常权,邱燕.大数据技术在环境信息中的应用.计算机系统应用,2015,24(1):60-64
LI An-Zeng,WANG Ning,WANG Chang-Quan,QIU Yan.Application of Big Data Technology to Environment Information.COMPUTER SYSTEMS APPLICATIONS,2015,24(1):60-64
