Study on massive data incremental processing method based on distributed parallel computing framework

杨世旺; 钟雨星; 杨春晨; 孙明铎; 施焕健

doi:10.3969/j.issn.1009-1831.2019.S1.002

您当前的位置：

首页 >

文章列表页 >

Study on massive data incremental processing method based on distributed parallel computing framework

更新时间：2026-03-05

- Study on massive data incremental processing method based on distributed parallel computing framework
- Vol. 21, Issue S1, (2019)
- 作者机构：
  
  1. 国网浙江省电力有限公司丽水供电公司,浙江,丽水,323000
  2. 国网浙江电力有限公司平湖市供电公司,浙江平湖,315300
  3. 浙江华云信息科技有限公司,杭州,310000
  4. 国网浙江电力有限公司慈溪市供电公司,浙江慈溪,315300
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.1009-1831.2019.S1.002
  CLC：
- Published：2019
- 稿件说明：
移动端阅览
杨世旺, 钟雨星, 杨春晨, et al. Study on massive data incremental processing method based on distributed parallel computing framework[J]. 2019, 21(S1).
DOI：

杨世旺, 钟雨星, 杨春晨, et al. Study on massive data incremental processing method based on distributed parallel computing framework[J]. 2019, 21(S1). DOI： 10.3969/j.issn.1009-1831.2019.S1.002.

摘要

随着大数据时代的到来，各大企业已经陆续建设数据仓库，如何提升海量数据处理的效率逐渐成为数仓应用中的重要问题。分析了企业大数据平台现状，提出了基于分布式并行计算框架的数据增量处理方法，并基于此方法对海量增量数据处理进行了实践，验证结果表明，该方法提升了数据处理效率，增强了企业数据仓库数据处理的准确性。

Abstract

With the arrival of the era of big data

how to improve the efficiency of mass data processing is more important in the application of enterprise data warehouse. Based on the analysis of the current situation of data sharing application platform of electric

a data incremental processing method based on a distributed parallel computing framework is proposed

and the practice of massive incremental data processing is carried out based on this method. Verification results show that the problem of low data processing efficiency is solved

and the accuracy of data processing in enterprise data warehouse is enhanced.

关键词

Keywords

references

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

AI问答

⁰