A Design of Decentralized Data Processing Scheme

Li Guang-cheng; Zhao Qing-lin; Xie Kan

doi:10.12052/gdutxb.210104

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

A Design of Decentralized Data Processing Scheme

更新时间：2026-05-27

- A Design of Decentralized Data Processing Scheme
- Vol. 38, Issue 6, Pages: 77-83(2021)
- 作者机构：
  
  1. 澳门科技大学资讯科技学院,澳门,999078
  2. 广东工业大学自动化学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.210104
  CLC： TP391
- Online First：10 November 2021，
  
  Published：2021
- 稿件说明：
移动端阅览
李光程, 赵庆林, 谢侃. 去中心化的数据处理方案设计[J]. 广东工业大学学报, 2021, 38(6): 77-83.

Li Guang-cheng, Zhao Qing-lin, Xie Kan. A Design of Decentralized Data Processing Scheme[J]. 2021, 38(6): 77-83.
李光程, 赵庆林, 谢侃. 去中心化的数据处理方案设计[J]. 广东工业大学学报, 2021, 38(6): 77-83. DOI： 10.12052/gdutxb.210104.

Li Guang-cheng, Zhao Qing-lin, Xie Kan. A Design of Decentralized Data Processing Scheme[J]. 2021, 38(6): 77-83. DOI： 10.12052/gdutxb.210104.

摘要

传统的基于主/从的数据处理框架容易受到主节点的单点故障和性能瓶颈的影响。相比之下

区块链系统采用去中心化的框架

能够聚合海量的计算资源。提出了一种基于区块链的数据处理框架

利用区块链的优点来解决中心化框架的缺点。在所提出的框架中

区块链存储任务信息

采用的有用工作共识证明共识机制使节点能够使用其计算资源处理任务

同时竞争领导者(将待处理的任务分派到区块链)。模拟表明

所提出的框架在吞吐量和任务响应时间方面优于集中式框架。

Abstract

Conventional master/slave-based data processing frameworks are vulnerable to single point of failure and performance bottlenecks of the master node. In contrast

blockchain systems adopt a decentralized framework and are capable of aggregating enormous computing resources. A blockchain-based data processing framework is proposed that utilizes the advantages of the blockchain for solving the drawbacks of the centralized framework. In this framework

the blockchain stores the task information and the adopted proof of useful work consensus enables nodes to process tasks using their computing resources

while competing for the leader (who dispatches pending tasks to the blockchain). Extensive simulations show that the proposed framework is better than the centralized framework in terms of the throughput and the task response time.

关键词

Keywords

references

DEAN J, GHEMAWAT S. MapReduce: simplified data processing on large clusters [J]. Communications of the ACM, 2008, 51(1): 107-113.

TOSHNIWAL A, TANEJA S, SHUKLA A, et al. Storm@ Twitter[C]//Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. [S.l.]: ACM, 2014: 147-156.

CARBONE P, KATSIFODIMOS A, EWEN S, et al . Apache flink: stream and batch processing in a single engine [J ] . Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015, 36(4): 28-38.

REN Z, XU X, WAN J, et al. Workload characterization on a production hadoop cluster: a case study on Taobao[C]//2012 IEEE International Symposium on Workload Characterization (IISWC). La Jolla, USA: IEEE, 2012: 3-13.

WAN J, LIU M, HU X, et al. Dual-JT: toward the high availability of Job Tracker in Hadoop[C]//4th IEEE International Conference on Cloud Computing Technology and Science Proceedings. Taipei: IEEE, 2012: 263-268.

SWAN M. Blockchain: blueprint for a new economy[M]. USA: O’Reilly Media, Inc., 2015.

VUKOLIĆ M. The quest for scalable blockchain fabric: Proof-of-Work vs. BFT replication[C]//International Workshop on Open Problems in Network Security. Cham: Springer, 2015.

WOOD G, BUTERIN V. Ethereum: a secure decentralised generalised transaction ledger [J]. Ethereum Project Yellow Paper, 2014: 1-32.

ZHANG F, EYAL I, ESCRIVA R, et al. REM: resource-efficient mining for blockchains[C]//26th USENIX Conference on Security Symposium. [S.l.]: ACM, 1427-1444.

BENDJOUDI A, MELAB N, TALBI E G. An adaptive hierarchical master-worker (AHMW) framework for grids—application to BB algorithms [J]. Journal of Parallel and Distributed Computing, 2012, 72(2): 120-131.

LI M, WENG J, YANG A, et al . Crowdbc: a blockchain-based decentralized framework for crowdsourcing [J ] . IEEE Transactions on Parallel and Distributed Systems, 2018, 30(6): 1251-1266.

LI Z, YANG Z, XIE S, et al . Credit-based payments for fast computing resource trading in edge-assisted internet of things [J ] . IEEE Internet of Things Journal, 2019, 6(4): 6606-6617.

YANG Z, YANG K, LEI L, et al . Blockchain-based decentralized trust management in vehicular networks [J ] . IEEE Internet of Things Journal, 2018, 6(2): 1495-1505.

HOEKSTRA M, LAL R, PAPPACHAN P, et al. Using innovative instructions to create trustworthy software solutions[C]//Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy. [S.l.]: ACM, 2013.

HUNT T, ZHU Z, XU Y, et al. Ryoan: a distributed sandbox for untrusted computation on secret data[J]. Proceedings of the 12th USENIX conference on Operating Systems Design and ImplementationNovember. [S.l.]: ACM, 2016: 533-549.

MCKEEN F, ALEXANDROVICH I, ANATI I, et al. Intel ® software guard extensions (Intel® SGX) support for dynamic memory management inside an enclave[C ] //Proceedings of the Hardware and Architectural Support for Security and Privacy. [S.l. ] : ACM, 2016.

EYAL I, GENCER A E, SIRER E G, et al. Bitcoin-NG: a scalable blockchain protocol[C]// Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation. [S.l.]: ACM, 2016.

LUU L, NARAYANAN V, ZHENG C, et al. A secure sharding protocol for open blockchains[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2016.

ZAMANI M, MOVAHEDI M, RAYKOVA M. RapidChain: scaling blockchain via full sharding[C]//Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2018.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Modeling of Grain Reserve Multi-chain Supervision System Based on Blockchain

A Quantum-encrypted Blockchain Based on Delegated Proof of Stake (DPoS) Extension

An Analysis of Blockchain Applications in Financial Scenarios and an Exploration of Enterprise Software Architecture of Blockchain as a Service(BaaS)

Revolution and Prospect of Private Equity Crowd-Funding Platform Based on Blockchain Technology

Cyber Intelligent Economy and Blockchain

Related Author

Liu Yang

Yang Wei-dong

Wang Yan-lin

Lao Nan-xin

Wang Bang-hai

Chen Bing-er

Dai Ke-mian

Wei Sheng

Related Institution

College of Information Science and Engineering, Henan University of Technology

Henan Provincial Key Laboratory of Grain Photoelectric Detection and Control

Ministry of Education Key Laboratory of Grain Information Processing and Control

School of Computers, Guangdong University of Technology

Bank of Guangzhou

AI问答

⁰