屈志坚, 洪应迪, 王子潇. 配电网调度监测数据的多线程集群共享内存折叠压缩方法[J]. 中国电机工程学报, 2021, 41(3): 921-931. DOI: 10.13334/j.0258-8013.pcsee.191947
引用本文: 屈志坚, 洪应迪, 王子潇. 配电网调度监测数据的多线程集群共享内存折叠压缩方法[J]. 中国电机工程学报, 2021, 41(3): 921-931. DOI: 10.13334/j.0258-8013.pcsee.191947
QU Zhijian, HONG Yingdi, WANG Zixiao. Multi-threaded Cluster Shared Memory Folding Compression Method for Distribution Network Monitoring Data[J]. Proceedings of the CSEE, 2021, 41(3): 921-931. DOI: 10.13334/j.0258-8013.pcsee.191947
Citation: QU Zhijian, HONG Yingdi, WANG Zixiao. Multi-threaded Cluster Shared Memory Folding Compression Method for Distribution Network Monitoring Data[J]. Proceedings of the CSEE, 2021, 41(3): 921-931. DOI: 10.13334/j.0258-8013.pcsee.191947

配电网调度监测数据的多线程集群共享内存折叠压缩方法

Multi-threaded Cluster Shared Memory Folding Compression Method for Distribution Network Monitoring Data

  • 摘要: 针对配电网中数据指数增长造成的读写时延越来越长的问题,提出一种多线程集群共享内存折叠压缩新方法。将数据结构扁平化处理融入于数据压缩之中,通过启用内存折叠方法,在写入内存过程中消除数据冗余,改变数据结构,减少刷新到磁盘的次数,同时缓解磁盘块缓存的压力,从而提高对数据的读写性能,以千万条数据记录的某动车段10kV配电网远动调度监控系统实测数据为例,搭建4个节点测试集群,进行集群内存压缩导入延时测试与读写性能测试。实验结果表明,启用内存压缩能优化内存结构,提升调度监测数据库的读写性能。

     

    Abstract: Aiming at the problem that the read and write delays caused by the exponential growth of data in the distribution network are getting longer, this paper proposed a new multi-threaded cluster shared memory folding compression method. The data structure flattening process was integrated into the data compression. By enabling the memory folding method, data redundancy was eliminated during the process of writing the memory, the data structure was changed, the number of times of flushing to the disk was reduced, and the pressure of the disk block buffer was alleviated. Therefore, the reading and writing performance of the data was improved. Taking the measured data of the remote movement dispatching monitoring system of a 10kV distribution network in Beijing as an example, a four-node test cluster was built to perform the cluster memory compression import delay test and the read/write performance test. The experimental results show that enabling memory compression can optimize the memory structure and significantly improve the read and write performance of the scheduling monitoring database.

     

/

返回文章
返回