A Power Text Classification Model Based on Dual-layer Attention Mechanism

WU Tongxin; JI Xin; YANG Chengyue; CHEN Yiting; YANG Zhiwei

doi:10.11930/j.issn.1004-9649.202411021

English Version Reader LoginJournal Login

您当前的位置：

首页 >

文章列表页 >

A Power Text Classification Model Based on Dual-layer Attention Mechanism

更新时间：2025-12-20

- A Power Text Classification Model Based on Dual-layer Attention Mechanism
- Vol. 58, Issue 11, Pages: 156-163(2025)
- 作者机构：
  
  国家电网有限公司大数据中心,北京,100031
- 作者简介：
- 基金信息：
- DOI：10.11930/j.issn.1004-9649.202411021
  CLC：
- Published Online：01 December 2025，
  
  Published：01 December 2025
- 稿件说明：
移动端阅览
武同心, 纪鑫, 杨成月, 陈屹婷, 杨智伟. 基于双层注意力机制的电力文本分类模型[J]. 中国电力, 2025, 58(11): 156-163.

WU Tongxin, JI Xin, YANG Chengyue, et al. A Power Text Classification Model Based on Dual-layer Attention Mechanism[J]. 2025, 58(11): 156-163.
武同心, 纪鑫, 杨成月, 陈屹婷, 杨智伟. 基于双层注意力机制的电力文本分类模型[J]. 中国电力, 2025, 58(11): 156-163. DOI： 10.11930/j.issn.1004-9649.202411021.

WU Tongxin, JI Xin, YANG Chengyue, et al. A Power Text Classification Model Based on Dual-layer Attention Mechanism[J]. 2025, 58(11): 156-163. DOI： 10.11930/j.issn.1004-9649.202411021.

摘要

电力领域中存在大量的中文文本数据，传统文本挖掘方法存在分词难度大、文本特征提取具有局限性、处理文本中的复杂关系时效果不好等问题，严重限制了对电力信息的深度理解与分类。基于此，提出了一种结合文本卷积网络（text convolutional neural networks，TextCNN）与注意力（Attention）机制的电力文本分类模型，对输入层、TextCNN层、第1个注意力层、池化层、第2个注意力层以及输出层进行了分层优化设计，并通过实验对模型进行了验证。结果显示，TextCNN-Attention模型在电力文本数据集上的文本分类准确率达到了96.8%，精确率达到了86.3%，召回率达到了90.3%，综合评价值达到了88.2%，表明TextCNN-Attention模型可有效提升电力文本分类的效率，为深度学习算法在电力文本分类领域的应用提供了经验借鉴。

Abstract

There are a large amount of Chinese text data in the electric power field

traditional text mining methods are faced with problems such as difficulty in word segmentation

limitations in text feature representation

and poor performance in handling complex relationships in text

which limit the deep understanding and classification of power information. This paper proposes a power text classification model that combines text convolutional neural networks (TextCNN) and Attention mechanism. A hierarchical optimization design was carried out for the input layer

TextCNN layer

first attention layer

pooling layer

second attention layer

and output layer

with experimental validation conducted to verify the model's performance. The results show that the proposed TextCNN-Attention model achieved a text classification accuracy of 96.8%

with a precision of 86.3%

a recall of 90.3%

and a F1 score (comprehensive evaluation metric) of 88.2% on the power text dataset

demonstrating the superior performance of the TextCNN-Attention model in processing power texts. This study can provide valuable experiences for application of deep learning in power text classification.

关键词

Keywords

references

1杨春霞, 马文文, 徐奔, 等. 融合标签信息的分层图注意力网络文本分类模型[J]. 计算机工程与科学, 2023, 45 (11): 2018- 2026. DOI

YANG Chunxia, MA Wenwen, XU Ben, et al. A hierarchical graph attention network text classification model that integrates label information[J]. Computer Engineering & Science, 2023, 45 (11): 2018- 2026. DOI

2周凯, 焦龄霄, 胡志坚, 等. 基于注意力机制的CNN-BiLSTM操作票自动校核方法[J]. 武汉大学学报(工学版), 2023, 56 (9): 1114- 1123. DOI

ZHOU Kai, JIAO Lingxiao, HU Zhijian, et al. Automatic verification method of operation ticket based on attention mechanism of CNN-BiLSTM[J]. Engineering Journal of Wuhan University, 2023, 56 (9): 1114- 1123. DOI

3杨茜. 基于Bi-LSTM和图注意力网络的多标签文本分类算法[J]. 计算机应用与软件, 2023, 40 (9): 145- 150. DOI

YANG Xi. Multi label text classification algorithm based on BI-LSTM and graph attention network[J]. Computer Applications and Software, 2023, 40 (9): 145- 150. DOI

4张爱军, 刘会强, 慕腾, 等. 计及经济-低碳时空匹配的高比例新能源电力系统扩展规划[J]. 智慧电力, 2024, 52 (11): 72- 80. DOI

ZHANG Aijun, LIU Huiqiang, MU Teng, et al. Expansion planning for high proportion of renewable energy power system considering economy-low carbon spatiotemporal matching[J]. Smart Power, 2024, 52 (11): 72- 80. DOI

5郭森, 王宇慧, 周劲松, 等. 数字经济发展对我国省级电力行业碳排放影响研究[J]. 智慧电力, 2024, 52 (11): 32- 39. DOI

GUO Sen, WANG Yuhui, ZHOU Jinsong, et al. Impact of digital economy development on carbon emissions in China's provincial electric power industry[J]. Smart Power, 2024, 52 (11): 32- 39. DOI

6侯慧, 吴文杰, 魏瑞增, 等. 基于注意力机制的CNN-LSTM-XGBoost台风暴雨电力气象混合预测模型[J]. 智慧电力, 2024, 52 (10): 96- 102. DOI

HOU Hui, WU Wenjie, WEI Ruizeng, et al. Attention mechanism based CNN-LSTM-XGBoost electric power meteorological hybrid forecasting model of typhoon rainstorm[J]. Smart Power, 2024, 52 (10): 96- 102. DOI

7庞清乐, 韩松易, 周泰, 等. 基于ASRUKF和IMC算法的电力信息物理系统虚假数据注入攻击检测[J]. 智慧电力, 2024, 52 (7): 111- 118. DOI

PANG Qingle, HAN Songyi, ZHOU Tai, et al. False data injection attack detection of cyber-physical power system based on ASRUKF and IMC algorithms[J]. Smart Power, 2024, 52 (7): 111- 118. DOI

8程钢, 陈秀明, 于翔. 基于TextCNN融合模型的离散情感分析[J]. 科学技术创新, 2023 (21): 124- 127. DOI

CHENG Gang, CHEN Xiuming, YU Xiang. Discrete emotion analysis based on TextCNN-BERT fusion model[J]. Scientific and Technological Innovation, 2023 (21): 124- 127. DOI

9胡容波, 郭诚, 王锦浩, 等. 基于深度学习的自然资源政策文本分类研究[J]. 高技术通讯, 2023, 33 (7): 692- 703. DOI

HU Rongbo, GUO Cheng, WANG Jinhao, et al. Research on classification of natural resources policy text based on deep learning[J]. Chinese High Technology Letters, 2023, 33 (7): 692- 703. DOI

10彭俊利, 王少泫, 陆正球, 等. 基于LDA-TF-IDF和Word2Vec文档表示[J]. 浙江纺织服装职业技术学院学报, 2023, 22 (2): 91- 96. DOI

PENG Junli, WANG Shaoxuan, LU Zhengqiu, et al. Document representation based on LDA-TF-IDF and Word2Vec[J]. Journal of Zhejiang Fashion Institute of Technology, 2023, 22 (2): 91- 96. DOI

11刘佳欣. 基于深度学习的情感文本分类研究[D]. 兰州: 兰州理工大学, 2023.

LIU Jiaxin. Research on sentiment text classification based on deep learning[D]. Lanzhou: Lanzhou University of Technology, 2023.

12王振, 杨国锋, 陈天池. 基于深度学习的长文本分类[J]. 中国新通信, 2019, 21 (6): 86- 88.

13HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313 (5786): 504- 507. DOI

14贾澎涛, 孙炜. 基于深度学习的文本分类综述[J]. 计算机与现代化, 2021 (7): 29- 37. DOI

JIA Pengtao, SUN Wei. A survey of text classification based on deep learning[J]. Computer and Modernization, 2021 (7): 29- 37. DOI

15彭清泉, 王丹. 基于深度学习的医疗文本分类模型[J]. 电子设计工程, 2023, 31 (5): 51- 54. DOI

PENG Qingquan, WANG Dan. Medical text classification model based on deep learning[J]. Electronic Design Engineering, 2023, 31 (5): 51- 54. DOI

16邵栩辉. 基于深度学习的训练词向量和文本分类[J]. 电子制作, 2020 (20): 80- 82,100. DOI

SHAO Xuhui. Training word vectors and text classification based on deep learning[J]. Practical Electronics, 2020 (20): 80- 82,100. DOI

17吕淑宝, 王明月, 翟祥, 等. 一种深度学习的信息文本分类算法[J]. 哈尔滨理工大学学报, 2017, 22 (2): 105- 111. DOI

LYU Shubao, WANG Mingyue, ZHAI Xiang, et al. AN information text classification algorithm based on DBN[J]. Journal of Harbin University of Science and Technology, 2017, 22 (2): 105- 111. DOI

18祝亮. 基于CNN深度学习的自媒体文本分类方法的研究[J]. 电脑知识与技术, 2021, 17 (21): 97- 100.

ZHU Liang. Research on we-media text classification method based on CNN deep learning[J]. Computer Knowledge and Technology, 2021, 17 (21): 97- 100.

19徐建飞, 吴跃成. 基于BERT-BiLSTM-CNN模型的新闻文本分类研究[J]. 软件工程, 2023, 26 (6): 11- 15. DOI

XU Jianfei, WU Yuecheng. Research on news text classification based on BERT-BiLSTM-CNN[J]. Software Engineering, 2023, 26 (6): 11- 15. DOI

20YANG X Z, PENG G J, ZHANG D N, et al. PowerDetector: malicious PowerShell script family classification based on multi-modal semantic fusion and deep learning[J]. China Communications, 2023, 20 (11): 202- 224. DOI

21杨秀璋, 武帅, 张苗, 等. 基于TextCNN和Attention的微博舆情事件情感分析[J]. 信息技术与信息化, 2021 (7): 41- 46. DOI

YANG Xiuzhang, WU Shuai, ZHANG Miao, et al. Emotional analysis of microblog public opinion events based on TextCNN and Attention[J]. Information Technology and Informatization, 2021 (7): 41- 46. DOI

22ZHOU X W, WANG H F, LI S S, et al. Complex traffic scene image classification based on sparse optimization boundary semantics deep learning[J]. Wuhan University Journal of Natural Sciences, 2023, 28 (2): 150- 162. DOI

23齐佳琪, 迟呈英, 战学刚. ERNIE-CNN文本分类模型[J]. 辽宁科技大学学报, 2021, 44 (1): 56- 61. DOI

QI Jiaqi, CHI Chengying, ZHAN Xuegang. ERNIE-CNN text classification model[J]. Journal of University of Science and Technology Liaoning, 2021, 44 (1): 56- 61. DOI

24王宣军, 于虹, 祁兵, 等. 基于注意力机制的混合神经网络电力设备缺陷文本挖掘方法[J]. 电力信息与通信技术, 2023, 21 (9): 44- 51. DOI

WANG Xuanjun, YU Hong, QI Bing, et al. Hybrid neural network text mining method for power equipment defects based on attention mechanism[J]. Electric Power Information and Communication Technology, 2023, 21 (9): 44- 51. DOI

25HU J X, HU W H, CHEN J J, et al. Fault location and classification for distribution systems based on deep graph learning methods[J]. Journal of Modern Power Systems and Clean Energy, 2023, 11 (1): 35- 51. DOI

26SALAMA W M, ALY M H. Framework for COVID-19 segmentation and classification based on deep learning of computed tomography lung images[J]. Journal of Electronic Science and Technology, 2022, 20 (3): 100161. DOI

27姜炎宏. 基于深度学习的中文文本多标签分类研究[D]. 鞍山: 辽宁科技大学, 2020.

JIANG Yanhong. Research on Chinese text multi-label classification based on deep learning[D]. Anshan: University of Science and Technology Liaoning, 2020.

28曹湘, 李誉坤, 钱叶, 等. 基于混合神经网络的电力短文本分类方法研究[J]. 计算机与数字工程, 2019, 47 (5): 1145- 1150. DOI

CAO Xiang, LI Yukun, QIAN Ye, et al. Short text classification of electric power based on hybrid neural network[J]. Computer & Digital Engineering, 2019, 47 (5): 1145- 1150. DOI

29李天昊, 霍其润, 闫跃, 等. 融合ERNIE和注意力机制的中文关系抽取模型[J]. 小型微型计算机系统, 2022, 43 (6): 1226- 1231. DOI

LI Tianhao, HUO Qirun, YAN Yue, et al. Chinese relation extraction model based on ERNIE and attention mechanism[J]. Journal of Chinese Computer Systems, 2022, 43 (6): 1226- 1231. DOI

30宇文梦柯. 基于领域知识的电网设备故障缺陷案例分析方法的研究与实现[D]. 北京: 北京邮电大学, 2019.

YUWEN Mengke. Research and implementation of grid equipment defect case analysis based on field knowledge[D]. Beijing: Beijing University of Posts and Telecommunications, 2019.

31王丽鹏, 张鹏云, 和志强. 基于特征词匹配的政策文本分类算法研究与实现[J]. 河北省科学院学报, 2017, 34 (3): 1- 6. DOI

WANG Lipeng, ZHANG Pengyun, HE Zhiqiang. Research and implementation of text categorization algorithm based on feature word matching[J]. Journal of the Hebei Academy of Sciences, 2017, 34 (3): 1- 6. DOI

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Short-Term Power Load Forecasting Based on MSCNN-BiGRU-Attention

State Estimation Method for Distribution Network Based on Incomplete Measurement Data

A Joint Scenario Generation Method for Wind-Solar Meteorological Resources Based on Improved Generative Adversarial Network

Day-Ahead Probabilistic Prediction Model for Photovoltaic Power Based on Combined Deep Learning

Related Author

Ke LI

Tinglong PAN

Dezhi XU

Zhenzhen ZHOU

Yunhai SONG

Yuhao HE

Liwei WANG

Heyan HUANG

Related Institution

School of Electrical Engineering, Southeast University

School of Internet of Things Engineering, Jiangnan University

Zhejiang University

Overhaul and Test Center of UHV Transmission Company of China Southern Power Grid Co., Ltd.

State Grid Henan Electric Power Company

AI问答

⁰