Sentiment Classification Based on Double Attention Convolutional Neural Network Model

Zeng Bi-qing; Han Xu-li; Wang Sheng-yu; Xu Ru-yang; Zhou Wu

doi:10.12052/gdutxb.190042

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Sentiment Classification Based on Double Attention Convolutional Neural Network Model

更新时间：2026-05-27

- Sentiment Classification Based on Double Attention Convolutional Neural Network Model
- Vol. 36, Issue 4, Pages: 10-17(2019)
- 作者机构：
  
  1. 华南师范大学软件学院,广东,佛山,528225
  2. 华南师范大学计算机学院,广东,广州,510631
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.190042
  CLC： TP391
- Online First：18 June 2019，
  
  Published：2019
- 稿件说明：
移动端阅览
曾碧卿, 韩旭丽, 王盛玉, 徐如阳, 周武. 基于双注意力卷积神经网络模型的情感分析研究[J]. 广东工业大学学报, 2019, 36(4): 10-17.

Zeng Bi-qing, Han Xu-li, Wang Sheng-yu, et al. Sentiment Classification Based on Double Attention Convolutional Neural Network Model[J]. 2019, 36(4): 10-17.
曾碧卿, 韩旭丽, 王盛玉, 徐如阳, 周武. 基于双注意力卷积神经网络模型的情感分析研究[J]. 广东工业大学学报, 2019, 36(4): 10-17. DOI： 10.12052/gdutxb.190042.

Zeng Bi-qing, Han Xu-li, Wang Sheng-yu, et al. Sentiment Classification Based on Double Attention Convolutional Neural Network Model[J]. 2019, 36(4): 10-17. DOI： 10.12052/gdutxb.190042.

摘要

卷积神经网络(Convolutional Neural Networks

CNN)无法判别输入文本中特征词与情感的相关性. 因此提出一种双注意力机制的卷积神经网络模型(Double Attention Convolutional Neural Networks

DACNN)

将词特征与词性特征有效融合后得到本文的特征表示

确定情感倾向. 本文提出局部注意力的卷积神经网络模型

改进卷积神经网络的特征提取能力

采用双通道的局部注意力卷积神经网络提取文本的词特征和词性特征. 然后使用全局注意力为特征分配不同的权重

有选择地进行特征融合

最后得到文本的特征表示. 将该模型在MR和SST-1数据集上进行验证

较普通卷积神经网络和传统机器学习方法

在准确率上分别取得0.7%和1%的提升.

Abstract

Convolutional Neural Networks (CNN) cannot discriminate the correlation between feature words and emotions in the input text. Therefore

this paper proposes a double attention mechanism convolutional neural network to incorporate word and POS(Part-of-speech Tagging) information into sentiment classification (Double Attention Convolutional Neural Networks

DACNN)

which combines the word features with the part-of-speech features to obtain the feature representation of the text and determine the sentiment tendency. In this paper

we introduce an attention convolutional neural network to incorporate word and POS information into sentiment classification. The first local attention mechanism captures local feature from word and POS channels. And then we propose a novel architect for combining two features by a global attention mechanism

which can decide how much information to use from word or POS information. The model is validated on MR and SST-1 datasets

achieving 0.7% and 1% improvement in accuracy compared to conventional convolutional neural networks and traditional machine learning methods.

关键词

Keywords

references

PANG B, LEE L, VAITHYANATHAN S. Thumbs up: sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Stroudsburg: ACM. 2002: 79-86.

PEREZ-ROSAZ V, BANEA C, MIHALCEA R. Learning sentiment lexicons in spanish[C]//LREC. Istanbul: ELRA. 2012, 12: 73.

KIRITCHENKO S, ZHU X, MOHAMMAD S M. Sentiment analysis of short informal text [J]. Journal of Artificial Intelligence Research, 2014, 50: 723-762.

WANG S, MANNING C D. Baselines and bigrams: simple, good sentiment and topic classification[C]// Meeting of the Association for Computational Linguistics: Short Papers. Jeju Island: ACL, 2012: 90-94.

ZHANG L, WANG S, LIU B. Deep learning for sentiment analysis: a survey [J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2018, 8(4): e1253.

ZHANG Y J, XU Z T, XUE X Y. Fusion of multiple features for chinese named entity recognition based on maximum entropy model [J]. Journal of Computer Research and Development, 2008, 45(6): 1004-1010.

CHEN P, SUN Z, BING L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017: 452-461.

WU Y, SCHUSTER M, CHEN Z, et al. Google's neural machine translation system: Bridging the gap between human and machine translation[J]. arXiv preprint arXiv: 1609.08144, 2016.

YIN W, SCHÜTZE H, XIANG B, et al . Abcnn: attention-based convolutional neural network for modeling sentence pairs [J ] . Transactions of the Association for Computational Linguistics, 2016, 4: 259-272.

GOLUB D, HE X. Character-level question answering with attention[J]. arXiv preprint arXiv: 1604.00727, 2016

RUSH A M, CHOPRA S, WESTON J. A neural attention model for abstractive sentence summarization[J]. arXiv preprint arXiv: 1509.00685, 2015.

REI M, CRICHTON G K O, PYYSALO S. Attending to characters in neural sequence labeling models[J]. arXiv preprint arXiv: 1611.04361, 2016.

KIM Y. Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv: 1408.5882, 2014

KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv: 1404.2188, 2014.

VASWANI A, SHAZEER N, PARMAR N, et al . Attention is All you Need [J ] . Neural information processing systems, 2017: 5998-6008.

MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems. Lake tahoe Nevada: NIPS, 2013: 3111-3119.

BENGIO Y, DUCHARME R, VINCENT P, et al . A neural probabilistic language model [J ] . Journal of machine learning research, 2003, 3(6): 1137-1155.

MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781, 2013.

LIANG B, LIU Q, XU J, et al . Aspec t-based sentiment analysis based on multi-attention CNN [J ] . Journal of Computer Research and Development, 2017, 54(8): 1724-1735.

ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification[J]. arXiv preprint arXiv: 1510.03820, 2015.

SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, Washington: ACL, 2013: 1631-1642.

TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[J]. arXiv preprint arXiv: 1503.00075, 2015.

TANG D, QIN B, LIU T. Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: ACL, 2015: 1422-1432.

BHATIA P, JI Y, EISENSTEIN J. Better document-level sentiment analysis from rst discourse parsing[J]. arXiv preprint arXiv: 1509.01599, 2015.

XU K, BA J, KIROS R, et al. Show, attend and tell: Neural image caption generation with visual attention[C]//International Conference on Machine Learning. Lille, France: ACM, 2015: 2048-2057.

BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473, 2014.

LUONG M T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[J]. arXiv preprint arXiv: 1508.04025, 2015.

YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Diego, California: ACL. 2016: 1480-1489.

WANG L, CAO Z, MELO G D, et al. Relation classification via multi-level attention CNNs[C]// Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016: 1298-1307.

WANG S, MANNING C D. Baselines and bigrams: simple, good sentiment and topic classification[C]// Meeting of the Association for Computational Linguistics: Short Papers. Association for Computational Linguistics, Jeju Island: ACL, 2012: 90-94.

SHIN B, LEE T, CHOI J D, et al . Lexicon integrated CNN models with attention for sentiment analysis [J ] . Empirical Methods in Natural Language Processing, 2017: 149-158.

LE Q V, MIKOLOV T. Distributed representations of sentences and documents [J]. International Conference on Machine Learning, 2014: 1188-1196.

DONG L, WEI F, LIU S, et al . A statistical parsing framework for sentiment classification [J ] . Computational Linguistics, 2014, 41(2): 293-336.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Unsupervised GAN Multi-focus Image Fusion Based on Attention Mechanism

Point Cloud Module Based on Hidden Features

Network Intrusion Detection for Web Security Scanner

Ground Deformation Monitoring and Influencing Factors Analysis in Fuzhou City Based on PSI Technology

An Optimal Operation Strategy for Community Microgrid Alliances Considering Power Exchange under Multi-package Shared Energy Storage Model

Related Author

Li Weitong

Jiang Yunfeng

Tan Zhe

Huang Guoheng

Chen Jiahong

Gao Xiang

Li Guangping

Zou Shuhang

Related Institution

School of Information Engineering, Guangdong University of Technology

School of Computer Science and Technology, Guangdong University of Technology

Samsung Electro-mechanics Co., Ltd., Suwon City

School of Automation, Guangdong University of Technology

School of Electronics and Information Engineering, Guangdong Polytechnic Normal University

AI问答

⁰