Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (8): 230-236.doi: 10.11896/jsjkx.210600042

• Artificial Intelligence • Previous Articles     Next Articles

Text Classification Method Based on Information Fusion of Dual-graph Neural Network

YAN Jia-dan, JIA Cai-yan   

  1. School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China
    Beijing Key Lab of Traffic Data Analysis and Mining,Beijing Jiaotong University,Beijing 100044,China
  • Received:2021-06-04 Revised:2021-10-23 Published:2022-08-02
  • About author:YAN Jia-dan,born in 1995,postgra-duate.Her main research interests include natural language processing and text classification.
    JIA Cai-yan,born in 1976,Ph.D,professor,is a member of China Computer Federation.Her main research interests include machine learning,complex network analysis and social computing.
  • Supported by:
    Fundamental Research Funds for the Central Universities of Ministry of Education of China(2019JBZ110).

PDF (PC)

3060

Abstract: Graph neural networks are recently applied in text classification tasks.Compared with graph convolution network,the text level graph neural network model based on message passing(MP-GNN) has the advantages of low memory usage and supporting online testing.However,MP-GNN model only builds a lexical graph using the word co-occurrence information, and the obtained information lacks diversity.To address this problem,a text classification method based on information fusion of dual-graph neural network is proposed.Besides preserving the original lexical graph built in MP-GNN,this method constructes the semantic graph based on the cosine similarity between pairs of words,and controls the sparsity of the graph through a threshold,which makes more effective use of the multi-directional semantic information of the text.In addition,the ability of direct fusion and attention mechanism fusion are tested to fuse the text representation learned on lexical graph and semantic graph.Experimental results on 12 datasets(R8,R52 and other datasets commonly used for text classification) show that the proposed model demonstrates an obvious improvement on performance of text classification compared with the SOTA(state-of-the-art) methods TextLevelGNN,TextING and MPAD.

Key words: Attention mechanism, Graph neural network, Information fusion, Natural language processing, Semantic information, Text classification

CLC Number: 

  • TP391
[1]SHERVIN M,NAL K,ERIK C,et al.Deep Learning BasedText Classification:A Comprehensive Review[EB/OL].https://arxiv.org/abs/2004.03705v1.
[2]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).2014:1746-1751.
[3]LIU P,QIU X,HUANG X.Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.2016:2873-2879.
[4]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377.
[5]NIKOLENTZOS G,TIXIER A J,VAZIRGIANNIS M.Message Passing Attention Networks for Document Understanding[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.2020:8544-8551.
[6]LIU X,YOU X,ZHANG X,et al.Tensor Graph Convolutional Networks for Text Classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8409-8416.
[7]WU F,ZHANG T,YU T,et al.Simplifying Graph Convolu-tional Networks[C]//Proceedings of the 36th International Conference on Machine Learning.2019:6861-6871.
[8]HU L,YANG T,SHI C,et al.Heterogeneous graph attention networks for semi-supervised short text classification [C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confe-rence on Natural Language Processing(EMNLP-IJCNLP).2019:4823-4832.
[9]HUANG L,MA D,LI S,et al.Text level graph neural network for text classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.2019:3435-3441.
[10]ZHANG Y,YU X,CUI Z,et al.Every Document Owns ItsStructure:Inductive Text Classification via Graph Neural Networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:334-339.
[11]GREENE D,CUNNINGHAM P.Practical Solutions to theProblem of Diagonal Dominance in Kernel Document Clustering[C]//Proceedings of the 23rd International Conference on Machine Learning.2006:377-384.
[12]LI X,ROTH D.Learning question classifiers:the role of semantic information[C]//Proceedings of the 19th International Conference on Computational Linguistics.2002:1-7.
[13]PANG B,LEE L.Seeing Stars:Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.2005:115-124.
[14]MAAS A L,DALY R E,PHAM P T,et al.Learning Word Vectors for Sentiment Analysis[C]//Proceedings of the 49th An-nual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:142-150.
[15]SOCHER R,PERELYGIN A,POTTS C,et al.Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1631-1642.
[16]WIEBE J,WILSON T,CARDIE C.Annotating Expressions of Opinions and Emotions in Language[J].Language Resources and Evaluation,2005,39(2/3):165-210.
[17]PANG B,LEE L.A Sentimental Education:Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts[C]//Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics.2004:271-278.
[18]IYYER M,MANJUNATHA V,BOYD-GRABER J,et al.Deep Unordered Composition Rivals Syntactic Methods for Text Classification[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.2015:1681-1691.
[19]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[20]PENNINGTON J,SOCHER R,MANNING C.Glove:Globalvectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2014:1532-1543.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[3] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[4] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[5] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[9] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[10] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[11] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[12] TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo. Review of Text Classification Methods Based on Graph Convolutional Network [J]. Computer Science, 2022, 49(8): 205-216.
[13] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[14] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[15] YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: [email protected]
Powered by Beijing Magtech Co., Ltd.