Skip to main content
SpringerLink
Log in

Path-based reasoning with constrained type attention for knowledge graph completion

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Multi-hop reasoning over paths in knowledge graphs has attracted rising research interest in the field of knowledge graph completion. Entity types and relation types both contain various kinds of information content though only a subset of them are helpful in the specific triples. Although significant progress has been made by existing models, they have two major shortcomings. First, these models seldom learn an explicit representation of entities and relations with semantic information. Second, they reason without discriminating distinct role types that the same entity with multiple types plays in different triples. To address these issues, we develop a novel path-based reasoning with constrained type attention model, which tries to identify entity types by leveraging relation type constraints in the corresponding triples. Our experimental evaluation shows that the proposed model outperforms the state of the art on a real-world dataset. Further analyses also confirm that both word-level and triple-level attention mechanisms of our model are effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: Proceedings of the 16th international conference on World Wide Web. ACM, pp 697–706

  2. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, pp 1247–1250

  3. Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka Jr ER, Mitchell TM (2010) Toward an architecture for never-ending language learning. In: AAAI, vol 5, p 3

  4. Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Bizer C (2015) DBpediaa large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 6(2):167–195

    Google Scholar 

  5. Bordes A, Weston J, Usunier N (2014) Open question answering with weakly supervised embedding models. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, Heidelberg, pp 165–180

    Google Scholar 

  6. Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. arXiv preprint arXiv:1406.3676

  7. Zhang F, Yuan NJ, Lian D, Xie X, Ma WY (2016) Collaborative knowledge base embedding for recommender systems. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 353–362

  8. Catherine R, Cohen W (2016) Personalized recommendations using knowledge graphs: a probabilistic logic programming approach. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 325–332

  9. Lao N, Mitchell T, Cohen WW (2011) Random walk inference and learning in a large scale knowledge base. In: Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 529–539

  10. Wen D, Liu Y, Yuan K, Si S, Shen Y (2017) Attention-aware path-based relation extraction for medical knowledge graph. In: International conference on smart computing and communication. Springer, Cham, pp 321–331

    Google Scholar 

  11. Neelakantan A, Roth B, Mc-Callum A (2015) Compositional vector space models for knowledge base inference. In: 2015 AAAI spring symposium series

  12. Das R, Neelakantan A, Belanger D, McCallum A (2017) Chains of reasoning over entities, relations, and text using recurrent neural networks. In: Proceedings of the 15th conference of the European Chapter of the Association for Computational Linguistics: volume 1, Long Papers, p 141

  13. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795

  14. Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), pp 687–696

  15. Ji G, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: AAAI, pp 985–991

  16. Zelenko D, Aone C, Richardella A (2003) Kernel methods for relation extraction. J Mach Learn Res 3:1083–1106

    MathSciNet  MATH  Google Scholar 

  17. Gormley MR, Yu M, Dredze M (2015) Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1774–1784

  18. Socher R, Huval B, Manning C D, Ng A Y (2012, July) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. Association for Computational Linguistics, pp 1201–1211

  19. Angeli G, Manning C (2013) Philosophers are mortal: Inferring the truth of unseen facts. In: Proceedings of the seventeenth conference on computational natural language learning, pp 133–142

  20. Lao N, Cohen WW (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67

    MathSciNet  Google Scholar 

  21. Socher R, Chen D, Manning C D, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp 926–934

  22. Gardner M, Talukdar P, Krishnamurthy J, Mitchell T (2014) Incorporating vector space similarity in random walk inference over knowledge bases. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 397–406

  23. Gardner M, Talukdar P P, Kisiel B, Mitchell T (2013) Improving learning and inference in a large knowledge-base using latent syntactic cues. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp 833-838

  24. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1499–1509

  25. Jiang X, Wang Q, Qi B, Qiu Y, Li P, Wang B (2017) Attentive path combination for knowledge graph completion. In: Asian conference on machine learning, pp 590–605

  26. Chen W, Xiong W, Yan X, Wang WY (2018) Variational knowledge graph reasoning. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (Long Papers), pp 1823–1832

  27. Shen Y, Deng Y, Yang M, Li Y, Du N, Fan W, Lei K (2018) Knowledge-aware attentive neural network for ranking question answer pairs. In: The 41st international ACM SIGIR conference on research development in information retrieval. ACM, pp 901–904

  28. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: Proceedings of international conference on learning representations

  29. Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pp 207–212

  30. Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pp 2124–2133

  31. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1480–1489

  32. Zhang H, Li J, Ji Y, Yue H (2017) Understanding subtitles by character-level sequence-to-sequence learning. IEEE Trans Ind Inf 13(2):616–624

    Google Scholar 

  33. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Google Scholar 

  34. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256

  35. Kingma D P, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

Download references

Acknowledgements

This work was financially supported by the National Natural Science Foundation of China (No. 61602013), Natural Science Foundation of Guangdong (No. 2018A030313017) and the Shenzhen Fundamental Research Project (No. JCYJ20170818091546869).

Author information

Authors and Affiliations

  1. Shenzhen Key Lab for Information Centric Networking and Blockchain Technology (ICNLAB), School of Electronics and Computer Engineering (SECE), Peking University, Shenzhen, 518055, People’s Republic of China

    Kai Lei, Jin Zhang, Yuexiang Xie, Desi Wen, Daoyuan Chen & Ying Shen

  2. Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, People’s Republic of China

    Min Yang

  3. PCL Research Center of Networks and Communications, Peng Cheng Laboratory, Shenzhen, 518055, People’s Republic of China

    Kai Lei

Authors
  1. Kai Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuexiang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Desi Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daoyuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Min Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ying Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Shen.

Ethics declarations

Conflict of interest

The authors declare that there is no actual or potential conflict of interest in relation to this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, K., Zhang, J., Xie, Y. et al. Path-based reasoning with constrained type attention for knowledge graph completion. Neural Comput & Applic 32, 6957–6966 (2020). https://doi.org/10.1007/s00521-019-04181-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04181-1

Keywords

Search

Navigation