Fork me on GitHub


This document provides a quick summary ofthe current efforts made towards bridging relational learning and representation or deep learning. It is made with three objectives in mind:

Contact

Acknowledgments

Getting started

Interesting venues





Vectorization approaches

Factorization approaches

Modelling Relational Data using Bayesian Clustered Tensor Factorization
Ilya Sutskever, Joshua B. Tenenbaum, Ruslan R. Salakhutdinov
NIPS 2009
Paper
Code

A Three-Way Model for Collective Learning on Multi-Relational Data
Maximilian Nickel, Volker Tresp, Hans-Peter Kriegel
ICML 2011
Paper
Code

Factorizing YAGO: Scalable Machine Learning for Linked Data
Maximilian Nickel, Volker Tresp, Hans-Peter Kriegel
WWW 2011
Paper
Code

Neural embeddings

Learning Structured Embeddings of Knowledge Bases
Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio
AAAI 2011
Paper
Code


A latent factor model for highly multi-relational data
Rodolphe Jenatton, Nicolas L Roux, Antoine Bordes, Guillaume R Obozinski
NIPS 2012
Paper
Code

A Semantic Matching Energy Function for Learning with Multi-relational Data
Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio
Machine Learning 2013: Special Issue on Learning Semantics
Paper
Code

Translating Embeddings for Modeling Multi-relational Data
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko
NIPS 2013
Paper
Code

Reasoning With Neural Tensor Networks for Knowledge Base Completion
Richard Socher, Danqi Chen, Christopher D. Manning, Andrew Ng
- 2013
Paper
Code

Multi-relational Learning Using Weighted Tensor Decomposition with Modular Loss
Ben London, Theodoros Rekatsinas, Bert Huang, Lise Getoor
NIPS 2013
Paper
Code

Low-Dimensional Embeddings of Logic
Tim Rocktaschel, Matko Bosnjak, Sameer Singh, Sebastian Riedel
ACL Workshop on Semantic Parsin 2014
Paper
Code

Learning Entity and Relation Embeddings for Knowledge Graph Completion
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu
AAAI 2015
Paper
Code

Compositional Vector Space Models for Knowledge Base Completion
Arvind Neelakantan, Benjamin Roth, Andrew McCallum
ACL 2015
Paper
Code

Embedding Entities and Relations for Learning and Inference in Knowledge Bases
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng
ICLR 2015
Paper
Code

Modeling Relation Paths for Representation Learning of Knowledge Bases
Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, Song Liu
EMNLP 2015
Paper
Code

Composing Relationships with Translations
Alberto García-Durán, Antoine Bordes, Nicolas Usunier
EMNLP 2015
Paper
Code

Traversing Knowledge Graphs in Vector Space
Kelvin Guu, John Miller, Percy Liang
EMNLP 2015
Paper
Code

Representation Learning of Knowledge Graphs with Entity Descriptions
Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, Maosong Sun
AAAI 2016
Paper
Code

Combining two and three-way embedding models for link prediction in knowledge bases
Alberto García-Durán, Antoine Bordes, Nicolas Usunier, Yves Grandvalet
JAIR 2016
Paper
Code

Complex Embeddings for Simple Link Prediction
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard
ICML 2016
Paper
Code

Efficient energy-based embedding models for link prediction in knowledge graphs
Pasquale Minervini, Claudia d’Amato, Nicola Fanizzi
Journal of Intelligent Information Systems 2016
Paper
Code

Representation Learning of Knowledge Graphs with Hierarchical Types
Ruobing Xie, Zhiyuan Liu, Maosong Sun
IJCAI 2016
Paper
Code

Holographic Embeddings Knowledge Graphs
Maximilian Nickel, Lorenzo Rosasco, Tomaso Poggio
AAAI 2016
Paper
Code

STransE: a novel embedding model of entities and relationships in knowledge bases
Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu and Mark Johnson
NAACL-HLT 2016
Paper
Code

Compositional Learning of Embeddings for Relation Paths in Knowledge Bases and Text
Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, Chris Quirk
ACL 2016
Paper
Code

TransG: A Generative Mixture Model for Knowledge Graph Embedding
Han Xiao, Minlie Huang, Xiaoyan Zhu
ACL 2016
Paper
Code

Deductive and Analogical Reasoning on a Semantically Embedded Knowledge Graph
Douglas Summers-Stay
AGI 2017
Paper
Code

Inductive Representation Learning on Large Graphs
William L. Hamilton, Rex Ying, Jure Leskovec
NIPS 2017
Paper
Code

Distributed representation learning for knowledge graphs with entity descriptions
Miao Fan, Qiang Zhou, Thomas Fang Zheng, Ralph Grishman
Pattern Recognition Letters 2017
Paper
Code

Analogical Inference for Multi-Relational Embeddings
Hanxiao Liu, Yuexin Wu, Yiming Yang
ICML 2017
Paper
Code

Learning Continuous Semantic Representations of Symbolic Expressions
Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, Charles Sutton
ICML 2017
Paper
Code

An Interpretable Knowledge Transfer Model for Knowledge Base Completion
Qizhe Xie, Xuezhe Ma, Zihang Dai, Eduard Hovy
ACL 2017
Paper
Code

Learning Graph Representations with Embedding Propagation
Alberto Garcia-Duran, Mathias Niepert
NIPS 2017
Paper
Code

2D Convolutional Graph Embeddings
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel
-
Paper
Code

Learning Knowledge Graph Embeddings with Type Regularizer
Bhushan Kotnis, Vivi Nastase
-
Paper
Code

Poincare Embeddings for Learning Hierarchical Representations
Maximilian Nickel, Douwe Kiela
-
Paper
Code

Deep Gaussian Embedding of Attributed Graphs: Unsupervised Inductive Learning via Ranking
Aleksandar Bojchevski, Stephan Günnemann
-
Paper
Code

HARP: Hierarchical Representation Learning for Networks
Haochen Chen, Bryan Perozzi, Yifan Hu, Steven Skiena
-
Paper
Code

Modeling Relational Data with Graph Convolutional Networks
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, Max Welling
-
Paper
Code

Deep Generative Models for Relational Data with Side Information
Changwei Hu, Piyush Rai, Lawrence Carin
-
Paper
Code

Regularizing embeddings

Injecting Logical Background Knowledge into Embeddings for Relation Extraction
Tim Rocktaschel, Sameer Singh, Sebastian Riedel
NAACL 2015
Paper
Code

Knowledge base completion using embeddings and rules
Quan Wang, Bin Wang, Li Guo
IJCAI 2015
Paper
Code

Entity embeddings with conceptual subspaces as a basis for plausible reasoning
Shoaib Jameel, Steven Schockaert
ECAI 2016
Paper
Code

Regularizing Knowledge Graph Embeddings via Equivalence and Inversion Axioms
Pasquale Minervini, Luca Costabello, Emir Muñoz, Vít Nováček, Pierre-Yves Vandenbussche
ECML PKDD 2017
Paper
Code

Adversarial Sets for Regularising Neural Link Predictors
Pasquale Minervini, Thomas Demeester, Tim Rocktäschel, Sebastian Riedel
UAI 2017
Paper
Code

Learning Knowledge Graph Embeddings with Type Regularizer
Bhushan Kotnis, Vivi Nastase
-
Paper
Code

Relational approaches

Change of representation for statistical relational learning
Jesse Davis, Irene Ong, Jan Struyf, Elizabeth Burnside, David Page, Vıtor Santos Costa
IJCAI'07
Paper
Code

Introducing DRAIL: a Step Towards Declarative Deep Relational Learning
Xiao Zhang, Maria Leonor Pacheco, Chang Li, Dan Goldwasser
Structured Prediction for NLP 2016
Paper
Code

A Clustering-based Relational Representation Learning with an Explicit Distributed Representation
Sebastijan Dumancic, Hendrik Blockeel
IJCAI 2017
Paper
Code

Demystifying Relational Latent Representations
Sebastijan Dumancic, Hendrik Blockeel
ILP 2017
Paper
Code

Hybrid approaches

Feeding relational features to neural networks

Deep relational machines
Huma Lodhi
ICONIP 2013
Paper
Code

Discriminative Gaifman models
Mathias Niepert
NIPS 2016
Paper
Code

Neuro-symbolic EDA-based Optimisation using ILP-enhanced DBNs
Sarmimala Saikia, Lovekesh Vig, Ashwin Srinivasan, Gautam Shroff, Puneet Agarwal, Richa Rawat
Cognitive Computing, NIPS 2016
Paper
Code

Relational Restricted Boltzmann Machines: A Probabilistic Logic Learning Approach
Navdeep Kaur, Gautam Kunapuli, Tushar Khot, Kristian Kersting, William Cohen and Sriraam Natarajan
ILP 2017
Paper
Code

Templating neural networks

Lifted Relational Neural Networks
Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, Ondrej Kuzelka
Cognitive Computation, NIPS 2015
Paper
Code

Stacked Structure Learning for Lifted Relational Neural Networks
Gustav Šourek, Martin Svatoš, Filip Železný, Steven Schockaert and Ondřej Kuželka
ILP 2017
Paper
Code

End-to-end Differentiable Proving
Tim Rocktäschel, Sebastian Riedel
NIPS 2017
Paper
Code

Other hybrid approaches

Interaction Networks for Learning about Objects, Relations and Physics
Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, Koray Kavukcuoglu
NIPS 2016
Paper
Code

Column Networks for Collective Classification
Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh
AAAI 2017
Paper
Code

Deep Dynamic Relational Classifiers
Hogun Park, John Moore and Jennifer Neville
MAISoN- WSDM Workshop 2017
Paper
Code

Deep Collective Inference
John Moore, Jennifer Neville
AAAI 2017
Paper
Code

Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks.
Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum
EACL 2017
Paper
Code

Variation Autoencoder Based Network Representation Learning for Classification
Hang Li, Haozheng Wang, Zhenglu Yang, Masato Odagaki
ACL 2017
Paper
Code

A Compositional Object-Based Approach to Learning Physical Dynamics
Michael B. Chang, Tomer Ullman, Antonio Torralba, Joshua B. Tenenbaum
ICLR 2017
Paper
Code

Deep Learning for Ontology Reasoning
Patrick Hohenecker, Thomas Lukasiewicz
-
Paper
Code

A simple neural network module for relational reasoning
Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap
-
Paper
Code

Related approaches

TensorLog: A Differentiable Deductive Database
William W. Cohen
-
Paper
Code

Differentiable Learning of Logical Rules for Knowledge Base Completion
Fan Yang, Zhilin Yang, William W. Cohen
-
Paper
Code

SLDR-DL: A Framework for SLD-Resolution with Deep Learning
Cheng-Hao Cai
-
Paper
Code