Answering questions using knowledge graphs and sequence translation

We’re going to create a system that is able to take an English language question, convert it into Cypher using a neural network, then run that query against a Neo4j graph database to produce an answer

Article: https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2acbaa35a21d
Slide source: https://docs.google.com/presentation/d/15PoBPIBGpoV9MpNTrdmBG65SgR0k7Y93gc0_85c2PSs/edit?usp=sharing

Transcript

1. Answering questions using knowledge graphs and sequence translation December 6th 2018 - Metis, San Francisco David Mack david@octavian.ai https://octavian.ai @octavian_ai
2. David Mack https://octavian.ai david@octavian.ai
3. Vision: from document search...
4. Nope! Vision: … to knowledge graph reasoning
5. The system we’ll create:
6. Article: https://medium.com/octavian-ai/answering-english-questions-us Code: https://github.com/Octavian-ai/english2cypher Find all of this on https://octavian.ai
7. Knowledge graphs
8. Knowledge graphs
9. The dataset we’ll use: CLEVR-Graph Questions and answers on randomly generated transit networks.
10. Graph Question Cypher Answer How many stations are between Diz Bridge and Wisk Lane? MATCH (var1) MATCH (var2) MATCH tmp1 = shortestPath((var1)-[*]-(var2)) WHERE var1.name="Diz Bridge" AND var2.name="Wisk Lane" WITH 1 AS foo, tmp1 AS var3 UNWIND nodes(var3) AS var4 RETURN length(collect(var4)) - 2 6 670,000 more examples... CLEVR-Graph data format A randomly generated synthetic dataset with 18 different types of questions.
11. Question answering system
12. Overview The complex bit!
13. How the translation works Whole model is 360 lines of tensorflow
14. Text must be turned into vectors...
15. Text must be turned into vectors... (Mapping learnt during training) (Fixed dictionary of 128 tokens precomputed)
16. Combined word-character tokenization scheme significantly helped translation accuracy Where is Dalkey Hospital? [WHERE, IS, D, A, L, K, E, Y, HOSPITAL] Rare word
17. Encode a sequence then decode it Encoder Decoder English Cypher
18. Recurrent neural network - a way to process sequences in a neural network Background input Neural network and hidden state output Learn more about RNNs: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
19. Simple translation Read the TensorFlow tutorial: https://github.com/tensorflow/nmt Encode Encode (2 layer, 1024 unit LSTM cell in a bidirectional layout) (2 layer, 1024 unit LSTM cell in a bidirectional layout) 3
20. Attention: let the network look back at the input D 2 D 1 D 2 D 1 hello world welt welt Attention Current With attention
21. Attentional Seq2Seq Image from Google. You can get the above technology as an a easy-to-use library by Google: https://google.github.io/seq2seq/
22. Training in progress - cypher output
23. Performance • Achieves >99.7% translation accuracy • Trained on 670,000 Question-Cypher pairs • Works on 18 different types of questions Trained for 300 steps, 2 hours on a Tesla K80
24. Limitations These limitations are primarily from the limited nature of the training data Does not generalize well to other questions Does not generalize to English variations
25. Strategies for improving • Start with pre-trained sentence model (e.g. BERT) and transfer learn on top of it • Train on a broader range of Cypher queries
26. What’s the theoretical limit of this system? • It can perform any knowledge transformation that Neo4j is capable of • The limit is the sophistication of the recurrent cell • This system should be as powerful at reasoning as any other RNN approach • Language models are still relatively shallow and struggle with long text sequences (e.g. paragraph understanding)
27. and one more thing...
28. Please check it out and share! https://medium.com/octavian-ai/how-to-get-started-with-machine-learning-on-graphs-7f0795c83763
29. Q&A David Mack david@octavian.ai https://octavian.ai @octavian_ai
30. Appendix https://octavian.ai @octavian_ai David Mack david@octavian.ai
31. Read more about everything mentioned: • Online Safari book on RNNs: https://www.safaribooksonline.com/library/view/neural-networks-and/9781492037354/ch04.html • Introduction to RNN/LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ • Attention and Augmented Recurrent Neural Networks https://distill.pub/2016/augmented-rnns/ • TensorFlow Neural Machine Translation Tutorial https://github.com/tensorflow/nmt • Differentiable neural computers (DeepMind nature publication) https://deepmind.com/blog/differentiable-neural-computers/ • MACnets https://arxiv.org/abs/1803.03067 • IQA: Visual Question Answering in Interactive Environments (Reinforcement learning reasoning): https://arxiv.org/abs/1712.03316 • English to Cypher translation https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2ac baa35a21d • Applying MACnet to knowledge graph (work in progress): https://github.com/Octavian-ai/mac-graph • Octavian’s research https://www.octavian.ai/articles https://twitter.com/Octavian_ai