Answering questions using knowledge graphs and sequence translation

We’re going to create a system that is able to take an English language question, convert it into Cypher using a neural network, then run that query against a Neo4j graph database to produce an answer

Article: https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2acbaa35a21d
Slide source: https://docs.google.com/presentation/d/15PoBPIBGpoV9MpNTrdmBG65SgR0k7Y93gc0_85c2PSs/edit?usp=sharing

Transcript

1. Answering questions using
   knowledge graphs and
   sequence translation
   December 6th 2018 - Metis, San Francisco
   David Mack
   david@octavian.ai
   https://octavian.ai @octavian_ai
   

   View Slide

2. David Mack
   https://octavian.ai
   david@octavian.ai
   

   View Slide

3. Vision: from document search...
   

   View Slide

4. Nope!
   Vision: … to knowledge graph reasoning
   

   View Slide

5. The system we’ll create:
   

   View Slide

6. Article: https://medium.com/octavian-ai/answering-english-questions-us
   Code: https://github.com/Octavian-ai/english2cypher
   Find all of this on https://octavian.ai
   

   View Slide

7. Knowledge graphs
   

   View Slide

8. Knowledge graphs
   

   View Slide

9. The dataset we’ll use: CLEVR-Graph
   Questions and
   answers on
   randomly generated
   transit networks.
   

   View Slide

10. Graph Question Cypher Answer
    How many stations are
    between Diz Bridge and
    Wisk Lane?
    MATCH (var1) MATCH (var2) MATCH tmp1 =
    shortestPath((var1)-[*]-(var2)) WHERE
    var1.name="Diz Bridge" AND var2.name="Wisk
    Lane" WITH 1 AS foo, tmp1 AS var3
    UNWIND nodes(var3) AS var4 RETURN
    length(collect(var4)) - 2
    6
    670,000 more examples...
    CLEVR-Graph data format
    A randomly generated synthetic dataset with 18 different types of questions.
    

    View Slide

11. Question answering
    system
    

    View Slide

12. Overview
    The complex bit!
    

    View Slide

13. How the translation works
    Whole model is 360 lines of tensorflow
    

    View Slide

14. Text must be turned into vectors...
    

    View Slide

15. Text must be turned into vectors...
    (Mapping learnt
    during training)
    (Fixed dictionary of 128
    tokens precomputed)
    

    View Slide

16. Combined word-character tokenization scheme
    significantly helped translation accuracy
    Where is Dalkey Hospital?
    [WHERE, IS, D, A, L, K, E, Y, HOSPITAL]
    Rare word
    

    View Slide

17. Encode a sequence then decode it
    Encoder Decoder
    English
    Cypher
    

    View Slide

18. Recurrent neural network - a way to
    process sequences in a neural network
    Background
    input
    Neural network
    and hidden state
    output
    Learn more about RNNs: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
    

    View Slide

19. Simple translation
    Read the TensorFlow tutorial: https://github.com/tensorflow/nmt
    Encode Encode
    (2 layer, 1024 unit
    LSTM cell in a
    bidirectional layout)
    (2 layer, 1024 unit
    LSTM cell in a
    bidirectional layout)
    3
    

    View Slide

20. Attention: let the network look back
    at the input
    D
    2
    D
    1
    D
    2
    D
    1
    hello
    world
    welt
    welt
    Attention
    Current With attention
    

    View Slide

21. Attentional Seq2Seq
    Image from Google. You can get the above technology as an a easy-to-use library by Google: https://google.github.io/seq2seq/
    

    View Slide

22. Training in progress - cypher output
    

    View Slide

23. Performance
    • Achieves >99.7%
    translation accuracy
    • Trained on 670,000
    Question-Cypher pairs
    • Works on 18 different
    types of questions
    Trained for 300 steps, 2 hours on a Tesla K80
    

    View Slide

24. Limitations
    These limitations are primarily from the limited nature of
    the training data
    Does not generalize well
    to other questions
    Does not generalize to
    English variations
    

    View Slide

25. Strategies for improving
    • Start with pre-trained sentence model (e.g.
    BERT) and transfer learn on top of it
    • Train on a broader range of Cypher queries
    

    View Slide

26. What’s the theoretical limit of this
    system?
    • It can perform any knowledge transformation that
    Neo4j is capable of
    • The limit is the sophistication of the recurrent cell
    • This system should be as powerful at reasoning as any
    other RNN approach
    • Language models are still relatively shallow and struggle
    with long text sequences (e.g. paragraph understanding)
    

    View Slide

27. and one more thing...
    

    View Slide

28. Please check it out
    and share!
    https://medium.com/octavian-ai/how-to-get-started-with-machine-learning-on-graphs-7f0795c83763
    

    View Slide

29. Q&A
    David Mack david@octavian.ai
    https://octavian.ai @octavian_ai
    

    View Slide

30. Appendix
    https://octavian.ai @octavian_ai David Mack david@octavian.ai
    

    View Slide

31. Read more about everything mentioned:
    • Online Safari book on RNNs:
    https://www.safaribooksonline.com/library/view/neural-networks-and/9781492037354/ch04.html
    • Introduction to RNN/LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
    • Attention and Augmented Recurrent Neural Networks https://distill.pub/2016/augmented-rnns/
    • TensorFlow Neural Machine Translation Tutorial https://github.com/tensorflow/nmt
    • Differentiable neural computers (DeepMind nature publication)
    https://deepmind.com/blog/differentiable-neural-computers/
    • MACnets https://arxiv.org/abs/1803.03067
    • IQA: Visual Question Answering in Interactive Environments (Reinforcement learning reasoning):
    https://arxiv.org/abs/1712.03316
    • English to Cypher translation
    https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2ac
    baa35a21d
    • Applying MACnet to knowledge graph (work in progress): https://github.com/Octavian-ai/mac-graph
    • Octavian’s research https://www.octavian.ai/articles https://twitter.com/Octavian_ai
    

    View Slide