Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Answering questions using knowledge graphs and sequence translation

David Mack
December 07, 2018

Answering questions using knowledge graphs and sequence translation

We’re going to create a system that is able to take an English language question, convert it into Cypher using a neural network, then run that query against a Neo4j graph database to produce an answer

Article: https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2acbaa35a21d
Slide source: https://docs.google.com/presentation/d/15PoBPIBGpoV9MpNTrdmBG65SgR0k7Y93gc0_85c2PSs/edit?usp=sharing

David Mack

December 07, 2018

Other Decks in Programming


  1. Answering questions using knowledge graphs and sequence translation December 6th

    2018 - Metis, San Francisco David Mack [email protected] https://octavian.ai @octavian_ai
  2. David Mack https://octavian.ai [email protected]

  3. Vision: from document search...

  4. Nope! Vision: … to knowledge graph reasoning

  5. The system we’ll create:

  6. Article: https://medium.com/octavian-ai/answering-english-questions-us Code: https://github.com/Octavian-ai/english2cypher Find all of this on https://octavian.ai

  7. Knowledge graphs

  8. Knowledge graphs

  9. The dataset we’ll use: CLEVR-Graph Questions and answers on randomly

    generated transit networks.
  10. Graph Question Cypher Answer How many stations are between Diz

    Bridge and Wisk Lane? MATCH (var1) MATCH (var2) MATCH tmp1 = shortestPath((var1)-[*]-(var2)) WHERE var1.name="Diz Bridge" AND var2.name="Wisk Lane" WITH 1 AS foo, tmp1 AS var3 UNWIND nodes(var3) AS var4 RETURN length(collect(var4)) - 2 6 670,000 more examples... CLEVR-Graph data format A randomly generated synthetic dataset with 18 different types of questions.
  11. Question answering system

  12. Overview The complex bit!

  13. How the translation works Whole model is 360 lines of

  14. Text must be turned into vectors...

  15. Text must be turned into vectors... (Mapping learnt during training)

    (Fixed dictionary of 128 tokens precomputed)
  16. Combined word-character tokenization scheme significantly helped translation accuracy Where is

    Dalkey Hospital? [WHERE, IS, D, A, L, K, E, Y, HOSPITAL] Rare word
  17. Encode a sequence then decode it Encoder Decoder English Cypher

  18. Recurrent neural network - a way to process sequences in

    a neural network Background input Neural network and hidden state output Learn more about RNNs: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  19. Simple translation Read the TensorFlow tutorial: https://github.com/tensorflow/nmt Encode Encode (2

    layer, 1024 unit LSTM cell in a bidirectional layout) (2 layer, 1024 unit LSTM cell in a bidirectional layout) 3
  20. Attention: let the network look back at the input D

    2 D 1 D 2 D 1 hello world welt welt Attention Current With attention
  21. Attentional Seq2Seq Image from Google. You can get the above

    technology as an a easy-to-use library by Google: https://google.github.io/seq2seq/
  22. Training in progress - cypher output

  23. Performance • Achieves >99.7% translation accuracy • Trained on 670,000

    Question-Cypher pairs • Works on 18 different types of questions Trained for 300 steps, 2 hours on a Tesla K80
  24. Limitations These limitations are primarily from the limited nature of

    the training data Does not generalize well to other questions Does not generalize to English variations
  25. Strategies for improving • Start with pre-trained sentence model (e.g.

    BERT) and transfer learn on top of it • Train on a broader range of Cypher queries
  26. What’s the theoretical limit of this system? • It can

    perform any knowledge transformation that Neo4j is capable of • The limit is the sophistication of the recurrent cell • This system should be as powerful at reasoning as any other RNN approach • Language models are still relatively shallow and struggle with long text sequences (e.g. paragraph understanding)
  27. and one more thing...

  28. Please check it out and share! https://medium.com/octavian-ai/how-to-get-started-with-machine-learning-on-graphs-7f0795c83763

  29. Q&A David Mack [email protected] https://octavian.ai @octavian_ai

  30. Appendix https://octavian.ai @octavian_ai David Mack [email protected]

  31. Read more about everything mentioned: • Online Safari book on

    RNNs: https://www.safaribooksonline.com/library/view/neural-networks-and/9781492037354/ch04.html • Introduction to RNN/LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ • Attention and Augmented Recurrent Neural Networks https://distill.pub/2016/augmented-rnns/ • TensorFlow Neural Machine Translation Tutorial https://github.com/tensorflow/nmt • Differentiable neural computers (DeepMind nature publication) https://deepmind.com/blog/differentiable-neural-computers/ • MACnets https://arxiv.org/abs/1803.03067 • IQA: Visual Question Answering in Interactive Environments (Reinforcement learning reasoning): https://arxiv.org/abs/1712.03316 • English to Cypher translation https://medium.com/octavian-ai/answering-english-questions-using-knowledge-graphs-and-sequence-translation-2ac baa35a21d • Applying MACnet to knowledge graph (work in progress): https://github.com/Octavian-ai/mac-graph • Octavian’s research https://www.octavian.ai/articles https://twitter.com/Octavian_ai