Answering questions using knowledge graphs and sequence translation

E160f32721359f446398b9e24b70c0db?s=47 David Mack
December 07, 2018

Answering questions using knowledge graphs and sequence translation

We’re going to create a system that is able to take an English language question, convert it into Cypher using a neural network, then run that query against a Neo4j graph database to produce an answer

Slide source:


David Mack

December 07, 2018


  1. Answering questions using knowledge graphs and sequence translation December 6th

    2018 - Metis, San Francisco David Mack @octavian_ai
  2. David Mack

  3. Vision: from document search...

  4. Nope! Vision: … to knowledge graph reasoning

  5. The system we’ll create:

  6. Article: Code: Find all of this on

  7. Knowledge graphs

  8. Knowledge graphs

  9. The dataset we’ll use: CLEVR-Graph Questions and answers on randomly

    generated transit networks.
  10. Graph Question Cypher Answer How many stations are between Diz

    Bridge and Wisk Lane? MATCH (var1) MATCH (var2) MATCH tmp1 = shortestPath((var1)-[*]-(var2)) WHERE"Diz Bridge" AND"Wisk Lane" WITH 1 AS foo, tmp1 AS var3 UNWIND nodes(var3) AS var4 RETURN length(collect(var4)) - 2 6 670,000 more examples... CLEVR-Graph data format A randomly generated synthetic dataset with 18 different types of questions.
  11. Question answering system

  12. Overview The complex bit!

  13. How the translation works Whole model is 360 lines of

  14. Text must be turned into vectors...

  15. Text must be turned into vectors... (Mapping learnt during training)

    (Fixed dictionary of 128 tokens precomputed)
  16. Combined word-character tokenization scheme significantly helped translation accuracy Where is

    Dalkey Hospital? [WHERE, IS, D, A, L, K, E, Y, HOSPITAL] Rare word
  17. Encode a sequence then decode it Encoder Decoder English Cypher

  18. Recurrent neural network - a way to process sequences in

    a neural network Background input Neural network and hidden state output Learn more about RNNs:
  19. Simple translation Read the TensorFlow tutorial: Encode Encode (2

    layer, 1024 unit LSTM cell in a bidirectional layout) (2 layer, 1024 unit LSTM cell in a bidirectional layout) 3
  20. Attention: let the network look back at the input D

    2 D 1 D 2 D 1 hello world welt welt Attention Current With attention
  21. Attentional Seq2Seq Image from Google. You can get the above

    technology as an a easy-to-use library by Google:
  22. Training in progress - cypher output

  23. Performance • Achieves >99.7% translation accuracy • Trained on 670,000

    Question-Cypher pairs • Works on 18 different types of questions Trained for 300 steps, 2 hours on a Tesla K80
  24. Limitations These limitations are primarily from the limited nature of

    the training data Does not generalize well to other questions Does not generalize to English variations
  25. Strategies for improving • Start with pre-trained sentence model (e.g.

    BERT) and transfer learn on top of it • Train on a broader range of Cypher queries
  26. What’s the theoretical limit of this system? • It can

    perform any knowledge transformation that Neo4j is capable of • The limit is the sophistication of the recurrent cell • This system should be as powerful at reasoning as any other RNN approach • Language models are still relatively shallow and struggle with long text sequences (e.g. paragraph understanding)
  27. and one more thing...

  28. Please check it out and share!

  29. Q&A David Mack @octavian_ai

  30. Appendix @octavian_ai David Mack

  31. Read more about everything mentioned: • Online Safari book on

    RNNs: • Introduction to RNN/LSTM: • Attention and Augmented Recurrent Neural Networks • TensorFlow Neural Machine Translation Tutorial • Differentiable neural computers (DeepMind nature publication) • MACnets • IQA: Visual Question Answering in Interactive Environments (Reinforcement learning reasoning): • English to Cypher translation baa35a21d • Applying MACnet to knowledge graph (work in progress): • Octavian’s research