$30 off During Our Annual Pro Sale. View Details »

Data Lineage made easy with Graph Databases

Data Lineage made easy with Graph Databases

Data Lineage has always been a topic, at least for auditing, and came back as a key element with regulations like GDPR and similar. The problem is that with the multiplications of tools, sources, transformations and movements of data it's getting harder and harder to have a clear picture of the whole data lineage in a company and even more complicated to use that information for auditing. This is where graph databases jump in to make things easier: data lineage is by nature a graph. It's possible to model every single flow, every single component down to a column in a database or a dashboard. Add the whole corporate security on top with the various abstraction layers of groups and roles on top of users and your graph is ready for analysis. This talk will cover why graph databases are a perfect match for data lineage and use an analytical enterprise platform as example, tracking a single column from a database table to the very end into dashboards and reports and the respective security. (Based on Oracle Property Graph engine PGX with Cytoscape for visualization, and OAC/OBIEE for the data lineage example).

Gianni Ceresa

May 07, 2019

More Decks by Gianni Ceresa

Other Decks in Technology


  1. View Slide

  2. View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. 25 May 2018

    View Slide

  9. Only heard about it
    few times, for not
    much $. Did you?

    View Slide

  10. GDPR compliant

    View Slide

  11. View Slide

  12. View Slide

  13. Vertex
    Graph Database
    (also called node)

    View Slide

  14. edge
    edge label
    edge properties
    edge ID
    directed edge
    vertex (node)
    vertex ID
    a vertex can
    have a label

    View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. mapped to reference
    page contains
    Catalog ACL
    member of
    member of

    View Slide

  22. View Slide

  23. View Slide

  24. A shortcut is like a symbolic link: it
    makes something accessible with a
    different path.
    It is invisible to most users and
    OOTB auditing tools.

    View Slide

  25. An alias is an alternative way to access a data.
    It is invisible to most users and OOTB auditing tools.

    View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. • Coming soon...

    View Slide

  30. • Coming soon...


    View Slide

  31. View Slide

  32. View Slide

  33. From 45700 nodes
    with 105406 edges,
    to 85 nodes with
    218 edges in

    View Slide

  34. Coming soon…

    View Slide

  35. DEMO

    View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. View Slide