Towards Fast and Scalable Graph Pattern Mining

Transcript

1. Towards Fast and Scalable Graph Pattern Mining Anand Iyer ⋆,

   Zaoxing Liu ⬩, Xin Jin⬩, Shivaram Venkataraman▴, Vladimir Braverman⬩, Ion Stoica ⋆ ⋆ UC Berkeley ⬩ Johns Hopkins University ▴ Microsoft Research / University of Wisconsin HotCloud, July 09, 2018

2. Graphs popular in big data analytics

3. Graphs popular in big data analytics

4. Graphs popular in big data analytics

5. None

6. Graph Analytics

7. Graph Analytics Processing Algorithms

8. Graph Analytics Processing Algorithms PageRank Connected Components

9. Graph Analytics Processing Algorithms Mining Algorithms PageRank Connected Components

10. Graph Analytics Processing Algorithms Mining Algorithms PageRank Connected Components Motifs

    Cliques

11. Graph Analytics Processing Algorithms Mining Algorithms PageRank Connected Components Motifs

    Cliques

12. Graph Analytics: State-of-the-Art Processing Algorithms Mining Algorithms PageRank Connected Components

    Motifs Cliques Computes properties of the underlying graph Discovers structural patterns in the underlying graph

13. Graph Analytics: State-of-the-Art Processing Algorithms Mining Algorithms PageRank Connected Components

    Motifs Cliques § Easy to implement § Massively parallelizable § Can handle large graphs Computes properties of the underlying graph Discovers structural patterns in the underlying graph

14. Graph Analytics: State-of-the-Art Processing Algorithms Mining Algorithms PageRank Connected Components

    Motifs Cliques § Easy to implement § Massively parallelizable § Can handle large graphs § Efficient custom algorithms § Exponential intermediate data § Limited to small graphs Computes properties of the underlying graph Discovers structural patterns in the underlying graph

15. Graph Analytics: State-of-the-Art Processing Algorithms Mining Algorithms PageRank Connected Components

    Motifs Cliques § Easy to implement § Massively parallelizable § Can handle large graphs § Efficient custom algorithms § Exponential intermediate data § Limited to small graphs Challenging to mine patterns in large graphs Computes properties of the underlying graph Discovers structural patterns in the underlying graph

16. Graph Analytics: Processing vs Mining Log scale # Edges Computation

    Time

17. Graph Analytics: Processing vs Mining 1 trillion 140 s PageRank

    Log scale # Edges Computation Time

18. Graph Analytics: Processing vs Mining 1 trillion 140 s PageRank

    Log scale # Edges Computation Time

19. Graph Analytics: Processing vs Mining 1 trillion 140 s PageRank

    ~1 billion 11 hours Motifs with size = 3 Log scale # Edges Computation Time Arabesque (SOSP’15)

20. Graph Analytics: Processing vs Mining 1 trillion 140 s PageRank

    ~1 billion 11 hours Motifs with size = 3 Log scale # Edges Computation Time Can graph pattern mining be made both fast and scalable? Arabesque (SOSP’15)

21. Many mining tasks ask for the number of occurrences and

    do not need exact answers

22. Leverage approximation for graph pattern mining Many mining tasks ask

    for the number of occurrences and do not need exact answers

23. General approach: Apply algorithm on subset(s) (sample) of the input

    data Approximate Analytics

24. General approach: Apply algorithm on subset(s) (sample) of the input

    data Approximate Analytics 0 1 4 2 3 graph

25. General approach: Apply algorithm on subset(s) (sample) of the input

    data Approximate Analytics 0 1 4 2 3 edge sampling (p=0.5) graph 0 1 4 2 3

26. General approach: Apply algorithm on subset(s) (sample) of the input

    data Approximate Analytics 0 1 4 2 3 edge sampling (p=0.5) graph e = 1 0 1 4 2 3 triangle counting

27. General approach: Apply algorithm on subset(s) (sample) of the input

    data Approximate Analytics 0 1 4 2 3 edge sampling (p=0.5) graph e = 1 0 1 4 2 3 triangle counting result $ % 2 = 2

28. Answer: 10 General approach: Apply algorithm on subset(s) (sample) of

    the input data Approximate Analytics 0 1 4 2 3 edge sampling (p=0.5) graph e = 1 0 1 4 2 3 triangle counting result $ % 2 = 2

29. Answer: 10 General approach: Apply algorithm on subset(s) (sample) of

    the input data Approximate Analytics 0 1 4 2 3 edge sampling (p=0.5) graph e = 1 0 1 4 2 3 triangle counting result $ % 2 = 2 Applying exact algorithm on sampled graph(s) not the right approach for pattern mining

30. Approximation by Sampling Patterns 0 1 4 2 3 graph

    edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

31. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

32. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

33. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

34. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

35. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

36. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

37. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

38. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

39. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

40. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

41. E0 Approximation by Sampling Patterns 0 1 4 2 3

    graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

42. ! = 1 10 ∗ 1 4 E0 Approximation by

    Sampling Patterns 0 1 4 2 3 graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

43. ! = 1 10 ∗ 1 4 E0 Approximation by

    Sampling Patterns 0 1 4 2 3 graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) '( = 40 Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

44. ! = 1 10 ∗ 1 4 E0 Approximation by

    Sampling Patterns 0 1 4 2 3 neighborhood sampling graph edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) '( = 40 Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

45. ! = 1 10 ∗ 1 4 E0 Approximation by

    Sampling Patterns 0 1 4 2 3 neighborhood sampling graph E1 E2 E3 edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) '( = 40 Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

46. ! = 1 10 ∗ 1 4 E0 Approximation by

    Sampling Patterns 0 1 4 2 3 estimator (r=4) neighborhood sampling graph E1 E2 E3 edge stream: (0,1), (0,2), (0,3), (0,4), (1,2), (1,3), (1,4), (2,3), (2,4), (3,4) '( = 40 result 1 ) * +,( -./ '+ = 10 '/ = 0 '0 = 0 '1 = 0 Pavan et al. Counting and sampling triangles from a graph stream, VLDB 2013

47. Potential Benefits § 16 node Apache Spark cluster § Two

    graphs: Live Journal (68.9B), Twitter (1.47B) § Count 3-Motifs (2 patterns: triangle, 3-chain) § Set error to 5%

48. 3-Motif System Graph |V| |E| Time Ours (5%) 16 x

    8 Twitter 41.7M 1.47B 4m Arabesque 20x32 Instagram 180M 0.9B 10h45m Potential Benefits § 16 node Apache Spark cluster § Two graphs: Live Journal (68.9B), Twitter (1.47B) § Count 3-Motifs (2 patterns: triangle, 3-chain) § Set error to 5% 3-Motif System Graph |V| |E| Time Ours (5%) 16 x 8 LiveJ 4.8M 68.9B 11.5s Arabesque 16 x 8 LiveJ 41.7M 1.47B 299.2s

49. 3-Motif System Graph |V| |E| Time Ours (5%) 16 x

    8 Twitter 41.7M 1.47B 4m Arabesque 20x32 Instagram 180M 0.9B 10h45m Potential Benefits § 16 node Apache Spark cluster § Two graphs: Live Journal (68.9B), Twitter (1.47B) § Count 3-Motifs (2 patterns: triangle, 3-chain) § Set error to 5% 3-Motif System Graph |V| |E| Time Ours (5%) 16 x 8 LiveJ 4.8M 68.9B 11.5s Arabesque 16 x 8 LiveJ 41.7M 1.47B 299.2s

50. 3-Motif System Graph |V| |E| Time Ours (5%) 16 x

    8 Twitter 41.7M 1.47B 4m Arabesque 20x32 Instagram 180M 0.9B 10h45m Potential Benefits § 16 node Apache Spark cluster § Two graphs: Live Journal (68.9B), Twitter (1.47B) § Count 3-Motifs (2 patterns: triangle, 3-chain) § Set error to 5% 3-Motif System Graph |V| |E| Time Ours (5%) 16 x 8 LiveJ 4.8M 68.9B 11.5s Arabesque 16 x 8 LiveJ 41.7M 1.47B 299.2s

51. Challenges Building a General Purpose Approximate Graph Mining System

52. Challenges Building a General Purpose Approximate Graph Mining System General

    Patterns

53. Challenges Building a General Purpose Approximate Graph Mining System General

    Patterns Distributed Settings

54. Challenges Building a General Purpose Approximate Graph Mining System General

    Patterns Distributed Settings Error Estimation

55. Challenges Building a General Purpose Approximate Graph Mining System General

    Patterns Distributed Settings Error Estimation Handling Updates

56. Challenge #1: General Patterns Problem: Neighborhood sampling is for triangle

    counting Break down neighborhood sampling into two phases: • Sampling phase • Closing phase ! = 1 10 ∗ 1 4 E0 0 1 4 2 3 estimator (r=4) neighborhood sampling graph E1 E2 E3 '( = 40 result 1 ) * +,( -./ '+ = 10 '/ = 0 '0 = 0 '1 = 0

57. Challenge #1: General Patterns Problem: Neighborhood sampling is for triangle

    counting Break down neighborhood sampling into two phases: • Sampling phase • Closing phase ! = 1 10 ∗ 1 4 E0 0 1 4 2 3 estimator (r=4) neighborhood sampling graph E1 E2 E3 '( = 40 result 1 ) * +,( -./ '+ = 10 '/ = 0 '0 = 0 '1 = 0 Can we restrict the implementation using a simple API ? How can we analyze programs written using the API?

58. Challenge #2: Distributed Setting Problem: Neighborhood sampling is for a

    single machine

59. Challenge #2: Distributed Setting graph Problem: Neighborhood sampling is for

    a single machine

60. Challenge #2: Distributed Setting graph subgraph 0 partial count c

    0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) Problem: Neighborhood sampling is for a single machine

61. Challenge #2: Distributed Setting graph map: w(=3) workers subgraph 0

    partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) Problem: Neighborhood sampling is for a single machine

62. Challenge #2: Distributed Setting graph map: w(=3) workers subgraph 0

    partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) Problem: Neighborhood sampling is for a single machine

63. Challenge #2: Distributed Setting graph ! "#$ %&' (" map:

    w(=3) workers subgraph 0 partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) Problem: Neighborhood sampling is for a single machine

64. Challenge #2: Distributed Setting graph ! "#$ %&' (" map:

    w(=3) workers reduce subgraph 0 partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) Problem: Neighborhood sampling is for a single machine

65. Challenge #2: Distributed Setting graph ! "#$ %&' (" map:

    w(=3) workers reduce subgraph 0 partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) )(+) Problem: Neighborhood sampling is for a single machine

66. Challenge #2: Distributed Setting graph ! "#$ %&' (" map:

    w(=3) workers reduce subgraph 0 partial count c 0 (using r estimators) subgraph 1 partial count c 1 (using r estimators) subgraph 2 partial count c 2 (using r estimators) )(+) Problem: Neighborhood sampling is for a single machine How do we compute f(w) for any pattern? How does f(w) affect error?

67. Challenge #3: Building Error-Latency Profile Problem: Given a time /

    error bound, how many estimators should we use? Need to build two profiles: • Time vs #estimators • Error vs #estimators Naïve approach: • Exhaustively run every possible point (infeasible)

68. Building Estimators vs Time Profile 1 2 3 0.5M 1M

    1.5M 2M Runtime (min) No. of Estimators Twitter Graph Time complexity linear in number of estimators

69. Building Estimators vs Error Profile 0 5 10 15 20

    25 30 35 40 50k 1m 1.5m 2.1m Error Rate (%) No. of Estimators Twitter Graph Error complexity non-linear in number of estimators

70. Building Estimators vs Error Profile 0 5 10 15 20

    25 30 35 40 50k 1m 1.5m 2.1m Error Rate (%) No. of Estimators Twitter Graph Error complexity non-linear in number of estimators Leverage techniques like experiment design/Bayesian optimization? How do we avoid the need to know the ground truth?

71. Challenge #4: Updates Problem: Graphs and queries can be updated/refined

    Several systems challenges: • Incremental pattern mining • Can the error-latency profiles be updated? • Caching • Re-use results • Pre-computation

72. Conclusion § Approximation is a promising solution for pattern mining

    § Significant benefits, and can handle much larger graphs… § … but cannot output all instances of the pattern § Several challenges in realizing it § How to extend the technique to general patterns? § How to do approximate pattern mining in a distributed setting? § How do we estimate the error? § How do we handle updates? http://www.cs.berkeley.edu/~api api@cs.berkeley.edu