Creating a PageRank Analytics Platform Using Spring Boot Microservices

These slides introduce you to a sample application that combines multiple microservices with a graph processing platform to rank communities of users on Twitter. The problem we’re going to solve is how to discover communities of influencers on Twitter using a set of seed profiles as inputs. To solve this problem without a background in machine learning or social network analytics might be a bit of a stretch, but we’re going to take a stab at it using a little bit of computer science history. We’re going to use a collection of popular tools as a part of this article’s sample application. The tools we’ll use, in the order of importance, will be: Spring Boot, Neo4j, Apache Spark, Docker, RabbitMQ.

Transcript

1. Unless otherwise indicated, these slides are © 2016 Pivotal Software, Inc. and licensed under a
   
   Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
   Creating a PageRank Analytics Platform Using
   Spring Boot Microservices
   Kenny Bastani, Spring Developer Advocate, Pivotal
   @kennybastani
   

   View Slide

2. https://github.com/kbastani/spring-boot-graph-processing-example
   Speaker Intro - Kenny Bastani
   2
   

   View Slide

3. https://github.com/kbastani/spring-boot-graph-processing-example
   Tools we’ll be using
   ! Spring Boot
   ! Neo4j
   ! Apache Spark
   ! Docker
   ! RabbitMQ
   3
   

   View Slide

4. https://github.com/kbastani/spring-boot-graph-processing-example
   4
   Agenda
   ! Sample Application Overview
   • Ranking Twitter profiles
   • PageRank algorithm
   ! Graph Processing Platform
   • Apache Spark GraphX
   ! Building Microservices
   • Reference architecture
   • Spring Data Neo4j repositories
   • Importing Twitter users
   • Scheduling PageRank jobs on the Twitter graph
   

   View Slide

5. Ranking Twitter Profiles
   Using PageRank
   5 https://github.com/kbastani/spring-boot-graph-processing-example
   

   View Slide

6. https://github.com/kbastani/spring-boot-graph-processing-example
   PageRank algorithm
   6
   

   View Slide

7. https://github.com/kbastani/spring-boot-graph-processing-example
   PageRank algorithm
   7
   

   View Slide

8. https://github.com/kbastani/spring-boot-graph-processing-example
   Ranking Dashboard
   8
   

   View Slide

9. https://github.com/kbastani/spring-boot-graph-processing-example
   Graph processing platform
   9
   

   View Slide

10. Building Microservices
    10 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

11. https://github.com/kbastani/spring-boot-graph-processing-example
    Building Microservices
    11
    

    View Slide

12. 12 https://github.com/kbastani/spring-boot-graph-processing-example
    Building Microservices
    

    View Slide

13. 13
    Creating Spring Data Neo4j Repositories
    https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

14. Creating Spring Data Neo4j Repositories
    14 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

15. Creating Spring Data Neo4j Repositories
    15 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

16. Creating Spring Data Neo4j Repositories
    16 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

17. Creating Spring Data Neo4j Repositories
    17 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

18. Creating Spring Data Neo4j Repositories
    18 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

19. Creating Spring Data Neo4j Repositories
    19 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

20. Exposing repository APIs using Spring Data REST
    20 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

21. Exposing repository APIs using Spring Data REST
    21 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

22. Exposing repository APIs using Spring Data REST
    22 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

23. Connecting to the Twitter API
    23 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

24. 24 https://github.com/kbastani/spring-boot-graph-processing-example
    Connecting to the Twitter API
    

    View Slide

25. Connecting to the Twitter API
    25 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

26. Connecting to the Twitter API
    26 https://github.com/kbastani/spring-boot-graph-processing-example
    We can override these properties as environment variables at runtime
    

    View Slide

27. 27 https://github.com/kbastani/spring-boot-graph-processing-example
    Connecting to the Twitter API
    

    View Slide

28. 28 https://github.com/kbastani/spring-boot-graph-processing-example
    Connecting to the Twitter API
    

    View Slide

29. Scheduling new PageRank jobs
    29 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

30. https://github.com/kbastani/spring-boot-graph-processing-example
    Graph processing platform
    30
    

    View Slide

31. Scheduling new PageRank jobs
    31 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

32. Ranking Dashboard
    32 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

33. 33 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

34. Ranking dashboard
    34 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

35. 35 https://github.com/kbastani/spring-boot-graph-processing-example
    Ranking dashboard
    

    View Slide

36. Adding static web content
    36 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

37. 37 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

38. 38 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

39. https://github.com/kbastani/spring-boot-graph-processing-example
    Ranking dashboard
    39
    

    View Slide

40. https://github.com/kbastani/spring-boot-graph-processing-example
    Add seed profiles
    40
    

    View Slide

41. https://github.com/kbastani/spring-boot-graph-processing-example
    Choose 3 seed profiles
    41
    

    View Slide

42. Creating Spring Data Neo4j Repositories
    42 https://github.com/kbastani/spring-boot-graph-processing-example
    

    View Slide

43. https://github.com/kbastani/spring-boot-graph-processing-example
    Discover new users and update rankings
    43
    

    View Slide

44. https://github.com/kbastani/spring-boot-graph-processing-example
    44
    Learn More. Stay Connected.
    ! @kennybastani on Twitter Twitter: twitter.com/springcentral
    YouTube: spring.io/video
    LinkedIn: spring.io/linkedin
    Google Plus: spring.io/gplus
    

    View Slide