Swarrot, a library to consume them all!

Talk given at BlaBlaCar Tech Meetup in Warsaw

Transcript

1. View Slide

2. @odolbeau
   Web Architect
   Olivier Dolbeau
   

   View Slide

3. View Slide

4. What happens when you
   publish a trip?
   

   View Slide

5. We send an email to
   the driver.
   We index the trip in
   ElasticSearch.
   We send the comment
   to moderation spooler.
   We store all relevant
   informations in Hadoop for BI.
   

   View Slide

6. I’m gonna sleep, tell me
   when you’re done!
   

   View Slide

7. View Slide

8. View Slide

9. “RabbitMQ is a message
   broker. The principal idea
   is pretty simple: it accepts
   and forwards messages.”
   

   View Slide

10. Publisher
    Queue1
    Exchange Queue2
    Queue3
    Consumer
    Consumer
    Consumer
    Consumer
    RabbitMQ
    

    View Slide

11. What happens when you
    publish a trip?
    

    View Slide

12. We send an email to
    the driver.
    We index the trip in
    ElasticSearch.
    We sent the comment
    in moderation.
    We store all relevant
    informations in Hadoop for BI.
    

    View Slide

13. RabbitMQ
    Application
    mail
    publication
    indexation
    bi
    Consumer
    mail
    Consumer
    bi
    Consumer
    indexation
    Consumer
    moderation
    moderation
    

    View Slide

14. So cool!
    

    View Slide

15. That’s fast!
    At least for the user.
    

    View Slide

16. We have several small applications.
    

    View Slide

17. We can scale!
    

    View Slide

18. Problems...
    

    View Slide

19. Long
    running
    processes.
    

    View Slide

20. Expect errors.
    Deal with them.
    ● Catch them!
    ● Retry when needed.
    ● Don’t stop your consumer.
    

    View Slide

21. Your worker need to
    be stopped correctly.
    

    View Slide

22. View Slide

23. Goals
    

    View Slide

24. Solve
    mentioned
    problems.
    

    View Slide

25. Be able to change
    the broker used.
    

    View Slide

26. Implementation
    

    View Slide

27. We need a MessageProvider.
    We use the pecl extension.
    

    View Slide

28. Here is our “business” logic.
    

    View Slide

29. Let’s consume them all!
    

    View Slide

30. Encapsulation FTW! <3
    

    View Slide

31. Let’s create a stack.
    

    View Slide

32. The result.
    

    View Slide

33. View Slide

34. Used in production.
    

    View Slide

35. View Slide

36. Sometimes, dealing with
    consumers can be a pain...
    

    View Slide

37. Use a message broker,
    that’s awesome!
    

    View Slide

38. Sometimes, dealing with consumers can be a pain,
    BUT
    you should definitively use a Message Broker! <3
    Olivier Dolbeau
    @odolbeau
    

    View Slide