$30 off During Our Annual Pro Sale. View Details »

Swarrot, a library to consume them all!

Swarrot, a library to consume them all!

Talk given at BlaBlaCar Tech Meetup in Warsaw

Olivier Dolbeau

October 20, 2015
Tweet

More Decks by Olivier Dolbeau

Other Decks in Programming

Transcript

  1. View Slide

  2. @odolbeau
    Web Architect
    Olivier Dolbeau

    View Slide

  3. View Slide

  4. What happens when you
    publish a trip?

    View Slide

  5. We send an email to
    the driver.
    We index the trip in
    ElasticSearch.
    We send the comment
    to moderation spooler.
    We store all relevant
    informations in Hadoop for BI.

    View Slide

  6. I’m gonna sleep, tell me
    when you’re done!

    View Slide

  7. View Slide

  8. View Slide

  9. “RabbitMQ is a message
    broker. The principal idea
    is pretty simple: it accepts
    and forwards messages.”

    View Slide

  10. Publisher
    Queue1
    Exchange Queue2
    Queue3
    Consumer
    Consumer
    Consumer
    Consumer
    RabbitMQ

    View Slide

  11. What happens when you
    publish a trip?

    View Slide

  12. We send an email to
    the driver.
    We index the trip in
    ElasticSearch.
    We sent the comment
    in moderation.
    We store all relevant
    informations in Hadoop for BI.

    View Slide

  13. RabbitMQ
    Application
    mail
    publication
    indexation
    bi
    Consumer
    mail
    Consumer
    bi
    Consumer
    indexation
    Consumer
    moderation
    moderation

    View Slide

  14. So cool!

    View Slide

  15. That’s fast!
    At least for the user.

    View Slide

  16. We have several small applications.

    View Slide

  17. We can scale!

    View Slide

  18. Problems...

    View Slide

  19. Long
    running
    processes.

    View Slide

  20. Expect errors.
    Deal with them.
    ● Catch them!
    ● Retry when needed.
    ● Don’t stop your consumer.

    View Slide

  21. Your worker need to
    be stopped correctly.

    View Slide

  22. View Slide

  23. Goals

    View Slide

  24. Solve
    mentioned
    problems.

    View Slide

  25. Be able to change
    the broker used.

    View Slide

  26. Implementation

    View Slide

  27. We need a MessageProvider.
    We use the pecl extension.

    View Slide

  28. Here is our “business” logic.

    View Slide

  29. Let’s consume them all!

    View Slide

  30. Encapsulation FTW! <3

    View Slide

  31. Let’s create a stack.

    View Slide

  32. The result.

    View Slide

  33. View Slide

  34. Used in production.

    View Slide

  35. View Slide

  36. Sometimes, dealing with
    consumers can be a pain...

    View Slide

  37. Use a message broker,
    that’s awesome!

    View Slide

  38. Sometimes, dealing with consumers can be a pain,
    BUT
    you should definitively use a Message Broker! <3
    Olivier Dolbeau
    @odolbeau

    View Slide