Swarrot, a library to consume them all!

Swarrot, a library to consume them all!

Talk given at BlaBlaCar Tech Meetup in Warsaw

418997665c4a3368515ecf9c3d746b95?s=128

Olivier Dolbeau

October 20, 2015
Tweet

Transcript

  1. None
  2. @odolbeau Web Architect Olivier Dolbeau

  3. None
  4. What happens when you publish a trip?

  5. We send an email to the driver. We index the

    trip in ElasticSearch. We send the comment to moderation spooler. We store all relevant informations in Hadoop for BI.
  6. I’m gonna sleep, tell me when you’re done!

  7. None
  8. None
  9. “RabbitMQ is a message broker. The principal idea is pretty

    simple: it accepts and forwards messages.”
  10. Publisher Queue1 Exchange Queue2 Queue3 Consumer Consumer Consumer Consumer RabbitMQ

  11. What happens when you publish a trip?

  12. We send an email to the driver. We index the

    trip in ElasticSearch. We sent the comment in moderation. We store all relevant informations in Hadoop for BI.
  13. RabbitMQ Application mail publication indexation bi Consumer mail Consumer bi

    Consumer indexation Consumer moderation moderation
  14. So cool!

  15. That’s fast! At least for the user.

  16. We have several small applications.

  17. We can scale!

  18. Problems...

  19. Long running processes.

  20. Expect errors. Deal with them. • Catch them! • Retry

    when needed. • Don’t stop your consumer.
  21. Your worker need to be stopped correctly.

  22. None
  23. Goals

  24. Solve mentioned problems.

  25. Be able to change the broker used.

  26. Implementation

  27. We need a MessageProvider. We use the pecl extension.

  28. Here is our “business” logic.

  29. Let’s consume them all!

  30. Encapsulation FTW! <3

  31. Let’s create a stack.

  32. The result.

  33. None
  34. Used in production.

  35. None
  36. Sometimes, dealing with consumers can be a pain...

  37. Use a message broker, that’s awesome!

  38. Sometimes, dealing with consumers can be a pain, BUT you

    should definitively use a Message Broker! <3 Olivier Dolbeau @odolbeau