Swarrot, a library to consume them all!

Talk given at BlaBlaCar Tech Meetup in Warsaw

Transcript

2. @odolbeau Web Architect Olivier Dolbeau
4. What happens when you publish a trip?
5. We send an email to the driver. We index the trip in ElasticSearch. We send the comment to moderation spooler. We store all relevant informations in Hadoop for BI.
6. I’m gonna sleep, tell me when you’re done!
9. “RabbitMQ is a message broker. The principal idea is pretty simple: it accepts and forwards messages.”
10. Publisher Queue1 Exchange Queue2 Queue3 Consumer Consumer Consumer Consumer RabbitMQ
11. What happens when you publish a trip?
12. We send an email to the driver. We index the trip in ElasticSearch. We sent the comment in moderation. We store all relevant informations in Hadoop for BI.
13. RabbitMQ Application mail publication indexation bi Consumer mail Consumer bi Consumer indexation Consumer moderation moderation
14. So cool!
15. That’s fast! At least for the user.
16. We have several small applications.
17. We can scale!
18. Problems...
19. Long running processes.
20. Expect errors. Deal with them. • Catch them! • Retry when needed. • Don’t stop your consumer.
21. Your worker need to be stopped correctly.
23. Goals
24. Solve mentioned problems.
25. Be able to change the broker used.
26. Implementation
27. We need a MessageProvider. We use the pecl extension.
28. Here is our “business” logic.
29. Let’s consume them all!
30. Encapsulation FTW! <3
31. Let’s create a stack.
32. The result.
34. Used in production.
36. Sometimes, dealing with consumers can be a pain...
37. Use a message broker, that’s awesome!
38. Sometimes, dealing with consumers can be a pain, BUT you should definitively use a Message Broker! <3 Olivier Dolbeau @odolbeau