If THIS Then Deploy: A look into IFTTT's Infrastructure

Transcript

1. IF THIS THEN DEPLOY A LOOK INTO IFTTT'S INFRASTRUCTURE @nettofarah 1 / 33
2. nettofarah github.com/nettofarah @nettofarah 2 / 33
3. Disclaimer I'm not a DevOps guy! 3 / 33
4. Senior Product Engineer @ IFTTT 4 / 33
5. Background Millions of Background Jobs every day Over a Billion Events every day 215 (and growing) partner integrations 5 / 33
6. a Rails Monolith 6 / 33
7. How did it all start? 7 / 33
8. Rails Monolith App Great for validating ideas Solid tools for dev and ops 8 / 33
9. Cron Jobs 9 / 33
10. Deploys in Rails 10 / 33
11. Capistrano 11 / 33
12. Capistrano is AWESOME! works for web apps works really well for background jobs too 12 / 33
13. It works amazing until you have to deploy it to a couple of hundred machines 13 / 33
14. Consequences Time outs Long Deploy Cycle Complex deploy code Simultaneous Restarts can kill you! 14 / 33
15. Provisioning with Chef Really good tool But can get complex over time 15 / 33
16. Personal opinion on Chef I have a really hard time figuring out where files should live. 16 / 33
17. Maybe there's a design problem here 17 / 33
18. That's probably true But we're a startup. We have limited resources and need to move fast. 18 / 33
19. A Solution for Deploying to hundreds of servers With Capistrano 19 / 33
20. Deploynamo Async Deploys 20 / 33
21. A Case for Micro Services Teams are growing fast The product and userbase are growing even faster Old code lurking around from years ago 21 / 33
22. We can take out some important parts of the system 22 / 33
23. And turn them into Micro Services Many different languages and datastores. Most of them deployed to Heroku. 23 / 33
24. We love Heroku! 24 / 33
25. Docker comes into the picture 25 / 33
26. What if we could... 26 / 33
27. deploy our services + app to a more homogenous infrastructure? 27 / 33
28. Docker + Mesos + Marathon + Chronos (Not live in production yet) 28 / 33
29. Multiple containers with different roles working together In the same infrastructure 29 / 33
30. We can now tweak our services and have more control over our deployment process 30 / 33
31. Of course it comes at a cost paradigm shift complexity 31 / 33
32. The Future Everything is a Container Images are built automatically New Services are just a new Docker image dropped onto Mesos 32 / 33
33. Questions 33 / 33