If THIS Then Deploy: A look into IFTTT's Infrastructure

Transcript

1. IF THIS THEN DEPLOY
   A LOOK INTO IFTTT'S INFRASTRUCTURE
   @nettofarah
   1 / 33
   

   View Slide

2. nettofarah
   github.com/nettofarah
   @nettofarah
   2 / 33
   

   View Slide

3. Disclaimer
   I'm not a DevOps guy!
   3 / 33
   

   View Slide

4. Senior Product Engineer @ IFTTT
   4 / 33
   

   View Slide

5. Background
   Millions of Background Jobs every day
   Over a Billion Events every day
   215 (and growing) partner integrations
   5 / 33
   

   View Slide

6. a Rails Monolith
   6 / 33
   

   View Slide

7. How did it all start?
   7 / 33
   

   View Slide

8. Rails Monolith App
   Great for validating ideas
   Solid tools for dev and ops
   8 / 33
   

   View Slide

9. Cron Jobs
   9 / 33
   

   View Slide

10. Deploys in Rails
    10 / 33
    

    View Slide

11. Capistrano
    11 / 33
    

    View Slide

12. Capistrano is AWESOME!
    works for web apps
    works really well for background jobs too
    12 / 33
    

    View Slide

13. It works amazing
    until you have to deploy it to a couple of
    hundred machines
    13 / 33
    

    View Slide

14. Consequences
    Time outs
    Long Deploy Cycle
    Complex deploy code
    Simultaneous Restarts can kill you!
    14 / 33
    

    View Slide

15. Provisioning with Chef
    Really good tool
    But can get complex over time
    15 / 33
    

    View Slide

16. Personal opinion on Chef
    I have a really hard time figuring out where
    files should live.
    16 / 33
    

    View Slide

17. Maybe there's a design problem
    here
    17 / 33
    

    View Slide

18. That's probably true
    But we're a startup. We have limited
    resources and need to move fast.
    18 / 33
    

    View Slide

19. A Solution for Deploying to
    hundreds of servers
    With Capistrano
    19 / 33
    

    View Slide

20. Deploynamo
    Async Deploys
    20 / 33
    

    View Slide

21. A Case for Micro Services
    Teams are growing fast
    The product and userbase are growing
    even faster
    Old code lurking around from years ago
    21 / 33
    

    View Slide

22. We can take out some important
    parts of the system
    22 / 33
    

    View Slide

23. And turn them into Micro Services
    Many different languages and datastores.
    Most of them deployed to Heroku.
    23 / 33
    

    View Slide

24. We love Heroku!
    24 / 33
    

    View Slide

25. Docker comes into the picture
    25 / 33
    

    View Slide

26. What if we could...
    26 / 33
    

    View Slide

27. deploy our services + app
    to a more homogenous infrastructure?
    27 / 33
    

    View Slide

28. Docker + Mesos + Marathon +
    Chronos
    (Not live in production yet)
    28 / 33
    

    View Slide

29. Multiple containers with different
    roles working together
    In the same infrastructure
    29 / 33
    

    View Slide

30. We can now tweak our services and
    have more control over our
    deployment process
    30 / 33
    

    View Slide

31. Of course it comes at a cost
    paradigm shift
    complexity
    31 / 33
    

    View Slide

32. The Future
    Everything is a Container
    Images are built automatically
    New Services are just a new Docker image
    dropped onto Mesos
    32 / 33
    

    View Slide

33. Questions
    33 / 33
    

    View Slide