Cloud and Devops at ticketea

Talk at Devops Azure Day Madrid 2015 about devops and cloud

Transcript

1. Cloud and Devops
   at ticketea
   Jose Ignacio Galarza, @igalarzab
   CTO
   

   View Slide

2. create tech solutions to simplify
   and enrich the way events are
   managed and attended
   

   View Slide

3. ticketea
   Ecosystem
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. ticketea
   Who we are?
   @esanchezm @JavierHdez3 @maraujop @igalarzab
   @sullymorland @patoroco edu
   @imanolcg
   @RafaRM20 @iamcarlosedo
   @andrea_mgr
   @Mc_Arena_pr
   @javitxudedios @Maquert @gnufede @willyfrog_
   

   View Slide

8. ticketea
   Team
   ✤ 12 developers
   ✤ 3 designers
   ✤ 1 QA
   

   View Slide

9. cloud
   

   View Slide

10. View Slide

11. ticketea
    Why the cloud?
    ✤ Offer high-availability despite of load-spikes.
    ✤ Scale from 3 servers to 350 in minutes.
    ✤ Access to awesome services provided by your IaSS.
    ✤ Be more cost efficient.
    

    View Slide

12. ticketea
    Why the cloud?
    ✤ Easier to be fault tolerant.
    ✤ They have a huge team improving always their service.
    ✤ Incredible uptime (some of them with 99.999999999%).
    

    View Slide

13. ticketea
    Why the cloud?
    IaaS to
    PaaS
    

    View Slide

14. Cloud
    provider
    Ticketea
    

    View Slide

15. ticketea
    Pitfalls
    ✤ Everything can fail… everything. Have a good plan B.
    ✤ A plan B isn't a plan B, until is tested.
    ✤ Pre-heat everything before big events.
    ✤ Read about your IaSS in forums/email groups.
    ✤ Try to distribute your machines to avoid thunderbolts ;)
    

    View Slide

16. architecture
    

    View Slide

17. ticketea
    server
    

    View Slide

18. ticketea
    Our architecture
    

    View Slide

19. ticketea
    Our architecture
    

    View Slide

20. ticketea
    Our architecture
    

    View Slide

21. ticketea
    Our architecture
    

    View Slide

22. ticketea
    Our architecture
    

    View Slide

23. ticketea
    Our architecture
    

    View Slide

24. ticketea
    servers
    

    View Slide

25. ticketea
    The multilith
    

    View Slide

26. ticketea
    Distributed systems
    ✤ Granular failing.
    ✤ Avoid single point of failure.
    ✤ Easy code maintainability
    ✤ Choose the best tech for each project.
    ✤ Scale the team easier.
    

    View Slide

27. ticketea
    Distributed systems
    ✤ Hard to deploy.
    ✤ Hard to see the whole picture.
    ✤ More complexity: easy to fail.
    ✤ When shit happens: you need a plan B.
    

    View Slide

28. don’t panic, we’re devops!
    

    View Slide

29. development
    

    View Slide

30. ticketea
    Team
    ✤ 12 developers
    ✤ 3 designers
    ✤ 1 QA
    ✤ 0 sysadmin
    

    View Slide

31. ticketea
    Team
    no specific devops team
    

    View Slide

32. understand your stack
    

    View Slide

33. ticketea
    Development cycle
    

    View Slide

34. ticketea
    Development keys
    ✤ Code reviews in every pull request.
    ✤ Continuous integration with a strong test suite.
    ✤ Cook your own receipts.
    ✤ One-click deployment.
    ✤ “Continuous” delivery. (not always)
    

    View Slide

35. and… easy rollbacks
    ;)
    

    View Slide

36. ticketea
    Deploying code
    ✤ We’ve changed between a few models, but finally:
    1. Create a complete new environment in prod machines.
    2. Deploy your code there.
    3. Extract one machine from the load-balancer.
    4. Create a golden-image
    

    View Slide

37. ticketea
    Deploying code
    ✤ Atomic deploys.
    ✤ 0s-downtime deploys.
    ✤ Easy rollback to the last N versions.
    ✤ Money-cost friendly ;)
    ✤ Insanely fast to scale machines when needed.
    

    View Slide

38. ticketea
    The Rat ‘devops’ race
    API
    Heracles
    Odin
    Frontend
    Chef
    Puppet
    Ansible
    Thor
    Caronte
    Ansible
    

    View Slide

39. View Slide

40. ticketea
    Production vs Development environments
    ✤ Using the same ansible roles
    ✤ Meta-roles:
    roles/
    | ——django/
    | ————tasks/main.yml
    | ——api/
    | ————tasks/main.yml
    |—— tkt-api/
    |———— meta/main.yml
    api.yml
    development.yml
    

    View Slide

41. ticketea
    Automating common procedures
    ✤ fab -f heracles.py stop_consuming_messages:queue
    ✤ fab -f frontend.py maintenance_on
    ✤ fab -f sg.py open_sg
    ✤ fab -f api.py fast_rollback:version
    

    View Slide

42. metrics
    

    View Slide

43. ticketea
    Provisioning
    ✤ Ansible
    ✤ Fabric
    

    View Slide

44. ticketea
    Error reporting
    ✤ sentry
    ✤ fabric (fka. crashlytics)
    

    View Slide

45. ticketea
    Graphing
    ✤ graphite
    ✤ grafana
    

    View Slide

46. ticketea
    Logging and alarming
    ✤ rsyslog
    ✤ graylog
    ✤ cabot
    ✤ slack/email/x connected through a queue
    

    View Slide

47. Thanks. ;-)
    Jose Ignacio Galarza, @igalarzab
    CTO
    

    View Slide