Ticketea, scaling a highly transactional business

Ticketea, scaling a highly transactional business

A talk about things that we've done to scale ticketea demanding growing business given by @maraujop at Mad scalability


Ticketea Engineering

March 22, 2016


  1. 3.

    Who are we? 3 @JavierHdez3 @maraujop @igalarzab @sullymorland @patoroco @imanolcg

    @RafaRM20 @iamcarlosedo @andrea_mgr @Mc_Arena_pr @javitxudedios @Maquert @gnufede @willyfrog_ @ShideShugo
  2. 6.

    “Microservices are about scaling the number of engineers not the

    number of requests” Jay Kreps (Co-Creator of kafka)
  3. 10.

    Pros 10 •5 coders worked at the company •Fast to

    build, easy to test and deploy •Simple stack PROS
  4. 11.

    Cons 11 •Workers blocked on API connections •Errors in backoffice

    could crash the site •Slow API means nothing loads CONS
  5. 12.
  6. 15.
  7. 18.
  8. 19.
  9. 21.

    Why the cloud? 21 •High availability despite of load-spikes •Scale

    from 3 servers to 400 in minutes •Awesome managed services
  10. 22.

    Why the cloud? 22 •Easier to be fault tolerant •They

    have great uptime •We build AMIs on deploys and use autoscaling •Preheating
  11. 23.
  12. 25.

    Scaling 25 MY LEGACY CODE IS SLOW •Scale In: Bigger

    machine •Scale Out: More machines behind a load balancer
  13. 26.
  14. 27.

    Things you can do 27 •Optimize where your real bottleneck

    is •Cache more things •Speed up your database usage •Denormalize your database •Execute less code •Speed up your code
  15. 29.

    Execute less code 29 •Each time your workers are executing

    your code they are not able to attend new requests •Move everything not essential to background tasks, specially I/O bound jobs •Watch your timeouts
  16. 30.

    Database 30 •Relational databases are “almost” never the problem •Check

    your slowlog (create indexes) •Optimize slow or recurrent queries •Use read replicas (careful with replication lag)
  17. 31.
  18. 34.

    Database contingency 34 •Wrapping everything in a transaction is not

    cost effective •Deadlocks are tricky to spot •Caching data in this situation can be counter-productive
  19. 38.