1M tickets, 50 lang and 35 currencies

Ab38213c583c40dff23d6244267de20c?s=47 Jose
November 22, 2015

1M tickets, 50 lang and 35 currencies

How we scaled-out our platform to serve 1M tickets in 50 lang and 35 different currencies without database hits

Ab38213c583c40dff23d6244267de20c?s=128

Jose

November 22, 2015
Tweet

Transcript

  1. 1M tickets, 50 langs and 35 currencies no database hits

  2. @jgargallo Jose Gargallo

  3. A marketplace where fans can buy and sell tickets

  4. Vendo boli, regalo entradas para los Rolling

  5. ~ 400 employees ~ 50 countries ~ 90 m€ in

    2015
  6. #Tickets histogram

  7. ~ 1M tickets ~ 50 langs ~ 35 currencies scenario

    ~ legal res. ~ logistic res. ~ high traffic
  8. CAP theorem Consistency Availability Partition tolerance

  9. CAP theorem Consistency Availability Partition tolerance ¡you can’t have it

    all!
  10. CAP theorem Consistency Availability Partition tolerance

  11. Ticketbis architecture

  12. Python daemons . . . . . .

  13. Python daemons Getting context

  14. Python daemons Running daemon

  15. Redis queues

  16. Redis queues (1) A LUA script recovers all failed tickets

  17. Redis queues (2) Changed tickets are pushed to the queue

    (LPUSH). (3) Consumer process. RPOPLPUSH allows us to keep queues reliable, if the process fails, the script will recover unprocessed entities. (4) Items are removed from the processing queue.
  18. ~ 1M tickets ~ 50 langs ~ 35 currencies Denormalization

    ~ legal res. ~ logistic res. ~ Event stats
  19. ~ 1M tickets ~ 50 langs ~ 35 currencies Denormalization

    ~ legal res. ~ logistic res. ~ Event stats
  20. Decimal Arithmetic Benchmark float cdecimal decimal 0.12s 0.27s 17s

  21. cdecimal in Python 2 __init__.py ~ cdecimal has been integrated

    into CPython 3.3, where it supersedes the pure Python version ~ cdecimal shipped with CPython 3.3 is significantly faster for numerical workload than cdecimal-2.3
  22. There are only two hard things in Computer Science: cache

    invalidation and naming things -- Phil Karlton
  23. Project name: Ironman

  24. ~ Inverted indexing ~ TTL Cache invalidation

  25. Cache invalidation Inverted indexing ~ Categories ~ Countries ~ others

  26. Cache invalidation TTLs EXPIRE Redis command allows us to set

    timeouts on keys based on event’s date
  27. Ticketbis architecture

  28. Elasticsearch

  29. ~ A distributed RESTful search engine ~ Apache Lucene is

    the search engine lib behind ES Elasticsearch
  30. ~ public data: categories & events ~ private data: purchases,

    users, ... ~ Ticket prices log Elasticsearch: clusters
  31. Elasticsearch: Ticket log World cup semifinal 2014 average ticket prices

    BRA - GER qualified 1day left
  32. Next steps ~ Use Redis as an LRU cache allkeys-lru,

    volatile-lru, ... ~ API hitting cached data
  33. Thank you! Jose Gargallo @jgargallo

  34. We are hiring! itjobs@ticketbis.com @TicketbisEng ...and we are remote friendly!