1M tickets, 50 lang and 35 currencies

How we scaled-out our platform to serve 1M tickets in 50 lang and 35 different currencies without database hits

Transcript

1. 1M tickets, 50 langs and
   35 currencies
   no database hits
   

   View Slide

2. @jgargallo
   Jose Gargallo
   

   View Slide

3. A marketplace
   where fans can buy
   and sell tickets
   

   View Slide

4. Vendo boli, regalo entradas para los Rolling
   

   View Slide

5. ~ 400 employees
   ~ 50 countries
   ~ 90 m€ in 2015
   

   View Slide

6. #Tickets histogram
   

   View Slide

7. ~ 1M tickets
   ~ 50 langs
   ~ 35 currencies
   scenario
   ~ legal res.
   ~ logistic res.
   ~ high traffic
   

   View Slide

8. CAP theorem
   Consistency Availability
   Partition
   tolerance
   

   View Slide

9. CAP theorem
   Consistency Availability
   Partition
   tolerance
   ¡you can’t have it all!
   

   View Slide

10. CAP theorem
    Consistency Availability
    Partition
    tolerance
    

    View Slide

11. Ticketbis architecture
    

    View Slide

12. Python daemons
    .
    .
    . .
    .
    .
    

    View Slide

13. Python daemons
    Getting context
    

    View Slide

14. Python daemons
    Running daemon
    

    View Slide

15. Redis queues
    

    View Slide

16. Redis queues
    (1) A LUA script recovers all failed tickets
    

    View Slide

17. Redis queues
    (2) Changed tickets are pushed to the queue (LPUSH).
    (3) Consumer process. RPOPLPUSH allows us to keep
    queues reliable, if the process fails, the script will
    recover unprocessed entities.
    (4) Items are removed from the processing queue.
    

    View Slide

18. ~ 1M tickets
    ~ 50 langs
    ~ 35 currencies
    Denormalization
    ~ legal res.
    ~ logistic res.
    ~ Event stats
    

    View Slide

19. ~ 1M tickets
    ~ 50 langs
    ~ 35 currencies
    Denormalization
    ~ legal res.
    ~ logistic res.
    ~ Event stats
    

    View Slide

20. Decimal Arithmetic Benchmark
    float cdecimal decimal
    0.12s 0.27s 17s
    

    View Slide

21. cdecimal in Python 2
    __init__.py
    ~ cdecimal has been integrated into CPython 3.3,
    where it supersedes the pure Python version
    ~ cdecimal shipped with CPython 3.3 is significantly
    faster for numerical workload than cdecimal-2.3
    

    View Slide

22. There are only two hard
    things in Computer Science:
    cache invalidation and
    naming things
    -- Phil Karlton
    

    View Slide

23. Project name: Ironman
    

    View Slide

24. ~ Inverted indexing
    ~ TTL
    Cache invalidation
    

    View Slide

25. Cache invalidation
    Inverted indexing
    ~ Categories
    ~ Countries
    ~ others
    

    View Slide

26. Cache invalidation
    TTLs
    EXPIRE Redis command allows us to set
    timeouts on keys based on event’s date
    

    View Slide

27. Ticketbis architecture
    

    View Slide

28. Elasticsearch
    

    View Slide

29. ~ A distributed RESTful search
    engine
    ~ Apache Lucene is the search
    engine lib behind ES
    Elasticsearch
    

    View Slide

30. ~ public data: categories & events
    ~ private data: purchases, users, ...
    ~ Ticket prices log
    Elasticsearch: clusters
    

    View Slide

31. Elasticsearch: Ticket log
    World cup semifinal 2014 average ticket prices
    BRA - GER
    qualified
    1day left
    

    View Slide

32. Next steps
    ~ Use Redis as an LRU cache
    allkeys-lru, volatile-lru, ...
    ~ API hitting cached data
    

    View Slide

33. Thank you!
    Jose Gargallo
    @jgargallo
    

    View Slide

34. We are hiring!
    itjobs@ticketbis.com
    @TicketbisEng
    ...and we are remote friendly!
    

    View Slide