Cache me if you can: memcached, caching patterns and best practices by Guillaume Ardaud

Transcript

1. CACHE ME
   IF YOU CAN
   Guillaume Ardaud

   @gardaud | gardaud.fr

   gardaud@acm.org
   Friday April 11th

   PyCon2014

   Montreal
   

   View Slide

2. MEMCACHED
   -What is it?

   -How does it work?

   -Patterns & Best Practices
   

   View Slide

3. Key/Object Store
   • Give it an object, identified by a key

   • Ask for a key

   • If it isn’t expired, you get the object
   

   View Slide

4. O(1) EVERYTHING
   

   View Slide

5. primary data:

   relational database
   data you can lose, regenerate fast: 

   RAM store
   data you can lose, regenerate slowly: 

   persistent store
   Where does memcached fit?
   

   View Slide

6. Typical memcached API
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. Key names
    - ASCII based

    - Not crazy long…

    - … but explicit enough
    

    View Slide

13. Good naming
    e.g.: 

    json.users.
    

    View Slide

14. Bad naming
    e.g.: 

    md5(sql_query)
    

    View Slide

15. HOW DO I LIST ALL KEYS IN
    MEMCACHED?
    YOU DON’T
    (or all keys matching a regex, or all...)
    

    View Slide

16. it’s a cache
    
    not a database
    

    View Slide

17. View Slide

18. memcached docs:
    

    View Slide

19. View Slide

20. memcached is distributed
    

    View Slide

21. your amazing
    application
    memcached
    box A
    memcached
    box B
    memcached
    box C
    memcached client
    

    View Slide

22. your amazing
    application
    memcached
    box A
    memcached
    box B
    memcached
    box C
    memcached client
    set $KEY
    send $KEY

    to box C
    Received!
    

    View Slide

23. your amazing
    application
    memcached
    box A
    memcached
    box B
    memcached client
    set $KEY
    send $KEY

    to box A
    

    View Slide

24. ~ some memcached subtleties ~
    

    View Slide

25. EXPIRATION
    It’s 8am, a request stores an
    object under the key ‘foo’, with
    a 2 hour expiration time.
    What happens at 10am?
    

    View Slide

26. But what happens when a client tries to access 'foo'
    
    at 1pm?
    …'foo' gets removed.
    NOTHING
    

    View Slide

27. Q: Sometimes
    fetching a key returns
    None even though it’s
    not supposed to have
    expired yet?
    

    View Slide

28. Memcached has a fixed amount
    of memory."
    If you try to store data when
    memory is full, data will get
    evicted.
    

    View Slide

29. How does memcached determine what to
    evict?
    Object 1 Object 2
    timestamp:
    
    1378399504
    timestamp:
    
    137839983
    Object 3
    timestamp:
    
    1378399939
    Object 4
    timestamp:
    
    13783992932
    Object 5
    timestamp:
    
    13783993829
    

    View Slide

30. Q: Sometimes fetching a
    key returns None
    even though it’s not
    supposed to have expired
    yet…
    … AND memcached still
    has free memory?
    

    View Slide

31. View Slide

32. page
    1
    page
    2
    page
    3
    page
    4
    page
    5
    page
    6
    page
    7
    ...
    1 page = 1MB
    

    View Slide

33. page
    chunk
    chunk
    chunk chunk chunk chunk chunk chunk chunk
    chunk chunk chunk chunk chunk chunk chunk
    1st_slab_chunk_size = 80 bytes!
    2nd_slab_chunk_size = 80 * 1.25 = 100 bytes!
    3rd_slab_chunk_size = 100 * 1.25 = 125 bytes!
    . . .!
    last_slab_chunk_size = 1048576 bytes
    Last slab has 1 chunk of 1 MB.
    

    View Slide

34. View Slide

35. page
    1
    page
    2
    page
    3
    page
    4
    page
    5
    page
    6
    page
    7
    ...
    class 1 class 2 class 3 class 4 class 5 free page free page
    

    View Slide

36. View Slide

37. If all pages of the needed class are full:
    
    get a free page and give it the needed class.
    If there are no more free pages: 

    the LRU kicks in and evicts data.
    Each slab class has its own LRU!
    "
    If your data needs class 3 and:
    
    - there are no free pages
    
    - all class 3 pages are full
    
    - some class 4 pages have free chunks
    
    Data in a class 3 page will still be evicted.
    

    View Slide

38. Q: Sometimes fetching a
    key returns None
    even though it’s not
    supposed to have expired
    yet…
    … AND memcached still
    has free memory?
    

    View Slide

39. some useful command line flags
    memcached -v : verbose output (more verbose: -vv)
    
    memcached -M: doesn’t evict when out of memory, but errors
    
    "
    memcached -I1k : change slab page size (1k - 128m)
    
    memcached -I1m!
    memcached -I128m!
    "
    memcached -f1.5 : change growth factor (def. 1.25, >1)
    
    "
    "
    man memcached is your friend
    

    View Slide

40. Let’s get our
    hands dirty.
    common memcached patterns &
    practices in Python
    

    View Slide

41. View Slide

42. View Slide

43. View Slide

44. MODEL VERSIONING
    

    View Slide

45. FETCH STRAIGHT FROM CACHE
    

    View Slide

46. View Slide

47. THUNDERING HERD
    

    View Slide

48. View Slide

49. CACHING LARGE VALUES
    

    View Slide

50. 2-PHASE FETCH
    

    View Slide

51. 1. Break big object in smaller chunks
    
    (for example chunks of 10 books)
    PAGINATED CACHE
    2. Store each chunk as a separate
    
    object in memcached
    3. Store the list of keys
    To fetch the data, fetch the list of keys,
    
    iterate through it, and concatenate all fetched lists.
    

    View Slide

52. View Slide

53. View Slide

54. View Slide

55. Questions?
    THANKS!
    

    View Slide

56. Great memcached resources
    Memcached doc:
    http://code.google.com/p/memcached/wiki/NewStart?tm=6
    
    "
    Especially those 2 pages:
    http://code.google.com/p/memcached/wiki/NewProgrammingTricks
    
    http://code.google.com/p/memcached/wiki/NewUserInternals
    
    "
    Various articles/posts about internals etc.:
    http://stackoverflow.com/questions/6868256/memcached-eviction-prior-to-key-
    expiry
    
    http://www.adayinthelifeof.nl/2011/02/06/memcache-internals/
    
    http://work.tinou.com/2011/04/memcached-for-dummies.html
    
    http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/
    
    

    View Slide

57. View Slide