Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Implementing ARC: For Fun, Not Profit

Implementing ARC: For Fun, Not Profit

This is a talk about implementing an adaptive replacement cache (ARC)
in haskell. This talk was presented at FP-Syd, July 2015.

Mark Hibberd

July 22, 2015
Tweet

More Decks by Mark Hibberd

Other Decks in Programming

Transcript

  1. arc
    Adaptive Replacement Cache: for fun, not profit
    @markhibberd

    View Slide

  2. “Yea, from the table of my memory,
    I'll wipe away all trivial fond records”
    William Shakespeare -
    Hamlet, Act I, Scene IV

    View Slide

  3. View Slide

  4. “We are therefore forced to
    recognize the possibility of
    constructing a hierarchy of
    memories, each of which has greater
    capacity than the preceding but
    which is less quickly accessible.”
    A. W. Burks, H. H. Goldstine, J. von Neumann: -
    Preliminary Discussion of the Logical Design of Electronic Computing
    Instrument, Part I, Vol. I, Report prepared for the U.S. Army Ord. Dept.

    View Slide

  5. ( http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf )

    View Slide

  6. View Slide

  7. View Slide

  8. Reliable Storage

    View Slide

  9. Ephemeral Computation

    View Slide

  10. Lazy Replication w/ Disk
    and Network Cache

    View Slide

  11. Durable / Replicated
    Intent Log

    View Slide

  12. View Slide

  13. View Slide

  14. GET /user/1
    { “user” : “ocelot” }

    View Slide

  15. GET /user/1
    { “user” : “ocelot” }

    View Slide

  16. GET /user/1
    { “user” : “ocelot” }

    View Slide

  17. A cache perhaps?

    View Slide

  18. LRU
    A “default” choice
    Constant time and space complexity
    *very bad* in the face of scans

    View Slide

  19. LFU
    Often better hit ratio
    Logarithmic time complexity
    resilient to scans

    View Slide

  20. Hybrid LRU + LFU
    Lots of attempts
    Most have logarithmic time complexity
    *very bad* for general purpose (tuning)

    View Slide

  21. View Slide

  22. ARC
    Combines frequency and recency
    in the most optimal way for
    routing money from your pocket to

    View Slide

  23. ARC
    Exploits frequency and recency
    Constant time and space complexity
    Self tuning, good for general purpose

    View Slide

  24. L1: recency - keys were seen
    at least once recently
    L1 L2
    ARC

    View Slide

  25. L1: recency - keys were seen
    at least once recently
    L1 L2
    MRU
    LRU
    ARC

    View Slide

  26. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU
    ARC

    View Slide

  27. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU LRU
    MRU
    ARC

    View Slide

  28. T1: Cached keys in L1
    L1 L2
    T1
    MRU
    LRU LRU
    MRU
    ARC

    View Slide

  29. B1: Tracked (but not cached)
    keys in L1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1
    ARC

    View Slide

  30. L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2
    T2: Cached keys in L2
    ARC

    View Slide

  31. B2: Tracked (but not cached)
    keys in L2
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  32. 1. If we get a hit in T1 or T2
    do nothing
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  33. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  34. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  35. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  36. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View Slide

  37. ARC
    This is interesting because…
    It is relatively easy to understand
    Basically LRU with an extra directory
    Easy to adapt LRU like algorithms
    ( https://dl.dropboxusercontent.com/u/91714474/Papers/oneup.pdf )

    View Slide

  38. L2 ARC

    View Slide

  39. L2 ARC
    ARC
    L2 ARC
    L2 ARC
    NETWORK

    View Slide

  40. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK

    View Slide

  41. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss

    View Slide

  42. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss

    View Slide

  43. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC

    View Slide

  44. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View Slide

  45. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View Slide

  46. L2 ARC: Challenges
    This doesn’t work
    If not careful about updating L2 ARC
    can bottleneck reads everywhere

    View Slide

  47. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View Slide

  48. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond
    Respond & Update ARC

    View Slide

  49. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE

    View Slide

  50. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update

    View Slide

  51. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update
    Very prepared to drop
    L2 updates on the floor

    View Slide

  52. View Slide

  53. Results

    View Slide

  54. Results
    Lack of properly implemented
    LRU caches in Haskell

    View Slide

  55. Results

    View Slide

  56. Results
    https://themonadreader.files.wordpress.com/2010/05/issue16.pdf

    View Slide

  57. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )

    View Slide

  58. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )

    View Slide

  59. View Slide