Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Implementing ARC: For Fun, Not Profit

Implementing ARC: For Fun, Not Profit

This is a talk about implementing an adaptive replacement cache (ARC)
in haskell. This talk was presented at FP-Syd, July 2015.

Mark Hibberd

July 22, 2015
Tweet

More Decks by Mark Hibberd

Other Decks in Programming

Transcript

  1. arc
    Adaptive Replacement Cache: for fun, not profit
    @markhibberd

    View full-size slide

  2. “Yea, from the table of my memory,
    I'll wipe away all trivial fond records”
    William Shakespeare -
    Hamlet, Act I, Scene IV

    View full-size slide

  3. “We are therefore forced to
    recognize the possibility of
    constructing a hierarchy of
    memories, each of which has greater
    capacity than the preceding but
    which is less quickly accessible.”
    A. W. Burks, H. H. Goldstine, J. von Neumann: -
    Preliminary Discussion of the Logical Design of Electronic Computing
    Instrument, Part I, Vol. I, Report prepared for the U.S. Army Ord. Dept.

    View full-size slide

  4. ( http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf )

    View full-size slide

  5. Reliable Storage

    View full-size slide

  6. Ephemeral Computation

    View full-size slide

  7. Lazy Replication w/ Disk
    and Network Cache

    View full-size slide

  8. Durable / Replicated
    Intent Log

    View full-size slide

  9. GET /user/1
    { “user” : “ocelot” }

    View full-size slide

  10. GET /user/1
    { “user” : “ocelot” }

    View full-size slide

  11. GET /user/1
    { “user” : “ocelot” }

    View full-size slide

  12. A cache perhaps?

    View full-size slide

  13. LRU
    A “default” choice
    Constant time and space complexity
    *very bad* in the face of scans

    View full-size slide

  14. LFU
    Often better hit ratio
    Logarithmic time complexity
    resilient to scans

    View full-size slide

  15. Hybrid LRU + LFU
    Lots of attempts
    Most have logarithmic time complexity
    *very bad* for general purpose (tuning)

    View full-size slide

  16. ARC
    Combines frequency and recency
    in the most optimal way for
    routing money from your pocket to

    View full-size slide

  17. ARC
    Exploits frequency and recency
    Constant time and space complexity
    Self tuning, good for general purpose

    View full-size slide

  18. L1: recency - keys were seen
    at least once recently
    L1 L2
    ARC

    View full-size slide

  19. L1: recency - keys were seen
    at least once recently
    L1 L2
    MRU
    LRU
    ARC

    View full-size slide

  20. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU
    ARC

    View full-size slide

  21. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU LRU
    MRU
    ARC

    View full-size slide

  22. T1: Cached keys in L1
    L1 L2
    T1
    MRU
    LRU LRU
    MRU
    ARC

    View full-size slide

  23. B1: Tracked (but not cached)
    keys in L1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1
    ARC

    View full-size slide

  24. L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2
    T2: Cached keys in L2
    ARC

    View full-size slide

  25. B2: Tracked (but not cached)
    keys in L2
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  26. 1. If we get a hit in T1 or T2
    do nothing
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  27. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  28. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  29. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  30. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC

    View full-size slide

  31. ARC
    This is interesting because…
    It is relatively easy to understand
    Basically LRU with an extra directory
    Easy to adapt LRU like algorithms
    ( https://dl.dropboxusercontent.com/u/91714474/Papers/oneup.pdf )

    View full-size slide

  32. L2 ARC
    ARC
    L2 ARC
    L2 ARC
    NETWORK

    View full-size slide

  33. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK

    View full-size slide

  34. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss

    View full-size slide

  35. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss

    View full-size slide

  36. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC

    View full-size slide

  37. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View full-size slide

  38. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View full-size slide

  39. L2 ARC: Challenges
    This doesn’t work
    If not careful about updating L2 ARC
    can bottleneck reads everywhere

    View full-size slide

  40. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC

    View full-size slide

  41. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond
    Respond & Update ARC

    View full-size slide

  42. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE

    View full-size slide

  43. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update

    View full-size slide

  44. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update
    Very prepared to drop
    L2 updates on the floor

    View full-size slide

  45. Results
    Lack of properly implemented
    LRU caches in Haskell

    View full-size slide

  46. Results
    https://themonadreader.files.wordpress.com/2010/05/issue16.pdf

    View full-size slide

  47. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )

    View full-size slide

  48. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )

    View full-size slide