Implementing ARC: For Fun, Not Profit

This is a talk about implementing an adaptive replacement cache (ARC)
in haskell. This talk was presented at FP-Syd, July 2015.

Transcript

1. arc
   Adaptive Replacement Cache: for fun, not profit
   @markhibberd
   

   View Slide

2. “Yea, from the table of my memory,
   I'll wipe away all trivial fond records”
   William Shakespeare -
   Hamlet, Act I, Scene IV
   

   View Slide

3. View Slide

4. “We are therefore forced to
   recognize the possibility of
   constructing a hierarchy of
   memories, each of which has greater
   capacity than the preceding but
   which is less quickly accessible.”
   A. W. Burks, H. H. Goldstine, J. von Neumann: -
   Preliminary Discussion of the Logical Design of Electronic Computing
   Instrument, Part I, Vol. I, Report prepared for the U.S. Army Ord. Dept.
   

   View Slide

5. ( http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf )
   

   View Slide

6. View Slide

7. View Slide

8. Reliable Storage
   

   View Slide

9. Ephemeral Computation
   

   View Slide

10. Lazy Replication w/ Disk
    and Network Cache
    

    View Slide

11. Durable / Replicated
    Intent Log
    

    View Slide

12. View Slide

13. View Slide

14. GET /user/1
    { “user” : “ocelot” }
    

    View Slide

15. GET /user/1
    { “user” : “ocelot” }
    

    View Slide

16. GET /user/1
    { “user” : “ocelot” }
    

    View Slide

17. A cache perhaps?
    

    View Slide

18. LRU
    A “default” choice
    Constant time and space complexity
    *very bad* in the face of scans
    

    View Slide

19. LFU
    Often better hit ratio
    Logarithmic time complexity
    resilient to scans
    

    View Slide

20. Hybrid LRU + LFU
    Lots of attempts
    Most have logarithmic time complexity
    *very bad* for general purpose (tuning)
    

    View Slide

21. View Slide

22. ARC
    Combines frequency and recency
    in the most optimal way for
    routing money from your pocket to
    

    View Slide

23. ARC
    Exploits frequency and recency
    Constant time and space complexity
    Self tuning, good for general purpose
    

    View Slide

24. L1: recency - keys were seen
    at least once recently
    L1 L2
    ARC
    

    View Slide

25. L1: recency - keys were seen
    at least once recently
    L1 L2
    MRU
    LRU
    ARC
    

    View Slide

26. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU
    ARC
    

    View Slide

27. L2: frequency - keys were seen
    at least twice recently
    L1 L2
    MRU
    LRU LRU
    MRU
    ARC
    

    View Slide

28. T1: Cached keys in L1
    L1 L2
    T1
    MRU
    LRU LRU
    MRU
    ARC
    

    View Slide

29. B1: Tracked (but not cached)
    keys in L1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1
    ARC
    

    View Slide

30. L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2
    T2: Cached keys in L2
    ARC
    

    View Slide

31. B2: Tracked (but not cached)
    keys in L2
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

32. 1. If we get a hit in T1 or T2
    do nothing
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

33. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

34. 2. If we get a hit in B1
    increase size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

35. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

36. 3. If we get a hit in B2
    decrease size of T1
    L1 L2
    MRU
    LRU LRU
    MRU
    T1
    B1 T2 B2
    ARC
    

    View Slide

37. ARC
    This is interesting because…
    It is relatively easy to understand
    Basically LRU with an extra directory
    Easy to adapt LRU like algorithms
    ( https://dl.dropboxusercontent.com/u/91714474/Papers/oneup.pdf )
    

    View Slide

38. L2 ARC
    

    View Slide

39. L2 ARC
    ARC
    L2 ARC
    L2 ARC
    NETWORK
    

    View Slide

40. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    

    View Slide

41. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    

    View Slide

42. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss
    

    View Slide

43. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    

    View Slide

44. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC
    

    View Slide

45. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC
    

    View Slide

46. L2 ARC: Challenges
    This doesn’t work
    If not careful about updating L2 ARC
    can bottleneck reads everywhere
    

    View Slide

47. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Update L2 ARC
    Respond & Update ARC
    

    View Slide

48. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond
    Respond & Update ARC
    

    View Slide

49. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    

    View Slide

50. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update
    

    View Slide

51. L2 ARC
    ARC
    L2 ARC
    GET /user/1
    L2 ARC
    NETWORK
    ARC Miss
    L2 ARC Miss Respond & Queue
    Respond & Update ARC
    WRITE
    Async L2 ARC update
    Very prepared to drop
    L2 updates on the floor
    

    View Slide

52. View Slide

53. Results
    

    View Slide

54. Results
    Lack of properly implemented
    LRU caches in Haskell
    

    View Slide

55. Results
    

    View Slide

56. Results
    https://themonadreader.files.wordpress.com/2010/05/issue16.pdf
    

    View Slide

57. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )
    

    View Slide

58. Cribbed Results
    ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )
    

    View Slide

59. View Slide