Implementing ARC: For Fun, Not Profit

This is a talk about implementing an adaptive replacement cache (ARC)
in haskell. This talk was presented at FP-Syd, July 2015.

Transcript

1. arc Adaptive Replacement Cache: for fun, not profit @markhibberd
2. “Yea, from the table of my memory, I'll wipe away all trivial fond records” William Shakespeare - Hamlet, Act I, Scene IV
4. “We are therefore forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity than the preceding but which is less quickly accessible.” A. W. Burks, H. H. Goldstine, J. von Neumann: - Preliminary Discussion of the Logical Design of Electronic Computing Instrument, Part I, Vol. I, Report prepared for the U.S. Army Ord. Dept.
5. ( http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf )
8. Reliable Storage
9. Ephemeral Computation
10. Lazy Replication w/ Disk and Network Cache
11. Durable / Replicated Intent Log
14. GET /user/1 { “user” : “ocelot” }
15. GET /user/1 { “user” : “ocelot” }
16. GET /user/1 { “user” : “ocelot” }
17. A cache perhaps?
18. LRU A “default” choice Constant time and space complexity *very bad* in the face of scans
19. LFU Often better hit ratio Logarithmic time complexity resilient to scans
20. Hybrid LRU + LFU Lots of attempts Most have logarithmic time complexity *very bad* for general purpose (tuning)
22. ARC Combines frequency and recency in the most optimal way for routing money from your pocket to
23. ARC Exploits frequency and recency Constant time and space complexity Self tuning, good for general purpose
24. L1: recency - keys were seen at least once recently L1 L2 ARC
25. L1: recency - keys were seen at least once recently L1 L2 MRU LRU ARC
26. L2: frequency - keys were seen at least twice recently L1 L2 MRU LRU ARC
27. L2: frequency - keys were seen at least twice recently L1 L2 MRU LRU LRU MRU ARC
28. T1: Cached keys in L1 L1 L2 T1 MRU LRU LRU MRU ARC
29. B1: Tracked (but not cached) keys in L1 L1 L2 MRU LRU LRU MRU T1 B1 ARC
30. L1 L2 MRU LRU LRU MRU T1 B1 T2 T2: Cached keys in L2 ARC
31. B2: Tracked (but not cached) keys in L2 L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
32. 1. If we get a hit in T1 or T2 do nothing L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
33. 2. If we get a hit in B1 increase size of T1 L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
34. 2. If we get a hit in B1 increase size of T1 L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
35. 3. If we get a hit in B2 decrease size of T1 L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
36. 3. If we get a hit in B2 decrease size of T1 L1 L2 MRU LRU LRU MRU T1 B1 T2 B2 ARC
37. ARC This is interesting because… It is relatively easy to understand Basically LRU with an extra directory Easy to adapt LRU like algorithms ( https://dl.dropboxusercontent.com/u/91714474/Papers/oneup.pdf )
38. L2 ARC
39. L2 ARC ARC L2 ARC L2 ARC NETWORK
40. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK
41. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss
42. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss
43. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Update L2 ARC
44. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Update L2 ARC Respond & Update ARC
45. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Update L2 ARC Respond & Update ARC
46. L2 ARC: Challenges This doesn’t work If not careful about updating L2 ARC can bottleneck reads everywhere
47. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Update L2 ARC Respond & Update ARC
48. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond Respond & Update ARC
49. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Queue Respond & Update ARC WRITE
50. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Queue Respond & Update ARC WRITE Async L2 ARC update
51. L2 ARC ARC L2 ARC GET /user/1 L2 ARC NETWORK ARC Miss L2 ARC Miss Respond & Queue Respond & Update ARC WRITE Async L2 ARC update Very prepared to drop L2 updates on the floor
53. Results
54. Results Lack of properly implemented LRU caches in Haskell
55. Results
56. Results https://themonadreader.files.wordpress.com/2010/05/issue16.pdf
57. Cribbed Results ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )
58. Cribbed Results ( http://www.cs.cmu.edu/~15-440/READINGS/megiddo-computer2004.pdf )