How to memoize

Memoization is an optimisation technique that works by caching the results of slow functions. It’s easy to implement, but a production-ready implementation is considerably trickier. And how do you test that your implementation is correct, and useful?

I’ll show you the tricky bits in implementing memoization, where I’ll touch on Ruby metaprogramming and memory management. Lastly, I’ll show how to measure and verify that memoization is meaningful.

Transcript

1. PRESENTATION TITLE:
   [REDACTED]
   AUTHOR:
   DENIS DEFREYNE (@DDFREYNE)
   OCCASION:
   RUG45B, 2018-03-01

   

   AUDIENCE:
   INHABITANTS OF EARTH
   

   View Slide

2. What do you do when you have an expensive function,
   and you need to use its return value multiple times?
   

   View Slide

3. def total_price
   vat = calc_base_price * VAT_RATE
   calc_base_price + vat
   end

   

   View Slide

4. def total_price
   base_price = calc_base_price
   vat = base_price * VAT_RATE
   base_price + vat
   end
   

   View Slide

5. 0 LOCAL VARIABLES
   

   View Slide

6. A B
   C D E F G H
   B1 B2
   

   View Slide

7. children
   

   View Slide

8. children.map(&:depth)
   

   View Slide

9. children.map(&:depth).max
   

   View Slide

10. children.map(&:depth).max || 0
    

    View Slide

11. (children.map(&:depth).max || 0) + 1
    

    View Slide

12. def depth
    (children.map(&:depth).max || 0) + 1
    end
    

    View Slide

13. A B
    C D E F G H
    B1 B2
    

    View Slide

14. a.depth = b.depth + 1
    A B
    C D E F G H
    B1 B2
    

    View Slide

15. a.depth = b.depth + 1
    b.depth = [b1.depth, c.depth].max + 1
    A B
    C D E F G H
    B1 B2
    

    View Slide

16. a.depth = b.depth + 1
    b.depth = [b1.depth, c.depth].max + 1
    …
    A B
    C D E F G H
    B1 B2
    

    View Slide

17. a.depth = b.depth + 1
    b.depth = [b1.depth, c.depth].max + 1
    …
    b2.depth = f.depth + 1
    A B
    C D E F G H
    B1 B2
    

    View Slide

18. a.depth = b.depth + 1
    b.depth = [b1.depth, c.depth].max + 1
    …
    b2.depth = f.depth + 1
    e.depth = f.depth + 1
    A B
    C D E F G H
    B1 B2
    

    View Slide

19. a.depth = b.depth + 1
    b.depth = [b1.depth, c.depth].max + 1
    …
    b2.depth = f.depth + 1
    e.depth = f.depth + 1
    …
    A B
    C D E F G H
    B1 B2
    

    View Slide

20. def depth
    (children.map(&:depth).max || 0) + 1
    end
    

    View Slide

21. def depth
    @_depth ||= (children.map(&:depth).max || 0) + 1
    end
    

    View Slide

22. 1 MEMOIZATION USING ||=
    

    View Slide

23. def fib(n)
    case n
    when 0
    0
    when 1
    1
    else
    fib(n - 1) + fib(n - 2)
    end
    end
    

    View Slide

24. def fib(n)
    case n
    when 0
    0
    when 1
    1
    else
    fib(n - 1) + fib(n - 2)
    end
    end
    fib(0) = 0
    fib(1) = 1
    fib(2) = 1
    fib(3) = 2
    fib(4) = 3
    fib(5) = 5
    fib(6) = 8
    fib(7) = 13
    fib(8) = 21
    fib(9) = 34
    

    View Slide

25. View Slide

26. fib(40)
    

    View Slide

27. fib(40)
    = fib(38) + fib(39)
    

    View Slide

28. fib(40)
    = fib(38) + fib(39)
    = fib(38) + fib(37) + fib(38)
    

    View Slide

29. fib(40)
    = fib(38) + fib(39)
    = fib(38) + fib(37) + fib(38)
    = fib(36) + fib(37) + fib(37) + fib(38)
    

    View Slide

30. fib(40)
    = fib(38) + fib(39)
    = fib(38) + fib(37) + fib(38)
    = fib(36) + fib(37) + fib(37) + fib(38)
    = fib(36) + fib(37) + fib(37) + fib(36) + fib(37)
    

    View Slide

31. fib(40)
    = fib(38) + fib(39)
    = fib(38) + fib(37) + fib(38)
    = fib(36) + fib(37) + fib(37) + fib(38)
    = fib(36) + fib(37) + fib(37) + fib(36) + fib(37)
    = …
    

    View Slide

32. fib(40) takes 15 seconds

    on my 3.1 GHz machine!
    

    View Slide

33. def fib(n)
    init = [0, 1]
    while init.size <= n
    init << init.last(2).sum
    end
    init[n]
    end
    

    View Slide

34. def fib(n)
    @fib ||= {}
    @fib[n] ||=
    case n
    when 0
    0
    when 1
    1
    else
    fib(n - 1) + fib(n - 2)
    end
    end
    

    View Slide

35. 2 ARGUMENT-AWARE MEMOIZATION USING ||=
    

    View Slide

36. def fib(n)
    …
    end
    memoize :fib
    

    View Slide

37. def memoize(method_name)
    end
    

    View Slide

38. def memoize(method_name)
    end
    

    View Slide

39. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    end
    

    View Slide

40. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    send(nonmemoized_method_name, *args)
    end
    end
    

    View Slide

41. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    send(nonmemoized_method_name, *args)
    end
    end
    

    View Slide

42. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    @cache ||= {}
    method_cache = (@cache[method_name] ||= {})
    send(nonmemoized_method_name, *args)
    end
    end
    

    View Slide

43. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    @cache ||= {}
    method_cache = (@cache[method_name] ||= {})
    method_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    

    View Slide

44. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    @cache ||= {}
    method_cache = (@cache[method_name] ||= {})
    if method_cache.key?(args)
    method_cache[args]
    else
    method_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    end
    

    View Slide

45. 3 GENERIC MEMOIZATION USING #MEMOIZE
    

    View Slide

46. class Node
    def depth
    (children.map(&:depth).max || 0) + 1
    end
    memoize :depth
    end
    

    View Slide

47. h = Node.new
    g = Node.new(h)
    f = Node.new(g)
    e = Node.new(f)
    d = Node.new(e)
    c = Node.new(d)
    b2 = Node.new(f)
    b1 = Node.new(b2)
    b = Node.new(c, b1)
    a = Node.new(b)

    

    p a.depth

    B
    C
    D
    E
    F
    G
    B1
    B2
    B
    C
    D
    E
    B1
    B2
    

    View Slide

48. h = Node.new
    g = Node.new(h)
    f = Node.new(g)
    e = Node.new(f)
    d = Node.new(e)
    c = Node.new(d)
    b2 = Node.new(f)
    b1 = Node.new(b2)
    b = Node.new(c, b1)
    a = Node.new(b)

    

    a.freeze

    p a.depth
    B
    C
    D
    E
    F
    G
    B1
    B2
    B
    C
    D
    E
    B1
    B2
    

    View Slide

49. node.rb:123:in `depth': can't modify frozen Node (FrozenError)
    

    View Slide

50. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    @cache ||= {}
    method_cache = (@cache[method_name] ||= {})
    if method_cache.key?(args)
    method_cache[args]
    else
    method_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    end
    

    View Slide

51. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    define_method(method_name) do |*args|
    @cache ||= {}
    method_cache = (@cache[method_name] ||= {})
    if method_cache.key?(args)
    method_cache[args]
    else
    method_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    end
    

    View Slide

52. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    method_cache = {}
    define_method(method_name) do |*args|
    instance_cache = (method_cache[self] ||= {})
    if instance_cache.key?(args)
    instance_cache[args]
    else
    instance_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    end
    

    View Slide

53. 4 GENERIC MEMOIZATION SUPPORTING #FREEZE
    

    View Slide

54. We’re running out of memory!
    

    View Slide

55. def memoize(method_name)
    nonmemoized_method_name = '__nonmemoized_' + method_name.to_s
    alias_method nonmemoized_method_name, method_name
    method_cache = {}
    define_method(method_name) do |*args|
    instance_cache = (method_cache[self] ||= {})
    if instance_cache.key?(args)
    instance_cache[args]
    else
    instance_cache[args] = send(nonmemoized_method_name, *args)
    end
    end
    end
    

    View Slide

56. View Slide

57. require 'weakref'

    

    ref = WeakRef.new("hi")

    ref.upcase
    

    View Slide

58. require 'weakref'

    

    ref = WeakRef.new("hi")

    ref.upcase
    # a> HI

    

    View Slide

59. require 'weakref'

    

    ref = WeakRef.new("hi")

    ref.upcase
    # a> HI

    GC.start

    ref.upcase
    

    View Slide

60. require 'weakref'

    

    ref = WeakRef.new("hi")

    ref.upcase
    # a> HI

    GC.start

    ref.upcase
    # WeakRef45RefError (Invalid Reference - probably recycled)
    

    View Slide

61. require 'ref'
    ref = RefnoSoftReference.new("hello")

    ref.object.upcase
    # a> HI
    

    View Slide

62. define_method(method_name) do |*args|
    instance_cache = (method_cache[self] ||= {})
    value = instance_cache[args]&.object
    if value
    value
    else
    instance_cache[args] =
    Ref45SoftReference.new(
    send(original_method_name, *args)
    )
    end
    end
    

    View Slide

63. RefnoSoftReference.new(123)
    # a> ArgumentError (cannot define finalizer for Integer)
    

    View Slide

64. class ValueWrapper
    attr_reader :value
    def initialize(value)
    @value = value
    end
    end
    

    View Slide

65. define_method(method_name) do |*args|
    instance_cache = (method_cache[self] ||= {})
    value = instance_cache[args]&.object
    if wrapper
    wrapper.value
    else
    value = send(original_method_name, *args)
    wrapper = ValueWrapper.new(value)
    instance_cache[args] = RefnoSoftReference.new(wrapper)
    value
    end
    end
    

    View Slide

66. 5 GENERIC MEMORY-EFFICIENT MEMOIZATION
    

    View Slide

67. Thread safety? Nöpe.
    

    View Slide

68. (I’ll get around to that, eventually.)
    

    View Slide

69. EXECUTIVE SUMMARY TIME
    

    View Slide

70. We now have a memoization mechanism that…
    * … has a nice API
    * … supports multiple arguments
    * … supports frozen objects
    * … is memory-efficient
    * … and soon will be thread-safe
    

    View Slide

71. I love immutability.
    

    View Slide

72. You don’t have to implement this yourself, ever.
    Give ddmemoize a try: github.com/ddfreyne/ddmemoize
    

    View Slide

73. require 'ddmemoize'
    class FibFast
    DDMemoize.activate(self)
    memoized def run(n)
    if n == 0
    0
    elsif n == 1
    1
    else
    run(n - 1) + run(n - 2)
    end
    end
    end
    

    View Slide

74. DDMemoize.print_metrics
    memoization │ hit miss %
    ────────────┼───────────────────
    FibFast#fib │ 998 1001 49.9%
    

    View Slide

75. Q&A
    

    View Slide