Beyond the language (RubyDay 2015)

Presented at RubyDay 2015.

Beyond the language - An introduction to algorithms

Mastering a programming language is not enough to write efficient code. Programmers often try to squeeze a programming language as much as they can in order to achieve the best performance, without realizing that the solution could be found in a more efficient algorithm or data structure.

The talk is meant to inspire you to learn more about algorithm design by demonstrating how a properly designed algorithm may drastically increase the efficiency of your code, regardless of the Ruby version you are using.

Transcript

1. Beyond the language
   An introduc2on to algorithms
   Simone Carle7 /
   / @weppos
   RubyDay Italy 2015
   

   View Slide

2. Problem
   • There is a coffee table with a lot of
   bignè
   • You want to eat some bignè
   • One bignè is poisoned and it weighs
   a liKle bit more than the others
   • The only way for you to find the
   poisoned one is to weigh the bignè
   using a scale (or with your hands)
   • Each 2me you weigh the bignè you
   waste precious 2me
   

   View Slide

3. Goal
   • You want a way

   to eat as much bignè as you can,
   ideally without ea2ng the
   poisoned one and before all the
   good bignè are gone
   

   View Slide

4. Goal
   • You want an algorithm

   to eat as much bignè as you can,
   ideally without ea2ng the
   poisoned one and before all the
   good bignè are gone
   

   View Slide

5. Simone Carle7
   @weppos
   

   View Slide

6. View Slide

7. View Slide

8. algorithm |ˈalgərɪð(ə)m|
   a process or set of rules to be followed in calculations or
   other problem-solving operations, especially by a
   computer.
   

   View Slide

9. algorithm |ˈalgərɪð(ə)m|
   a word used by programmers when they don't want to
   explain what they did.
   https://twitter.com/kellabyte/status/557736373319643137
   

   View Slide

10. An Algorithm is not a Program
    A program can be viewed as an elaborate algorithm.
    

    View Slide

11. An Algorithm is not a Program
    Computer programs contain algorithms that detail the
    specific instruc2ons a computer should perform, in a
    specific order, to execute a specific task.
    

    View Slide

12. As a programmer,
    you use a programming language
    to implement algorithms
    to write programs.
    

    View Slide

13. http://programmers.stackexchange.com/q/18406/3820
    

    View Slide

14. Back to the problem…
    

    View Slide

15. Problem
    • There is a coffee table with a lot of bignè
    • You want to eat some bignè
    • One bignè is poisoned and it weighs a liKle
    bit more than the others
    • The only way for you to find the poisoned
    one is to weigh the bignè using a scale (or
    with your hands)
    • Each 2me you weigh the bignè you waste
    precious 2me
    • You want to eat as much bignè as you can,
    ideally without ea2ng the poisoned one and
    before all the good bignè are gone
    

    View Slide

16. Problem
    Instance
    Computa;on

    model
    Find the heaviest item among n items.
    n items including the item we want to
    find.
    Scale. Each weight taken has a fixed cost.
    Algorithm Weighing strategy.
    Goal We want an algorithm that finds the
    heaviest item in the collec2on.
    The algorithm should be correct and
    efficient.
    

    View Slide

17. The simplest algorithm
    Take the first item and compare it to all the other items in
    the collec2on.
    

    View Slide

18. The simplest algorithm
    

    View Slide

19. The simplest algorithm
    

    View Slide

20. The simplest algorithm
    1
    

    View Slide

21. The simplest algorithm
    2
    

    View Slide

22. The simplest algorithm
    3
    

    View Slide

23. The simplest algorithm
    4
    

    View Slide

24. The simplest algorithm
    5
    

    View Slide

25. The simplest algorithm
    6
    

    View Slide

26. The simplest algorithm
    Correct: YES
    Efficient:
    

    View Slide

27. Cost of an algorithm
    

    View Slide

28. For simplicity, let's define the
    cost of an algorithm as an
    es;ma;on of the resources
    needed by an algorithm to solve
    a computa;on problem.
    

    View Slide

29. Resources can be
    CPU-;me, storage, …
    

    View Slide

30. The lower the cost,
    The higher the efficiency
    of the algorithm.
    

    View Slide

31. In computer science,
    the Big-O nota;on is used to
    classify the cost of an algorithm
    rela;ve to changes in input size.
    

    View Slide

32. Big-O nota;on
    is a way to measure the
    complexity of the algorithm
    in the worst-case scenario.
    

    View Slide

33. Complexity of an algorithm
    We need some basic Math to understand the
    

    View Slide

34. Big-O n = 2 n = 10 n = 20
    constant O(1) 1 1 1
    linear O(n) 2 10 20
    logarithmic O(log2n) 1 ~ 3,3 ~ 4,3
    quadra;c O(n2) 4 100 400
    exponen;al O(2n) 4 1.024 1.048.576
    

    View Slide

35. http://bigocheatsheet.com/
    

    View Slide

36. Let's analyze the
    1st algorithm
    

    View Slide

37. 1st algorithm: "one vs all"
    Take the first item and compare it to all the other items in
    the collec2on.
    

    View Slide

38. 1st algoritm
    Correct: YES
    Cost:
    n

    O(n)
    Efficient: It depends…
    

    View Slide

39. 2nd algorithm: "each pair"
    Compare a pair of items at 2me.
    

    View Slide

40. 2nd algorithm
    

    View Slide

41. 2nd algorithm
    1
    

    View Slide

42. 2nd algorithm
    2
    

    View Slide

43. 2nd algorithm
    3
    

    View Slide

44. 2nd algorithm
    4
    

    View Slide

45. 2nd algoritm
    Correct: YES
    Cost:
    n/2
    O(n)
    Efficient: It depends…
    

    View Slide

46. 3rd algorithm: "divide et pesa"
    Split the collec2on of items in two sets with the same
    number of items, and weigh them. Keep the heaviest set
    and discard eat the other. (*)
    Split the new collec2on of items in two sets and iterate.
    (*) If the # of items is odd, take out the last element. If the two sets have the
    same weight, the last element is the result. Otherwise discard the element
    along with the non-relevant set.
    

    View Slide

47. 3rd algorithm
    

    View Slide

48. 3rd algorithm
    1
    

    View Slide

49. 3rd algorithm
    2
    

    View Slide

50. 3rd algorithm
    3
    

    View Slide

51. 3rd algoritm
    Correct: YES
    Cost:
    log2
    n

    O(log n)
    Efficient: It depends…
    

    View Slide

52. 4th algorithm: "divide /3 et pesa"
    Split the collec2on of items in 3 sets, and weigh the ones
    with the same number of elements.
    If the two sets have the same weight, discard both and
    con2nue recursively on the third. Otherwise, keep the
    heaviest and con2nue recursively.
    

    View Slide

53. 4th algorithm
    

    View Slide

54. 4th algorithm
    1
    

    View Slide

55. 4th algorithm
    2
    

    View Slide

56. 4th algoritm
    Correct: YES
    Cost:
    log3
    n

    O(log n)
    Efficient:
    

    View Slide

57. n 10 100 1.000 10.000
    1st 9m 1h 39m 16h 6d
    2nd 5m 50m 8h 3,5d
    3rd 3m 6m 9m 13m
    4th 3m 5m 7m 9m
    

    View Slide

58. Language vs Algorithm
    Shouldn't I simply use a faster language?
    

    View Slide

59. Data Structures
    Ruby Hash Table
    

    View Slide

60. rubyists = {}
    # insert
    rubyists[:weppos] = "Simone Carletti"
    rubyists[:jodosha] = "Luca Guidi"
    rubyists[:john] = "John Doe"
    # delete
    rubyists.delete(:john)
    # search
    rubyists[:weppos] # => "Simone Carletti"
    

    View Slide

61. all opera;ons should cost O(1)
    In an ideal world
    

    View Slide

62. key1: "value1"
    key2: "value2"
    key3: "value3"
    Linked list
    

    View Slide

63. key1: "value1"
    key2: "value2"
    key3: "value3"
    The search costs O(n)!
    Linked list
    

    View Slide

64. search (ruby 2.2)
    10.000 itera;ons First Key Last Key
    4 0,00305 0,00282
    8 0,00417 0,00318
    16 0,00391 0,00359
    32 0,00476 0,00399
    64 0,00335 0,00275
    128 0,00447 0,00366
    256 0,00435 0,00342
    512 0,00325 0,00439
    1024 0,00317 0,00300
    2048 0,00413 0,00334
    4096 0,00490 0,00370
    8192 0,00342 0,00452
    16384 0,00522 0,00302
    32768 0,00367 0,00408
    0,01
    0,1
    4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768
    

    View Slide

65. Hash Table in Ruby
    16
    #define ST_DEFAULT_MAX_DENSITY 5
    #define ST_DEFAULT_INIT_TABLE_SIZE 16
    #define ST_DEFAULT_PACKED_TABLE_SIZE 18
    num_bins
    

    View Slide

66. Hash Table in Ruby
    16 whatever[:key1] = "value"
    

    View Slide

67. Hash Table in Ruby
    16 whatever[:key1] = "value"
    :key1.hash
    # => 707933195402600884
    707933195402600884 % 16
    # => 4
    

    View Slide

68. Hash Table in Ruby
    16 whatever[:key1] = "value"
    key1
    

    View Slide

69. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    key1 key2
    

    View Slide

70. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    key3 key2
    key1
    

    View Slide

71. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    whatever[:key4] = "value"
    whatever[:key5] = "value"
    whatever[:key6] = "value"
    whatever[:key7] = "value"
    whatever[:key8] = "value"
    whatever[:key9] = "value"
    key9 key2
    key3
    key1
    key5 key4
    key8
    key6
    key7
    

    View Slide

72. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    whatever[:key4] = "value"
    whatever[:key5] = "value"
    whatever[:key6] = "value"
    whatever[:key7] = "value"
    ...
    whatever[:keyN] = "value"
    #define ST_DEFAULT_MAX_DENSITY 5
    #define ST_DEFAULT_INIT_TABLE_SIZE 16
    #define ST_DEFAULT_PACKED_TABLE_SIZE 18
    

    View Slide

73. Hash Table in Ruby
    17
    

    View Slide

74. Hash Table in Ruby
    17 static void
    rehash(register st_table *table)
    {
    ...
    }
    

    View Slide

75. Hash Table in Ruby
    17
    

    View Slide

76. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]
    

    View Slide

77. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5
    

    View Slide

78. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5
    5
    

    View Slide

79. Hash Table in Ruby
    17 # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5
    5
    keyN
    

    View Slide

80. Object#hash
    class BadKey
    def hash
    3
    end
    end
    class GoodKey
    end
    

    View Slide

81. Lookup time with BadKey
    10.000 itera;ons First Key Last Key
    4 0,00362 0,00580
    8 0,00587 0,00397
    16 0,00851 0,00442
    32 0,0182 0,00300
    64 0,02397 0,00295
    128 0,05332 0,00348
    256 0,09378 0,00476
    512 0,2033 0,00294
    1024 0,4146 0,00297
    2048 0,92197 0,00453
    4096 1,58056 0,00289
    8192 3,323 0,00306
    16384 6,66864 0,00414
    32768 14,44931 0,00324
    Lookup time with GoodKey
    10.000 itera;ons First Key Last Key
    4 0,00305 0,00282
    8 0,00417 0,00318
    16 0,00391 0,00359
    32 0,00476 0,00399
    64 0,00335 0,00275
    128 0,00447 0,00366
    256 0,00435 0,00342
    512 0,00325 0,00439
    1024 0,00317 0,00300
    2048 0,00413 0,00334
    4096 0,00490 0,00370
    8192 0,00342 0,00452
    16384 0,00522 0,00302
    32768 0,00367 0,00408
    0,01
    0,1
    1
    10
    100
    4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768
    Bad key Good Key
    

    View Slide

82. a good
    

    View Slide

83. http://amzn.to/1LI4VAj http://amzn.to/1LI5bPC
    

    View Slide

84. Thanks to
    Prof. Luciano Gualà
    Department of Mathema2cs

    of the University of Rome "Tor Vergata".
    Pat Shaughnessy
    Author of Ruby Under a Microscope
    

    View Slide

85. Photos
    hKps:/
    /www.flickr.com/photos/stevefrazier/21397889609/
    hKps:/
    /www.flickr.com/photos/chanmelmel/15546712170/
    

    View Slide

86. Thanks!
    Simone Carle7
    @weppos
    

    View Slide