$30 off During Our Annual Pro Sale. View Details »

Beyond the language (RubyDay 2015)

Beyond the language (RubyDay 2015)

Presented at RubyDay 2015.

Beyond the language - An introduction to algorithms

Mastering a programming language is not enough to write efficient code. Programmers often try to squeeze a programming language as much as they can in order to achieve the best performance, without realizing that the solution could be found in a more efficient algorithm or data structure.

The talk is meant to inspire you to learn more about algorithm design by demonstrating how a properly designed algorithm may drastically increase the efficiency of your code, regardless of the Ruby version you are using.

Simone Carletti

November 13, 2015
Tweet

More Decks by Simone Carletti

Other Decks in Programming

Transcript

  1. Beyond the language
    An introduc2on to algorithms
    Simone Carle7 /
    / @weppos
    RubyDay Italy 2015

    View Slide

  2. Problem
    • There is a coffee table with a lot of
    bignè
    • You want to eat some bignè
    • One bignè is poisoned and it weighs
    a liKle bit more than the others
    • The only way for you to find the
    poisoned one is to weigh the bignè
    using a scale (or with your hands)
    • Each 2me you weigh the bignè you
    waste precious 2me

    View Slide

  3. Goal
    • You want a way

    to eat as much bignè as you can,
    ideally without ea2ng the
    poisoned one and before all the
    good bignè are gone

    View Slide

  4. Goal
    • You want an algorithm

    to eat as much bignè as you can,
    ideally without ea2ng the
    poisoned one and before all the
    good bignè are gone

    View Slide

  5. Simone Carle7
    @weppos

    View Slide

  6. View Slide

  7. View Slide

  8. algorithm |ˈalgərɪð(ə)m|
    a process or set of rules to be followed in calculations or
    other problem-solving operations, especially by a
    computer.

    View Slide

  9. algorithm |ˈalgərɪð(ə)m|
    a word used by programmers when they don't want to
    explain what they did.
    https://twitter.com/kellabyte/status/557736373319643137

    View Slide

  10. An Algorithm is not a Program
    A program can be viewed as an elaborate algorithm.

    View Slide

  11. An Algorithm is not a Program
    Computer programs contain algorithms that detail the
    specific instruc2ons a computer should perform, in a
    specific order, to execute a specific task.

    View Slide

  12. As a programmer,
    you use a programming language
    to implement algorithms
    to write programs.

    View Slide

  13. http://programmers.stackexchange.com/q/18406/3820

    View Slide

  14. Back to the problem…

    View Slide

  15. Problem
    • There is a coffee table with a lot of bignè
    • You want to eat some bignè
    • One bignè is poisoned and it weighs a liKle
    bit more than the others
    • The only way for you to find the poisoned
    one is to weigh the bignè using a scale (or
    with your hands)
    • Each 2me you weigh the bignè you waste
    precious 2me
    • You want to eat as much bignè as you can,
    ideally without ea2ng the poisoned one and
    before all the good bignè are gone

    View Slide

  16. Problem
    Instance
    Computa;on

    model
    Find the heaviest item among n items.
    n items including the item we want to
    find.
    Scale. Each weight taken has a fixed cost.
    Algorithm Weighing strategy.
    Goal We want an algorithm that finds the
    heaviest item in the collec2on.
    The algorithm should be correct and
    efficient.

    View Slide

  17. The simplest algorithm
    Take the first item and compare it to all the other items in
    the collec2on.

    View Slide

  18. The simplest algorithm

    View Slide

  19. The simplest algorithm

    View Slide

  20. The simplest algorithm
    1

    View Slide

  21. The simplest algorithm
    2

    View Slide

  22. The simplest algorithm
    3

    View Slide

  23. The simplest algorithm
    4

    View Slide

  24. The simplest algorithm
    5

    View Slide

  25. The simplest algorithm
    6

    View Slide

  26. The simplest algorithm
    Correct: YES
    Efficient:

    View Slide

  27. Cost of an algorithm

    View Slide

  28. For simplicity, let's define the
    cost of an algorithm as an
    es;ma;on of the resources
    needed by an algorithm to solve
    a computa;on problem.

    View Slide

  29. Resources can be
    CPU-;me, storage, …

    View Slide

  30. The lower the cost,
    The higher the efficiency
    of the algorithm.

    View Slide

  31. In computer science,
    the Big-O nota;on is used to
    classify the cost of an algorithm
    rela;ve to changes in input size.

    View Slide

  32. Big-O nota;on
    is a way to measure the
    complexity of the algorithm
    in the worst-case scenario.

    View Slide

  33. Complexity of an algorithm
    We need some basic Math to understand the

    View Slide

  34. Big-O n = 2 n = 10 n = 20
    constant O(1) 1 1 1
    linear O(n) 2 10 20
    logarithmic O(log2n) 1 ~ 3,3 ~ 4,3
    quadra;c O(n2) 4 100 400
    exponen;al O(2n) 4 1.024 1.048.576

    View Slide

  35. http://bigocheatsheet.com/

    View Slide

  36. Let's analyze the
    1st algorithm

    View Slide

  37. 1st algorithm: "one vs all"
    Take the first item and compare it to all the other items in
    the collec2on.

    View Slide

  38. 1st algoritm
    Correct: YES
    Cost:
    n

    O(n)
    Efficient: It depends…

    View Slide

  39. 2nd algorithm: "each pair"
    Compare a pair of items at 2me.

    View Slide

  40. 2nd algorithm

    View Slide

  41. 2nd algorithm
    1

    View Slide

  42. 2nd algorithm
    2

    View Slide

  43. 2nd algorithm
    3

    View Slide

  44. 2nd algorithm
    4

    View Slide

  45. 2nd algoritm
    Correct: YES
    Cost:
    n/2
    O(n)
    Efficient: It depends…

    View Slide

  46. 3rd algorithm: "divide et pesa"
    Split the collec2on of items in two sets with the same
    number of items, and weigh them. Keep the heaviest set
    and discard eat the other. (*)
    Split the new collec2on of items in two sets and iterate.
    (*) If the # of items is odd, take out the last element. If the two sets have the
    same weight, the last element is the result. Otherwise discard the element
    along with the non-relevant set.

    View Slide

  47. 3rd algorithm

    View Slide

  48. 3rd algorithm
    1

    View Slide

  49. 3rd algorithm
    2

    View Slide

  50. 3rd algorithm
    3

    View Slide

  51. 3rd algoritm
    Correct: YES
    Cost:
    log2
    n

    O(log n)
    Efficient: It depends…

    View Slide

  52. 4th algorithm: "divide /3 et pesa"
    Split the collec2on of items in 3 sets, and weigh the ones
    with the same number of elements.
    If the two sets have the same weight, discard both and
    con2nue recursively on the third. Otherwise, keep the
    heaviest and con2nue recursively.

    View Slide

  53. 4th algorithm

    View Slide

  54. 4th algorithm
    1

    View Slide

  55. 4th algorithm
    2

    View Slide

  56. 4th algoritm
    Correct: YES
    Cost:
    log3
    n

    O(log n)
    Efficient:

    View Slide

  57. n 10 100 1.000 10.000
    1st 9m 1h 39m 16h 6d
    2nd 5m 50m 8h 3,5d
    3rd 3m 6m 9m 13m
    4th 3m 5m 7m 9m

    View Slide

  58. Language vs Algorithm
    Shouldn't I simply use a faster language?

    View Slide

  59. Data Structures
    Ruby Hash Table

    View Slide

  60. rubyists = {}
    # insert
    rubyists[:weppos] = "Simone Carletti"
    rubyists[:jodosha] = "Luca Guidi"
    rubyists[:john] = "John Doe"
    # delete
    rubyists.delete(:john)
    # search
    rubyists[:weppos] # => "Simone Carletti"

    View Slide

  61. all opera;ons should cost O(1)
    In an ideal world

    View Slide

  62. key1: "value1"
    key2: "value2"
    key3: "value3"
    Linked list

    View Slide

  63. key1: "value1"
    key2: "value2"
    key3: "value3"
    The search costs O(n)!
    Linked list

    View Slide

  64. search (ruby 2.2)
    10.000 itera;ons First Key Last Key
    4 0,00305 0,00282
    8 0,00417 0,00318
    16 0,00391 0,00359
    32 0,00476 0,00399
    64 0,00335 0,00275
    128 0,00447 0,00366
    256 0,00435 0,00342
    512 0,00325 0,00439
    1024 0,00317 0,00300
    2048 0,00413 0,00334
    4096 0,00490 0,00370
    8192 0,00342 0,00452
    16384 0,00522 0,00302
    32768 0,00367 0,00408
    0,01
    0,1
    4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768

    View Slide

  65. Hash Table in Ruby
    16
    #define ST_DEFAULT_MAX_DENSITY 5
    #define ST_DEFAULT_INIT_TABLE_SIZE 16
    #define ST_DEFAULT_PACKED_TABLE_SIZE 18
    num_bins

    View Slide

  66. Hash Table in Ruby
    16 whatever[:key1] = "value"

    View Slide

  67. Hash Table in Ruby
    16 whatever[:key1] = "value"
    :key1.hash
    # => 707933195402600884
    707933195402600884 % 16
    # => 4

    View Slide

  68. Hash Table in Ruby
    16 whatever[:key1] = "value"
    key1

    View Slide

  69. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    key1 key2

    View Slide

  70. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    key3 key2
    key1

    View Slide

  71. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    whatever[:key4] = "value"
    whatever[:key5] = "value"
    whatever[:key6] = "value"
    whatever[:key7] = "value"
    whatever[:key8] = "value"
    whatever[:key9] = "value"
    key9 key2
    key3
    key1
    key5 key4
    key8
    key6
    key7

    View Slide

  72. Hash Table in Ruby
    16 whatever[:key1] = "value"
    whatever[:key2] = "value"
    whatever[:key3] = "value"
    whatever[:key4] = "value"
    whatever[:key5] = "value"
    whatever[:key6] = "value"
    whatever[:key7] = "value"
    ...
    whatever[:keyN] = "value"
    #define ST_DEFAULT_MAX_DENSITY 5
    #define ST_DEFAULT_INIT_TABLE_SIZE 16
    #define ST_DEFAULT_PACKED_TABLE_SIZE 18

    View Slide

  73. Hash Table in Ruby
    17

    View Slide

  74. Hash Table in Ruby
    17 static void
    rehash(register st_table *table)
    {
    ...
    }

    View Slide

  75. Hash Table in Ruby
    17

    View Slide

  76. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]

    View Slide

  77. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5

    View Slide

  78. Hash Table in Ruby
    17
    keyN
    # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5
    5

    View Slide

  79. Hash Table in Ruby
    17 # search
    whatever[:keyN]
    :keyN.hash
    # => 2149668665177381990
    2149668665177381990 % 17
    # => 5
    5
    keyN

    View Slide

  80. Object#hash
    class BadKey
    def hash
    3
    end
    end
    class GoodKey
    end

    View Slide

  81. Lookup time with BadKey
    10.000 itera;ons First Key Last Key
    4 0,00362 0,00580
    8 0,00587 0,00397
    16 0,00851 0,00442
    32 0,0182 0,00300
    64 0,02397 0,00295
    128 0,05332 0,00348
    256 0,09378 0,00476
    512 0,2033 0,00294
    1024 0,4146 0,00297
    2048 0,92197 0,00453
    4096 1,58056 0,00289
    8192 3,323 0,00306
    16384 6,66864 0,00414
    32768 14,44931 0,00324
    Lookup time with GoodKey
    10.000 itera;ons First Key Last Key
    4 0,00305 0,00282
    8 0,00417 0,00318
    16 0,00391 0,00359
    32 0,00476 0,00399
    64 0,00335 0,00275
    128 0,00447 0,00366
    256 0,00435 0,00342
    512 0,00325 0,00439
    1024 0,00317 0,00300
    2048 0,00413 0,00334
    4096 0,00490 0,00370
    8192 0,00342 0,00452
    16384 0,00522 0,00302
    32768 0,00367 0,00408
    0,01
    0,1
    1
    10
    100
    4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768
    Bad key Good Key

    View Slide

  82. a good

    View Slide

  83. http://amzn.to/1LI4VAj http://amzn.to/1LI5bPC

    View Slide

  84. Thanks to
    Prof. Luciano Gualà
    Department of Mathema2cs

    of the University of Rome "Tor Vergata".
    Pat Shaughnessy
    Author of Ruby Under a Microscope

    View Slide

  85. Photos
    hKps:/
    /www.flickr.com/photos/stevefrazier/21397889609/
    hKps:/
    /www.flickr.com/photos/chanmelmel/15546712170/

    View Slide

  86. Thanks!
    Simone Carle7
    @weppos

    View Slide