Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Algorithms to live by and why should we care

Algorithms to live by and why should we care

Presented at Full Stack Toronto Conference

Elle Meredith

October 23, 2017
Tweet

More Decks by Elle Meredith

Other Decks in Programming

Transcript

  1. Algorithms to live by
    Elle Meredith
    @aemeredith

    View Slide

  2. Algorithms
    1
    2
    3a 3b
    = Step by step
    instructions

    View Slide

  3. https://www.instagram.com/p/BaesTAPFaEK2_n5QA06hO7w3Nwd1iaoCS0KIL40/

    View Slide

  4. View Slide

  5. View Slide

  6. Algorithm
    Detailed

    View Slide

  7. Algorithm
    Detailed Efficiency
    Perfection

    View Slide

  8. https://imgur.com/Xz3Z2iL

    View Slide

  9. In everyday life
    • Learnt
    • Figure out ourselves
    • Require written instructions

    View Slide

  10. A precise,
    systematic
    method for
    producing a
    specified result
    Definition

    View Slide

  11. Why?

    View Slide

  12. Suppose we want to
    search for a word in
    the dictionary
    Binary Search

    View Slide

  13. 1 2 3 4 100

    View Slide

  14. 1 2 3 4 100

    X
    Too low

    View Slide

  15. 1 2 3 4 100

    XX
    Too low

    View Slide

  16. 1 2 3 4 100

    XXX
    Too low

    View Slide

  17. 1 2 3 4 100

    XXXX
    Too low

    View Slide

  18. These are all too low
    1
    50 100
    Too low

    View Slide

  19. Eliminated 25 more
    75
    51 100
    Too high

    View Slide

  20. And we eliminated some more
    51
    63 74
    Too low

    View Slide

  21. 7 STEPS
    100 50 25 13
    7
    4
    2
    1

    View Slide

  22. 10 STEPS
    1000 -> 500 -> 250
    -> 125 -> 63 -> 32
    -> 16 -> 8 -> 4
    -> 2 -> 1

    View Slide

  23. 17 STEPS
    100,000 -> 50,000 -> 25,000
    -> 12,500 -> 6,300 -> 3,150
    -> 1,575 -> 788 -> 394
    -> 197 -> 99 -> 50 -> 25
    -> 13 -> 7 -> 4 -> 2 -> 1

    View Slide

  24. 22 = 4
    23 = 8
    24 = 16
    25 = 32
    26 = 64

    View Slide

  25. 22 = 4
    23 = 8
    24 = 16
    25 = 32
    26 = 64
    log2
    4 = 2
    log2
    8 = 3
    log2
    100 => 6.643
    log2
    100000 => 16.609

    View Slide

  26. * Binary search only
    works when our list
    is sorted

    View Slide

  27. Searching for a new
    place to live…
    Optimal stopping

    View Slide

  28. or finding a significant
    other

    View Slide

  29. The secretary problem

    View Slide

  30. https://giphy.com/gifs/scooby-doo-wfOe7SdZ3XyHm

    View Slide

  31. http://gph.is/15twRiZ

    View Slide

  32. 37%

    View Slide

  33. * When we don’t
    know all the options,
    optimal stopping
    tells us when to stop
    and make a decision

    View Slide

  34. Digging at grandma’s
    attic
    Recursion

    View Slide

  35. View Slide

  36. box
    box
    container box

    View Slide

  37. Make a pile
    of boxes
    while the pile
    is not empty
    Grab
    a box
    if you find a box,
    add it to the
    pile of boxes
    Go back to
    the pile
    if you find
    a diary,
    you’re done!

    View Slide

  38. Go through
    each item in the box
    if you find a box…
    if you find a diary,
    you’re done!

    View Slide

  39. View Slide

  40. def factorial(x)
    if x == 1
    1
    else
    x * factorial(x-1)
    end
    end

    View Slide

  41. def factorial(x)
    if x == 1
    1
    else
    x * factorial(x-1)
    end
    end

    View Slide

  42. def factorial(x)
    if x == 1
    1
    else
    x * factorial(x-1)
    end
    end

    View Slide

  43. factorial(4) = 4 * factorial(3)
    factorial(3) = 3 * factorial(2)
    factorial(2) = 2 * factorial(1)
    factorial(1) = 1

    View Slide

  44. factorial(4) = 4 * factorial(3)
    factorial(3) = 3 * factorial(2)
    factorial(2) = 2 * factorial(1)
    factorial(1) = 1
    4 * 3 * 2 * 1 = 24

    View Slide

  45. * Recursion can be
    applied whenever a
    problem can be
    solved by dividing it
    into smaller
    problems

    View Slide

  46. * … and needs a
    recursion case and a
    base case

    View Slide

  47. Sorting a book shelf
    Sorting

    View Slide

  48. Bubble sort
    https://giphy.com/gifs/foxhomeent-book-books-3o7btW1Js39uJ23LAA

    View Slide

  49. Insertion sort
    https://giphy.com/gifs/atcqQ5PuX41J6

    View Slide

  50. https://imgur.com/Xz3Z2iL
    Merge sort

    View Slide

  51. empty array
    array with
    one element
    No need to sort
    these arrays
    33
    Quicksort

    View Slide

  52. check if first element
    is small than
    the second one,
    and if it isn’t => switch
    4 2

    View Slide

  53. pivot
    5 2 4 1
    3
    3

    View Slide

  54. 3
    2 1 5 4

    View Slide

  55. 3
    2 1 5 4
    qsort( ) qsort( )

    View Slide

  56. 3
    2
    1 5
    4
    +
    +
    3
    2
    1 5
    4

    View Slide

  57. * Should we be
    sorting at all?
    https://people.ucsc.edu/~swhittak/papers/chi2011_refinding_email_camera_ready.pdf

    View Slide

  58. Getting things done
    Single machine scheduling

    View Slide

  59. There’s nothing
    so fatiguing as
    the eternal
    hanging on of an
    uncompleted
    task
    William James

    View Slide

  60. Make goals explicit

    View Slide

  61. Strategy: earliest due
    date

    View Slide

  62. https://giphy.com/gifs/nickelodeon-animation-nick-nicktoons-3o7TKTc8NHnZrVFlFm

    View Slide

  63. Strategy: Moore’s
    algorithm

    View Slide

  64. http://gifsgallery.com/watermelon+animated+gif?image=323981005

    View Slide

  65. Strategy: shortest
    processing time

    View Slide

  66. Client 1: 4 days task
    Client 2: 1 day task
    = 5 days of work

    View Slide

  67. Client 1: 4 days task = 4 days waiting
    Client 2: 1 day task = 5 days waiting
    = 9 days of waiting

    View Slide

  68. Client 2: 1 day task = 1 days waiting
    Client 1: 4 days task = 5 days waiting
    = 6 days of waiting

    View Slide

  69. Shortest processing time
    Client 2: 1 day task = 1 days waiting
    Client 1: 4 days task = 5 days waiting
    = 6 days of waiting
    Metric: sum of completion times

    View Slide

  70. Suppose we want to
    find a magician
    Breadth first search

    View Slide

  71. Node Node
    Edge

    View Slide

  72. Elle
    Hannah Caleb Lachlan
    Keith
    Schneem
    Michelle

    View Slide

  73. Elle
    Hannah Caleb Lachlan
    Keith
    Schneem
    Michelle

    View Slide

  74. https://vimeo.com/90177460

    View Slide

  75. Elle
    Hannah Caleb Lachlan
    Keith
    Schneem
    Michelle

    View Slide

  76. Elle
    Hannah Caleb Lachlan
    Keith
    Schneem
    Michelle

    View Slide

  77. graph = {
    "elle"=>["hannah", "caleb", "lachlan"],
    "hannah"=>["michelle", "schneem"],
    "caleb"=>["schneem"],
    "lachlan"=>["keith"],
    "michelle"=>[],
    "schneem"=>[],
    "keith"=>[]
    }

    View Slide

  78. graph = {
    "elle"=>["hannah", "caleb", "lachlan"],
    "hannah"=>["michelle", "schneem"],
    "caleb"=>["schneem"],
    "lachlan"=>["keith"],
    "michelle"=>[],
    "schneem"=>[],
    "keith"=>[]
    }

    View Slide

  79. * Breadth first search
    works only we
    search in the same
    order in which the
    people (nodes) were
    added

    View Slide

  80. Travelling salesperson

    View Slide

  81. Melbourne
    Geelong
    Ballarat
    Frankston
    Kew
    Eltham
    Epping

    View Slide

  82. Melbourne
    Geelong
    Ballarat
    Frankston
    Kew
    Eltham
    Epping

    View Slide

  83. Melbourne
    Geelong
    Ballarat
    Frankston
    Kew
    Eltham
    Epping

    View Slide

  84. * Just relax! by
    relaxing the
    constraints, we make
    it easier to find
    solutions

    View Slide

  85. Building a
    recommendation
    system
    K nearest neighbours

    View Slide

  86. A (2,1)
    B (1,3)
    C (5,5)

    View Slide

  87. A (2,1)
    B (1,3)
    X
    Y
    (X1
    -X2
    )2 + (Y1
    -Y2
    )2
    Distance between A to B
    C (5,5)

    View Slide

  88. (1-3)2 + (2-
    1)2
    A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    Distance between A to B

    View Slide

  89. (1-3)2 + (2-
    1)2
    A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    22 + 12
    Distance between A to B

    View Slide

  90. (1-3)2 + (2-
    1)2
    A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    22 + 12
    4 + 1
    K = 2.236
    Distance between A to B

    View Slide

  91. (5-3)2 + (5-
    1)2
    A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    Distance between C to B

    View Slide

  92. A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    22 + 42
    Distance between C to B
    (5-3)2 + (5-
    1)2

    View Slide

  93. A (2,1)
    C (5,5)
    B (1,3)
    X
    Y
    22 + 42
    Distance between C to B
    (5-3)2 + (5-
    1)2
    4 + 16
    K = 4.
    472

    View Slide

  94. Comedy
    Action
    Drama
    Horror
    Romance
    4
    4
    5
    1
    1
    5
    5
    3
    2
    1
    2
    1
    5
    3
    5

    View Slide

  95. hannah => (4, 4, 5, 1, 1)
    caleb => (5, 5, 3, 2, 1)
    lachlan => (2, 1, 5, 3, 5)

    View Slide

  96. (4-5)2 + (4-5)2 + (5-3)2 + (1-2)2 + (1-
    1)2
    hannah => (4, 4, 5, 1, 1)
    caleb => (5, 5, 3, 2, 1)

    View Slide

  97. 1 + 1 + 4 + 1 + 0
    7
    K = 2.64

    View Slide

  98. (4-2)2 + (4-
    1)2 + (5-5)2 + (1-3)2 + (1-5)2
    hannah => (4, 4, 5, 1, 1)
    lachlan => (2, 1, 5, 3, 5)

    View Slide

  99. 4 + 9 + 0 + 4 + 16
    33
    K = 5.74

    View Slide

  100. * K-Nearest Neighbours
    uses feature extraction,
    which means converting
    an item into a list of
    numbers that can be
    compared

    View Slide

  101. Thinking less
    Overfitting

    View Slide

  102. https://www.zmescience.com/other/charles-darwin-marry-or-not/
    It being proved necessary to marry

    View Slide

  103. The case against
    complexity

    View Slide

  104. If you can’t
    explain it simply,
    you don’t
    understand it
    well enough.
    Anonymous

    View Slide

  105. Strategies
    • Regularisation

    View Slide

  106. Strategies
    • Regularisation
    • Add weight to points

    View Slide

  107. Strategies
    • Regularisation
    • Add weight to points
    • Early stopping

    View Slide

  108. Strategies
    • Regularisation
    • Add weight to points
    • Early stopping
    • Stay clear from finer details

    View Slide

  109. It is intolerable to
    think of spending
    one’s whole life
    like a neuter bee,
    working, working,
    and nothing after
    all.
    Charles Darwin

    View Slide

  110. When algorithms
    go wrong
    https://www.bloomberg.com/view/articles/2017-04-18/united-airlines-exposes-our-twisted-idea-of-dignity

    View Slide

  111. https://en.wikipedia.org/wiki/United_Express_Flight_3411_incident

    View Slide

  112. Every algorithm
    reflects the
    subjective
    choices of its
    human designer
    Cathy O’Neil

    View Slide

  113. Elle Meredith @aemeredith

    View Slide