Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Gradient Descent Easy

Gradient Descent Easy

Easy/brief version of Gradient Descent from Artificial Intelligence Lecture

soulchild

July 23, 2014
Tweet

More Decks by soulchild

Other Decks in Science

Transcript

  1. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 Minimum point
  2. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 Minimum point How does a computer know that this is a minimum point?
  3. Gradient Descent Ans : By brute-forcing the derivative until a

    value equal or near to 0 is found y = x2 dy dx = 2x Then guess x by starting from, eg: -6 to 6
  4. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 dy dx = -12
  5. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8
  6. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4
  7. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0
  8. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0 Minimum point found, stop
  9. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 In short, it works like this
  10. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 In short, it works like this
  11. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 In short, it works like this
  12. Gradient Descent y 0 10 20 30 40 x -6

    -4 -2 0 2 4 6 y = x2 In short, it works like this
  13. Gradient Descent When to stop searching ? Set a maximum

    number of iteration dy dx < n When n can be 0.1, 0.01,.. etc
  14. Gradient Descent Weakness of Gradient Descent 0 3 6 9

    12 -8 -6 -4 -2 0 2 4 let say start from here
  15. Gradient Descent Weakness of Gradient Descent 0 3 6 9

    12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0
  16. Gradient Descent Weakness of Gradient Descent 0 3 6 9

    12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding
  17. Gradient Descent Weakness of Gradient Descent 0 3 6 9

    12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding What about this?! smaller than previous point wor
  18. Gradient Descent Weakness of Gradient Descent Gradient Descent may stuck

    in a local minima thus can’t find the global minima