Intro to Machine Learning

March 15, 2018 @ Code BEAM SF

Transcript

1. Intro to Machine Learning
3. Intro to Machine Learning Sometime in the late 1990s...
4. Calculus Intro to Machine Learning Racquet Sports Pick one:
5. Calculus Intro to Machine Learning Racquet Sports Pick one: X
6. Intro to Machine Learning
7. Intro to Machine Learning Math is just a language.
8. Machine Learning What is learning?
9. to acquire knowledge or skill What is learning? Machine Learning through study, instruction, or experience
10. What is learning? Machine Learning to acquire knowledge or skill through experience
11. Learning is reducing error Machine Learning
12. Simple Neural Network Movie Recommendations
13. Simple Neural Network Star Wars: The Last Jedi
14. Simple Neural Network 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com
15. Simple Neural Network 90% Simple Neural Network
16. Simple Neural Network 90%
17. Simple Neural Network 90 90 input
18. 90 90 Simple Neural Network input
19. 90 90 Simple Neural Network input output
20. 90 90 Simple Neural Network input output 90 90 |> simple() def simple(input), do: input
21. 90 90 Simple Neural Network input output 90 90 |> simple() def simple(input), do: input
22. 90 90 Simple Neural Network 0.01 input output 0.9 weight
23. 0.01 0.9 90 Simple Neural Network input output 90 weight
24. 0.01 Simple Neural Network weight 0.9 90 input output 0.9 90 |> simple(0.01) def simple(input, weight), do: input * weight
25. 0.01 Simple Neural Network weight 0.9 90 input output 0.9 90 |> simple(0.01) def simple(input, weight), do: input * weight
26. 0.01 Simple Neural Network 0.9 90
27. 0.01 Multiple Output Neural Network 0.9 90 recommended box office oscar odds
28. 0.01 0.9 recommended 0.002 500_000 90 Multiple Output Neural Network $45m 0.18 box office oscar odds
29. 0.01 0.9 recommended box office 0.002 500_000 90 Multiple Output Neural Network $45m 0.18 oscar odds
30. 0.01 0.9 recommended box office oscar odds 0.002 500_000 90 Multiple Output Neural Network $45m 0.18
31. 0.01 0.9 $45m 0.18 500_000 0.002 recommended box office oscar odds 90 90 90 Multiple Output Neural Network
32. 0.01 0.9 $45m 0.18 500_000 0.002 recommended box office oscar odds 90 90 90 Multiple Output Neural Network
33. 0.01 0.9 $45m 0.18 500_000 0.002 recommended box office oscar odds 90 90 90 90 |> mult_outputs([0.01, 500_000, 0.002]) [0.9, 45_000_000, 0.18] def mult_outputs(input, list_of_weights) do Enum.map(list_of_weights, fn weight -> input * weight end) end Multiple Output Neural Network
34. recommended box office oscar odds 90 |> mult_outputs([0.01, 500_000, 0.002]) [0.9, 45_000_000, 0.18] def mult_outputs(input, list_of_weights) do Enum.map(list_of_weights, fn weight -> input * weight end) end Multiple Output Neural Network
35. 90 |> mult_outputs([0.01, 500_000, 0.002]) [0.9, 45_000_000, 0.18] Multiple Output Neural Network
36. oscar odds box office recommended |> mult_outputs([ , , ]) [ , , ] 0.18 _000_000 m 45 $ 0.9 0.002 500_000 0.01 90 Multiple Output Neural Network
37. oscar odds box office recommended 0.18 _000_0 m 45 $ 0.9 0.002 500_000 0.01 90 Multiple Output Neural Network %
38. Simple Neural Network 90% Input Data
39. 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com 48% audience rottentomatoes.com Input Data
40. 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com 48% audience rottentomatoes.com Input Data
41. 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com 48% audience rottentomatoes.com director ? Rian Johnson Input Data
42. 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com 48% audience rottentomatoes.com Rian Johnson director ? true Input Data directed by
43. 90% tomatometer Star Wars: The Last Jedi rottentomatoes.com 48% audience rottentomatoes.com Rian Johnson 1 ? ? Input Data directed by
44. 90% 48 % 1 Input Data
45. ? 90 Multiple Input / Single Output 48 1
46. 1 48 90 ? 90 Multiple Input / Single Output 48 1
47. 1 48 90 ? 90 Multiple Input / Single Output 48 1 + +
48. 1 48 90 ? 90 Multiple Input / Single Output 48 1 + +
49. 1 48 90 90 Multiple Input / Single Output 48 1 + + output 139
50. 0.2 1 0.12 48 0.45 90 1 48 90 Multiple Input / Single Output + + 139 output 0.005 0.0025 0.2 =
51. 0.2 0.12 0.45 1 48 90 90 Multiple Input / Single Output 48 1 + + 0.77 output 0.005 0.0025 0.2 =
52. 0.2 0.12 0.45 1 48 90 90 Multiple Input / Single Output 48 1 + + 0.77 output 0.005 0.0025 0.2 =
53. 0.2 0.12 0.45 1 48 90 + + output 0.77 0.2 0.0025 0.005 = Multiple Input / Single Output [0.45, 0.12, 0.2] [90, 48, 1] |> mult_inputs([0.005, 0.0025, 0.02]) end def mult_inputs(inputs, weights) do Enum.zip(inputs, weights) |> Enum.map(fn {i, w} -> i * w end)
54. 0.2 0.12 0.45 1 48 90 + + output 0.77 0.2 0.0025 0.005 = Multiple Input / Single Output [0.45, 0.12, 0.2] [90, 48, 1] |> mult_inputs([0.005, 0.0025, 0.02]) end def mult_inputs(inputs, weights) do Enum.zip(inputs, weights) |> Enum.map(fn {i, w} -> i * w end)
55. 0.2 0.12 0.45 1 48 90 + + output 0.77 0.2 0.0025 0.005 = Multiple Input / Single Output [0.45, 0.12, 0.2] [90, 48, 1] |> mult_inputs([0.005, 0.0025, 0.02]) end |> Enum.sum() def mult_inputs(inputs, weights) do Enum.zip(inputs, weights) |> Enum.map(fn {i, w} -> i * w end)
56. 0.2 0.12 0.45 1 48 90 + + output 0.77 0.2 0.0025 0.005 = Multiple Input / Single Output 0.77 [90, 48, 1] |> mult_inputs([0.005, 0.0025, 0.02]) end |> Enum.sum() def mult_inputs(inputs, weights) do Enum.zip(inputs, weights) |> Enum.map(fn {i, w} -> i * w end)
57. 0.2 0.12 0.45 90 Multiple Input / Single Output 48 1 + + 0.77 output 0.005 0.0025 0.2 =
58. 90 Multiple Input / Single Output 48 1 0.77 0.005 0.0025 0.2
59. 90 Multiple Input / Multiple Output 48 1 $88m 0.77 600_000 500_000 10_000_000
60. 90 Multiple Input / Multiple Output 48 1 0.49 $88m 0.77 0.008 -0.005 0.01
61. 90 48 1 0.49 $88m 0.77 0.008 -0.005 0.01 [0.77, 88_000_000, 0.49] [90, 48, 1] |> network(weights) def network(inputs, weights) do weights |> Enum.map(&mult_inputs(&1)) end Multiple Input / Multiple Output
62. 90 48 1 0.49 $88m 0.77 0.008 -0.005 0.01 [0.77, 88_000_000, 0.49] [90, 48, 1] |> network(weights) weights = [[0.005, 0.0025, 0.02], [600_000, 500_000, 10_000_000], [0.008, -0.005, 0.01]] Multiple Input / Multiple Output def network(inputs, weights) do weights |> Enum.map(&mult_inputs(&1)) end
63. 90 Multiple Input / Multiple Output 48 1 0.49 $88m 0.77
64. 90 Multiple Input / Multiple Output 48 1 0.49 $88m 0.77
65. 90 Multiple Input Neural Network 48 1 0.49 $88m 0.77
66. 90 Multiple Input Neural Network 48 1 0.49 $88m 0.77
67. 90 Neural Network 48 1 0.49 $88m 0.77
68. 1 48 90 ? ? ? 0.49 $88m 0.77 Hidden Layers
69. ? Hidden Layers
70. 0.84 Hidden Layers blockbuster
71. 0.32 Hidden Layers oscar bait
72. 0.14 Hidden Layers cult classic
73. 0.97 Hidden Layers sci-fi
74. ? Hidden Layers output
75. ? Hidden Layers output
76. ? Hidden Layers input output
77. input ? Hidden Layers
78. input Hidden Layers inputs |> apply_weights(weights) |> Enum.sum() |> activation_fn()
79. Hidden Layers |> activation_fn() inputs |> apply_weights(weights) |> Enum.sum() linear / nonlinear
80. Hidden Layers linear / nonlinear input_b input_a input_c output
81. Linear / Nonlinear input_b input_a input_c output
82. Linear / Nonlinear input_b input_a input_c output 90
83. Linear / Nonlinear input_b input_a input_c output 0.05 x 0.25 x 120 x 90
84. Linear / Nonlinear input_b input_a input_c output ( ) 0.05 x 0.25 x 120 x 90
85. Linear / Nonlinear input_b input_a input_c output 1.5 x 90
86. input Linear / Nonlinear input_b input_a input_c output
87. input Linear / Nonlinear input_b input_a input_c output
88. input Linear / Nonlinear input_b input_a input_c output
89. input Activation Function inputs |> apply_weights(weights) |> Enum.sum() |> activation_fn()
90. input Activation Function |> activation_fn() inputs |> apply_weights(weights) |> Enum.sum()
91. input Activation Function |> activation_fn() inputs |> apply_weights(weights) |> Enum.sum() sigmoid() tanh() {0, :infinity} {0, 1} {-1, 1} ) ( relu ) ( relu
92. input Activation Function def num when num < 0, do: 0 def num , do: num ( ) relu ( ) relu
93. input Activation Function inputs |> apply_weights(weights) |> Enum.sum() |> activation_fn() relu
94. input Activation Function inputs |> apply_weights(weights) |> Enum.sum() |> activation_fn () relu
95. input Linear / Nonlinear input_b input_a input_c output
96. input Linear / Nonlinear input_b input_a input_c output
97. input Linear / Nonlinear input_b input_a input_c output
98. input Linear / Nonlinear input_b input_a input_c output
99. input Machine Learning inputs |> apply_weights( ) |> Enum.sum() |> apply_bias() |> relu() weights
100. input |> relu() |> apply_bias() weights inputs |> apply_weights( ) |> Enum.sum() Machine Learning
101. input weights inputs |> apply_weights( ) |> Enum.sum() |> apply_bias() |> relu() Machine Learning Learning
102. input weights inputs |> apply_weights( ) |> Enum.sum() |> apply_bias() |> relu() Machine Learning Learning
103. input _weights( ) sum() _bias() weights Machine Learning Learning
104. Training s( ) weights Machine Lear Lear
105. Training
106. Training Star Wars: The Last Jedi
107. Training Star Wars: The Last Jedi
108. Star Wars: The Last Jedi target: 1.0 48 90 1 Training critics audience rian johnson
109. Star Wars: The Last Jedi target: 1.0 48 90 1 Training critics audience rian johnson
110. 48 90 1 Training Star Wars: The Last Jedi target: 1.0
111. 48 90 1 Training Star Wars: The Last Jedi target: 1.0
112. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
113. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
114. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
115. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
116. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
117. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
118. Training 48 90 1 Star Wars: The Last Jedi target: 1.0
119. Training 48 90 1 0.6 0.6 : The Last Jedi Star Wars 1.0 : target
120. Training 48 90 1 0.6 0.6 he ast edi TLJ Star Wars: 1.0 : target pred. iction T L J
121. Training 48 90 1 0.6 target iction pred. he ast edi Star Wars: 1.0 0.6 T L J
122. Training target pred. 1.0 0.6 : The Last Jedi Star Wars:TLJ
123. Training target pred. 1.0 0.6 : The Last Jedi Star Wars:TLJ M. Night Shyamalan
124. Training target pred. 1.0 0.6 : The Last Jedi Star Wars:TLJ M. Night Shyamalan's The Happening
125. Training target pred. 1.0 0.6 : The Last Jedi Star Wars:TLJ The Happening 0.0 M. Night Shyamalan's The Happening
126. Training target iction pred. : The Last Jedi Star Wars:TLJ 1.0 0.6 The Happening The Happening 0.0 target: 0.0 ? 24 18 0
127. Training target iction pred. : The Last Jedi Star Wars:TLJ 1.0 0.6 24 18 0 The Happening 0.0 ? The Happening target: 0.0
128. Training target iction pred. : The Last Jedi Star Wars:TLJ 1.0 0.6 24 18 0 0.4 0.4 The Happening 0.0 ? The Happening target: 0.0
129. 24 18 0 0.4 0.0 0.4 The Happening 1.0 0.6 Star Wars:TLJ target pred. Training
130. 0.0 0.4 The Happening 1.0 0.6 Star Wars:TLJ Training target pred.
131. 0.0 0.4 The Happening 1.0 0.6 Star Wars:TLJ target pred. Training - -
132. 0.0 0.4 The Happening 1.0 0.6 Star Wars:TLJ target pred. Training - - = = 0.4 -0.4 error
133. 0.0 0.4 The Happening 1.0 0.6 Star Wars:TLJ target pred. Training - - = = 0.4 -0.4 0.0 avg error
134. The Happening Star Wars:TLJ Training 0.4 -0.4 error
135. The Happening Star Wars:TLJ Training 0.4 -0.4 delta
136. The Happening Star Wars:TLJ Training 0.4 -0.4 x x 0.4 -0.4 delta
137. The Happening Star Wars:TLJ Training 0.4 -0.4 x x = = 0.4 -0.4 0.16 0.16 delta error
138. The Happening Star Wars:TLJ Training 0.4 -0.4 x x = = 0.4 -0.4 0.16 0.16 (large error) 1.0 x = 1.0 1.0 (small error) 0.1 x = 0.1 0.01 delta error
139. The Happening Star Wars:TLJ Training 0.4 -0.4 x x = = 0.4 -0.4 0.16 0.16 delta error
140. The Happening Star Wars:TLJ Training 0.4 -0.4 x x = = 0.4 -0.4 0.16 0.16 avg 0.16 delta error
141. Training
142. Training
143. Training 0.4 -0.4
144. Training How can we reduce our error rate?
145. Training How can we reduce our error rate?
146. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
147. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
148. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
149. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
150. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
151. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
152. Gradient Descent 0.0 error rate 0.1 1.0 prediction 1.1 1.2 1.3 0.9 0.8 0.7 0.2 0.3 0.4
153. Reducing Error 1.0 weight target prediction input 0.4 0.8 0.5
154. Reducing Error 1.0 input weight target prediction 0.4 0.8 0.5
155. Reducing Error 1.0 weight input target prediction 0.4 0.8 0.5
156. Reducing Error 1.0 weight prediction input target 0.4 0.8 0.5
157. Reducing Error 1.0 weight target prediction input 0.4 0.8 0.5
158. 0.4 0.8 Reducing Error 0.5 1.0 weight target prediction input delta = (prediction - target) -0.6
159. 0.4 0.8 Reducing Error 0.5 1.0 weight target prediction input delta = (prediction - target) -0.6 error = (delta * delta) 0.36
160. 0.4 0.8 0.5 1.0 weight target prediction input delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 Reducing Error
161. 0.4 0.8 0.5 1.0 weight target prediction input Why Multiply By Input? nudge = (delta * input)
162. You can't learn if the input is zero. Stopping nudge = (delta * input) Why Multiply By Input?
163. A negative input will reverse delta direction. Negative Reversal You can't learn if the input is zero. Stopping nudge = (delta * input) Why Multiply By Input?
164. A weight adjustment should scale to the input size. Scaling A negative input will reverse delta direction. Negative Reversal You can't learn if the input is zero. Stopping nudge = (delta * input) Why Multiply By Input?
165. 0.4 0.8 0.5 1.0 weight target prediction input A weight adjustment should scale to the input size. Scaling A negative input will reverse delta direction. Negative Reversal You can't learn if the input is zero. Stopping nudge = (delta * input) Why Multiply By Input? *This can also cause problems delta = (prediction - target) error = (delta * delta)
166. 0.4 0.8 0.5 1.0 weight target prediction input delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 Reducing Error
167. delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 1.0 input 0.8 0.5 0.4 prediction target weight delta error nudge -0.6 0.36 -0.48 >
168. delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 1.0 input 0.8 0.5 0.4 prediction target weight nudge error delta -0.48 0.36 -0.6 > 01 :: Prediction: 0.4 Error: 0.36 >
169. delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 1.0 input 0.8 0.5 0.4 prediction target weight new_weight = (weight - nudge) nudge error delta -0.48 0.36 -0.6 > > 01 :: Prediction: 0.4 Error: 0.36
170. delta = (prediction - target) -0.6 error = (delta * delta) nudge = (delta * input) 0.36 -0.48 1.0 input 0.8 0.5 0.4 prediction target weight new_weight = (weight - nudge) 0.98 nudge error delta -0.48 0.36 -0.6 > 01 :: Prediction: 0.4 Error: 0.36
171. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 0.98 prediction target weight new_weight = (weight - nudge) nudge error delta > 01 :: Prediction: 0.4 Error: 0.36
172. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 0.98 .784 prediction target weight new_weight = (weight - nudge) nudge error delta > 01 :: Prediction: 0.4 Error: 0.36
173. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 0.98 .784 prediction target weight new_weight = (weight - nudge) nudge error delta -0.216 -0.216 > 01 :: Prediction: 0.4 Error: 0.36
174. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 0.98 .784 prediction target weight new_weight = (weight - nudge) nudge error delta -0.216 -0.216 0.046656 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
175. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 0.98 .784 prediction target weight new_weight = (weight - nudge) nudge error delta -0.216 -0.216 0.046656 0.046656 -0.1728 -0.1728 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
176. delta = (prediction - target) -0.216 error = (delta * delta) nudge = (delta * input) 0.046656 -0.1728 1.0 input 0.8 0.98 .784 prediction target weight new_weight = (weight - nudge) 1.1528 nudge error delta -0.1728 0.046656 -0.216 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
177. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.1528 prediction target weight new_weight = (weight - nudge) nudge error delta > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
178. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.1528 prediction target weight new_weight = (weight - nudge) nudge error delta -0.062208 -0.062208 0.006046618 0.006046618 -0.07776 -0.07776 .92224 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
179. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.1528 prediction target weight new_weight = (weight - nudge) nudge error delta -0.062208 -0.062208 0.006046618 0.006046618 -0.07776 -0.07776 .92224 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
180. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.1528 prediction target weight new_weight = (weight - nudge) nudge error delta 1.215008 -0.062208 -0.062208 0.006046618 0.006046618 -0.07776 -0.07776 .92224 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
181. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.215008 prediction target weight new_weight = (weight - nudge) nudge error delta > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
182. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.215008 prediction target weight new_weight = (weight - nudge) nudge error delta 1.23740288 -0.02239488 -0.02239488 0.000783642 0.000783642 -0.0279936 -0.0279936 .9720064 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
183. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 1.23740288 prediction target weight new_weight = (weight - nudge) nudge error delta > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
184. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 prediction target weight new_weight = (weight - nudge) nudge error delta 1.24546504 -0.00806216 -0.00806216 0.00010156 0.00010156 -0.0100777 -0.0100777 .9899223 1.23740288 > 05 :: Prediction: 0.9899223 Error: 0.00010156 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
185. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) 1.0 input 0.8 prediction target weight new_weight = (weight - nudge) nudge error delta 1.24546504 > 05 :: Prediction: 0.9899223 Error: 0.00010156 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
186. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 1.248367416 -0.002902376 0.000013162 -0.00362797 1.0 input 0.8 prediction target weight nudge error delta -0.002902376 0.000013162 -0.00362797 .99637203 1.24546504 > 06 :: Prediction: 0.99637203 Error: 0.000013162 > 05 :: Prediction: 0.9899223 Error: 0.00010156 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
187. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 1.248367416 -0.002902376 0.000013162 -0.00362797 1.0 input 0.8 prediction target weight nudge error delta -0.002902376 0.000013162 -0.00362797 .99637203 1.24546504 > 06 :: Prediction: 0.99637203 Error: 0.000013162 > 05 :: Prediction: 0.9899223 Error: 0.00010156 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
188. > 06 :: Prediction: 0.99637203 Error: 0.000013162 > 05 :: Prediction: 0.9899223 Error: 0.00010156 > 04 :: Prediction: 0.9720064 Error: 0.000783642 > 03 :: Prediction: 0.92224 Error: 0.006046618 > 02 :: Prediction: 0.784 Error: 0.046656 > 01 :: Prediction: 0.4 Error: 0.36
190. 0.8 1.0
191. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution
192. 0.8 1.0 Error Attribution delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) -0.2
193. 0.8 1.0 Error Attribution delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) -0.2
194. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution prev_delta = (delta * weight) -0.2
195. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution prev_delta = (delta * weight) -0.2
196. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution prev_delta = (delta * weight) 1 0.5 -0.2
197. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution prev_delta = 1 0.5 -0.2 (-0.2 * 0.5)
198. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1 prev_
199. delta = (prediction - target) error = (delta * delta) nudge = (delta * input) new_weight = (weight - nudge) 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1
200. delta = (prediction - target) error = (delta * delta) nudge = -0.1 * 1 new_weight = (weight - nudge) 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1
201. delta = (prediction - target) error = (delta * delta) nudge = -0.1 new_weight = (weight - nudge) 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1
202. delta = (prediction - target) error = (delta * delta) nudge = -0.1 new_weight = 0.5 - -0.1 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1
203. delta = (prediction - target) error = (delta * delta) nudge = -0.1 new_weight = 0.8 1.0 Error Attribution 1 0.5 -0.2 delta = -0.1 0.6 0.6
204. 0.8 1.0 1 0.6 -0.2 delta = (prediction - target) error = (delta * delta) nudge = -0.1 new_weight = Error Attribution delta = -0.1 0.6
205. 0.8 1.0 1 0.6 -0.2 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 nudge = (delta * input) new_weight = (weight - nudge)
206. 0.8 1.0 1 0.6 -0.2 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 0.2 3 nudge = (delta * input) new_weight = (weight - nudge)
207. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge)
208. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge)
209. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 prev_delta = (delta * weight) 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge)
210. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 prev_delta = (-0.1 * 0.2) 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge)
211. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 prev_delta = -0.02 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge) -0.02
212. 1 delta = (prediction - target) error = (delta * delta) Error Attribution delta = -0.1 prev_delta = -0.02 0.2 3 -0.1 nudge = (delta * input) new_weight = (weight - nudge) -0.02
213. 1 0.2 3 -0.1 -0.02 [ , 3, 0.5, -4.2, 0.018]
214. 1 0.2 3 -0.1 -0.02 [ , 3, 0.5, -4.2, 0.018]
215. 1 0.2 3 -0.1 -0.1404
216. 1 3
217. 48 90 1
218. 48 90 1
219. 48 90 1
220. 48 90 1
221. 48 90 1
222. 48 90 1
223. 48 90 1
224. 48 90 1
225. 48 90 1
226. 48 90 1
228. Training Data Overfitting
229. Training Data We need diverse data
230. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 41 66 75 81 34 49 45 92 76 65 96 65 73 45 61 62 52 59 61 84 36 48 66 19 66 64 29 19 33 93 75 24 87 76 73 64 87 69 94 63 55 90 50 24 35 51 77 94 81 39 17 51 68 89 73 23 44 54 94 81 66 88 77 72 35 48 53 46 74 51 80 31 57 55 26 77 55 25 33 45 97 78 35 74 88 58 58 76 73 80 56 55 93 46 32 45 57 88 90 84 29 29 Six Days Seven Nights Species Vertical Limit I.Q. The Princess Diaries Unbreakable Everything is Illuminated Miracle on 34th Street A Christmas Story Broken Arrow National Lampoon's Loaded Weapon 1 Some Like It Hot Half Nelson Maid in Manhattan Red Planet Weekend at Bernie's II Rain Man A Charlie Brown Christmas Mr. Magoo Looper Jack Reacher Hollow Man Terminator 2: Judgment Day An American Tail: Fievel Goes West Don't Tell Mom the Babysitter's Dead Beetlejuice Mr. Boogedy Gravity 21 Pride & Prejudice The Machinist All About My Mother The Avengers Friday Memoirs of an Invisible Man Problem Child The NeverEnding Story Willy Wonka & the Chocolate Factory Mission: Impossible - Ghost Protocol Inception Eagle Eye Vantage Point Band of Brothers The One The Perfect Storm Stigmata Teaching Mrs. Tingle Quigley Down Under Despicable Me The Island Spy Game 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 52 46 89 67 83 22 87 82 101 85 87 45 78 95 42 43 36 82 48 78 40 78 55 18 11 17 54 89 62 61 92 54 43 82 52 64 77 80 93 92 29 53 40 92 61 79 33 21 85 34 8 34 64 32 89 66 73 20 90 92 86 82 82 47 71 92 57 43 38 94 48 88 48 65 40 21 25 22 44 96 61 71 97 69 42 84 54 69 82 77 79 82 35 68 53 80 56 89 42 29 71 39 23 Bridget Jones's Diary Godzilla Robin Hood: Men in Tights The Breakfast Club True Grit The Blob Rope The Road A Serious Man Be Kind Rewind Lucky Number Slevin Resident Evil: Apocalypse Panic Room Bubba Ho-tep Lara Croft: Tomb Raider Super Troopers Office Space Chasing Amy The Flintstones Street Fighter Much Ado About Nothing Toys Bill & Ted's Excellent Adventure Predator The Shining Close Encounters of the Third Kind Star Trek Beyond Without a Paddle Old School Joe Dirt Bio-Dome Home Alone Knight and Day Synecdoche, New York Planet Terror Mystic River The Nightmare Before Christmas So I Married an Axe Murderer Beethoven Captain America: Civil War Where the Wild Things Are I Am Legend The Terminal Mission: Impossible II Cruel Intentions Pleasantville The Edge Fargo Major Payne Cabin Boy Singin' in the Rain The Cloverfield Paradox 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 59 13 49 14 46 49 38 93 17 88 44 42 57 90 37 91 41 71 72 54 98 29 87 42 91 64 93 24 48 40 58 77 20 92 54 90 35 51 74 82 47 45 66 45 86 67 37 38 64 46 33 68 20 51 28 55 50 36 97 28 84 42 39 59 86 46 98 34 75 85 67 85 43 96 29 88 78 98 20 61 38 61 63 21 82 69 90 47 37 85 79 59 50 64 52 77 81 30 28 67 44 42 Changing Lanes Coyote Ugly Police Story 3: Supercop Tron All About Eve (500) Days of Summer The Wrestler Defiance What Happens in Vegas War of the Worlds Spider-Man 2 Cellular Sphere Good Will Hunting George of the Jungle Army of Darkness The Addams Family Say Anything... Thor: The Dark World The Manchurian Candidate The Matrix Revolutions Hulk About a Boy The Mummy Returns Swordfish Galaxy Quest Notting Hill Big Daddy SLC Punk Down Periscope Tank Girl Benny & Joon Teenage Mutant Ninja Turtles III Captain Ron The Cutting Edge The Aristocats Wanted I, Robot The Transporter A Walk to Remember The Master of Disguise Bandits How the Grinch Stole Christmas Run Lola Run Heat The Lion King Stargate Major League II Hard Boiled Love Potion No. 9 Dog Day Afternoon 0 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 1 1 1 0 0 1 0 1 1 1 0 1 0 0 0 1 0 0 0 1 0 1 0 0 0 1 1 0 0 1 0 1 0 0 0 1 1 1 0 0 1 1 1 0 0 1 0 0 0 1 1 0 1 1 1 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 1 1 0 1 0 0 1 0 0 1 1 1 1 0 0 0 1 1 0 0 1 1 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 0 0 1 Training Data
231. 48 90 1
232. 48 90 1
233. 48 90 1 3.2
234. 54 32 0 3.2
235. 54 32 0 0.6 3.2
236. 89 73 1 0.6 3.2
237. 89 73 1 -4.2 0.6 3.2
238. 89 73 1 0.6 3.2 -4.2
239. 89 73 1 -4.2 0.6 3.2 0.018
240. 1.138 1.138 1.138 3.2 0.6 -4.2 0.018 3.58 0.5 -9.2 0.431 0.2 -0.212 -3.22 0.021 0.234 -5.2 0.098 ...
241. 1.138 1.138 1.138 3.2 0.6 -4.2 0.018 3.58 0.5 -9.2 0.431 0.2 -0.212 -3.22 0.021 0.234 -5.2 0.098
242. 1.138 1.138 1.138 3.2 0.6 -4.2 0.018 3.58 0.5 -9.2 0.431 0.2 -0.212 -3.22 0.021 0.234 -5.2 0.098
243. 1.138 1.138 1.138 3.2 0.6 -4.2 0.018 3.58 0.5 -9.2 0.431 0.2 -0.212 -3.22 0.021 0.234 -5.2 0.098
244. Making predictions with unknown data Testing the Network
245. 0.993723589 0.992233171 0.991408145 0.990603478 0.984346526 0.984270099 0.982508441 0.975472634 0.970683652 0.970432433 0.968719318 Annihilation Ready Player One Split Black Panther Baby Driver The Big Sick The Post Get Out Darkest Hour Lady Bird Game Night Testing the Network
246. 0.517596672 0.017379878 0.015183403 0.006707093 0.006251533 0.005204257 0.001276453 0.000970709 0.000651533 0.000200425 0.000127453 0.000067070 0.00000000001 Mute The Disaster Artist Justice League Bright Happy Death Day The Fate of the Furious IT Geostorm Emoji Movie The Snowman Boss Baby Zoolander No. 2 Batman v Superman Testing the Network
247. Machine Learning
248. Additional Resources "Grokking Deep Learning" book by Andrew W. Trask "Deep Learning" video series by 3Blue1Brown "Erlang and Deep Learning" talk by Garrett Smith "Building a Neural Network From Scratch" talk by Karmen Blake "A visual introduction to machine learning" site by R2D3 "Machine Learning" Coursera course by Andrew Ng "Intro to Machine Learning" Udacity course by kaggle
249. Additional Resources playground.tensorflow.org
250. @jessejanderson follow me Thanks!