like Von Neumann and Turing didn’t believe in Symbolic AI. They were far more inspired by the brain. Unfortunately, they both died very young and their voice wasn’t heard.” - Geoffrey Hinton in Heroes of Deep Learning
learns and what kind of data is used to learn it. 2. Tries to capture the causality between the input variables and output variables. 3. Requires training features and labels.
of one or more variables onto a real number intuitively representing some “cost” associated with the event. (Wikipedia) Examples are MSE, Cross Entropy, KL Divergence, Hinge, MAE
right answers are — we just look. Every so often, your mother says “that’s a dog”, but that’s very little information. You’d be lucky if you got a few bits of information — even one bit per second — that way. The brain’s visual system has 10¹⁴ neural connections. And you only live for 10⁹ seconds. So it’s no use learning one bit per second. You need more like 10⁵ bits per second. And there’s only one place you can get that much information: from the input itself.” — Geoffrey Hinton, 1996
mathematical or computational model for information processing • Learns by changing the strength of the connections • Perceptrons • Deep Neural Networks