Machine Learning: The Bare Math Behind Libraries - Supervised Learning

During this presentation, we will answer how much you’ll need to invest in a superhero costume to be as popular as Superman. We will generate a unique logo which will stand against the ever popular Batman and create new superhero teams. We shall achieve it using linear regression and neural networks.

Machine learning is one of the hottest buzzwords in technology today as well as one of the most innovative fields in computer science – yet people use libraries as black boxes without basic knowledge of the field. In this session, we will strip them to bare math, so next time you use a machine learning library, you’ll have a deeper understanding of what lies underneath.

During this session, we will first provide a short history of machine learning and an overview of two basic teaching techniques: supervised and unsupervised learning.

We will start by defining what machine learning is and equip you with an intuition of how it works. We will then explain the gradient descent algorithm with the use of simple linear regression to give you an even deeper understanding of this learning method. Then we will project it to supervised neural networks training.

Transcript

1. achine Learning: The Bare Math Behind Libraries 23.04.2020 @CoffeeJUG by Łukasz Gebel & Piotr Czajka
2. Super Heroes & Super Powers
4. Agenda • What is Machine Learning? • Supervised learning • Q&A
5. Machine Learning „Field of study that gives computers the ability to learn without being explicitly programmed.” Arthur Samuel
6. Machine Learning „I consider every method that needs training as intelligent or machine learning method.” Our Lecturer
7. Supervised learning • It is similar to be tought by a teacher.
8. Supervised learning • Build your model that performs particular task: – Prepare data set consisting of examples & expected outputs
9. Supervised learning • Build your model that performs particular task: – Prepare data set consisting of examples & expected outputs – Present examples to your model
10. Supervised learning • Build your model that performs particular task: – Prepare data set consisting of examples & expected outputs – Present examples to your model – Check how it responds (model’s output values)
11. Supervised learning • Build your model that performs particular task: – Prepare data set consisting of examples & expected outputs – Present examples to your model – Check how it responds (model’s output values) – Adjust model’s params by comparing output values with expected output values
12. Neural Networks • Inspired by biological brain mechanisms • Many applications: – Computer vision – Speech recognition – Compression
13. Biological Neuron http://www.marekrei.com/blog/wp-content/uploads/2014/01/neuron.png
14. Artificial Neuron • Inputs (x 1, … , x n ) are features of single example • Multiply each input by weight, sum it and put the sum as an argument of activation function w 1 w 2 w n w 0 Σ x 1 x 2 x n ... s y
15. Activation function • Sigmoid – Maps sum of neurons signals to value from 0 to 1 – Continous, nonlinear – If input is positive it gives values > 0.5 f (x)= 1 1+e(−βx)
16. Linear Regression • Method for modelling relationship between variables • Simplest form: how x relates to y • Examples: – House size vs house price – Voltage vs electric current
17. Real life problem
18. What defines a superhero?
19. Costume
20. Costume
21. Costume price vs number of issues • For given amount of money predict in how many comic book issues you’ll appear. Costume price(x) Number of issues (y) 240 6370 480 8697 ... ... 26 2200
22. Interesting fact • Invisible woman costume costs $120 Invisible woman
23. Linear regression • Let's have a function: f (x ,Θ)=Θ1 x+Θ0 f (x,Θ)−number of comicbookissues x−costume price Θ−parameters
24. Let's plot
25. Let's plot
26. Let's plot
27. Warning!
28. Objective function Q(Θ)= 1 2 N ∑ j=0 N (f ( xj ,Θ)−y j )2 Q(Θ)−objectivefunction N−numberof datasamples j−indexof particulardatasample
29. Objective function Q(Θ)= 1 2 N ∑ j=0 N (f ( xj ,Θ)−y j )2 Q(Θ)−objectivefunction N−numberof datasamples j−indexof particulardatasample
30. Objective function Q(Θ)= 1 2 N ∑ j=0 N (f ( xj ,Θ)−y j )2 Q(Θ)−objectivefunction N−numberof datasamples j−indexof particulardatasample
31. Objective function Q(Θ)= 1 2 N ∑ j=0 N (f ( xj ,Θ)−y j )2 Q(Θ)−objectivefunction N−numberof datasamples j−indexof particulardatasample
32. Objective function - intuition f (xj ,Θ)=4 y=2 (4−2)2 =22 =4 sum+=4
33. Objective function - intuition f (xj ,Θ)=1 y=1 (1−1)2 =02 =0 sum+=0
34. Gradient descent • Find the minimum of the objective function • Iteratively update function parameters: Θ0 (t +1)=Θ0 (t)−α 1 N ∑ j=0 N (f ( xj ,Θ)− y j ) Θ1 (t +1)=Θ1 (t )−α 1 N ∑ j=0 N (f (xj ,Θ)− yj ) x t−number of iteration α−learning step
35. Gradient descent • Find the minimum of the objective function • Iteratively update function parameters: Θ0 (t +1)=Θ0 (t)−α 1 N ∑ j=0 N (f ( xj ,Θ)− y j ) Θ1 (t +1)=Θ1 (t )−α 1 N ∑ j=0 N (f (xj ,Θ)− yj ) x t−number of iteration α−learning step
36. Gradient descent • Find the minimum of the objective function • Iteratively update function parameters: Θ0 (t +1)=Θ0 (t)−α 1 N ∑ j=0 N (f ( xj ,Θ)− y j ) Θ1 (t +1)=Θ1 (t )−α 1 N ∑ j=0 N (f (xj ,Θ)− yj ) x t−number of iteration α−learning step
37. Gradient descent
38. Gradient descent −α ∗ +derivative < 0
39. Gradient descent −α ∗ +derivative < 0
40. Gradient descent −α ∗ +derivative < 0
41. Gradient descent −α ∗ +derivative < 0
42. Gradient descent −α ∗ −derivative > 0
43. Gradient descent −α ∗ −derivative > 0
44. Gradient descent −α ∗ −derivative > 0
45. Gradient descent −α ∗ −derivative > 0
46. Demo
47. Linearly separable problem
48. Not linearly separable problem
49. NN – random weights init +1 +1 x 1 x 2 y
50. NN – feed forward +1 +1 x 1 x 2 y
51. NN – feed forward +1 +1 x 1 x 2 y
52. NN – feed forward +1 +1 x 1 x 2 y
53. NN – feed forward +1 +1 x 1 x 2 y
54. NN – compute error +1 +1 x 1 x 2 y ( y−expected output)2
55. NN – backpropagation step • Use gradient descent and computed error • Update every weight of every neuron from hidden and output layer
56. NN – backpropagation step +1 +1 x 1 x 2 y ( y−expected output)2
57. NN – backpropagation step +1 +1 x 1 x 2 y
58. Neural Network „Neural networks are nothing more than randomized optimization.” Another Lecturer
59. Real life problem • You said it can solve non linear problems, let's generate superhero logo using it.
60. Summary • Defined Machine Learning
61. Summary • Solved costume problem – supervised learning flavoured linear regression
62. Summary • Generated superhero logo – neural network
63. Call to Action! • Implement linear regression and simple neural network in your favourite programming language • Don’t care about its performance • Check how it works – generate your data sets, get simple ones from the Internet • You’ll gain intuition and understand basic mathematical aparathus
64. Call to Action! • Play with Machine Learning and become superhero ;)
65. Bibliography • Presentation + code: https://bitbucket.org/medwith/public/downloads/mluvr-coffeJugPart1.zip • https://www.coursera.org/learn/machine-learning • https://www.coursera.org/specializations/deep-learning • Math for Machine Learning - Amazon Training and Certification • Linear and Logistic Regression - Amazon Training and Certification • Grus J., Data Science from Scratch: First Principles with Python • Patterson J., Gibson A., Deep Learning: A Practitioner's Approach • Trask A., Grokking Deep Learning • Stroud K. A., Booth D. J, Engineering Mathematics • https://github.com/massie/octave-nn- neural network Octave implementation • https://www.desmos.com/calculator/dnzfajfpym - Nanananana … Batman equation ;) • https://xkcd.com/605/ - extrapolating ;) • http://dilbert.com/strip/2013-02-02 - Dilbert & Machine Learning
66. Thank you