Introduction to Machine Learning & Neural Network

Machine Learning?
Neural Network?
Let’s create a model!

Transcript

1. Introduction to Machine Learning & Neural Network DSC Universitas Komputer Indonesia Alexander Manuel S DSC Lead Universitas Komputer Indonesia 2019/2020 lexmanuel.com
2. Universitas Komputer Indonesia
3. Machine Learning Neural Network Let’s create a model
4. What is Machine Learning?
5. Universitas Komputer Indonesia Deep learning e.g: Multilayer Perceptrons Representation Learning e.g: Autoencoders Machine Learning e.g: Logistic Regression Artificial Intelegence e.g: Knowledge Base -> MYCIN (for treating blood infections, Stanford University) src: Deep Learning, Ian Good Fellow
6. Universitas Komputer Indonesia Input Input Input Input Hand-designed features Hand-designed program Features Simple features Output Mapping from features Mapping from features Additional layers of more abstract features Output *Shaded boxes indicate components that are able to learn from data. Output Mapping from features Output Classic machine learning Rule-based systems Representation learning Deep learning src: Deep Learning, Ian Good Fellow
7. Universitas Komputer Indonesia Which approach to use? Machine Learning Machine Learning Rules-based Predicting Salary based on Age
8. Universitas Komputer Indonesia Which approach to use? Processing online payments Machine Learning Rules-based Rules-based
9. Universitas Komputer Indonesia Which approach to use? Process and analyze natural language data. Machine Learning Rules-based Rules-based Machine Learning
10. Universitas Komputer Indonesia Which approach to use? detecting face in a photo Machine Learning Machine Learning Rules-based
11. What is Machine Learning?
12. What is Neural Network?
13. Universitas Komputer Indonesia First, Let’s Go Back Linear Regression For predicting planets and pea growth 1800s Single Layer Perceptron Precursor to neural network 1940s 1960s Neural Network src: Coursera
14. Universitas Komputer Indonesia Neurons
15. Universitas Komputer Indonesia Activation Function There’s a lot of them!
16. Universitas Komputer Indonesia Sigmoid activation function converts the weighted sum to a value between 0 and 1. ReLU (Rectified Linear Unit) activation function often works a little better than a smooth function like the sigmoid, while also being significantly easier to compute. Common Activation Functions
17. Universitas Komputer Indonesia A Tale of Neural Network Squeeze it Take some image & do some preprocess Feed it to the model *oversimplified Checking how bad the model is • Takes the weights & biases as inputs • Measuring the differences (cost) between the prediction and what really is. • And take the average of it. • E.g: MSE • The model has weight, biases, and activations Calculate the gradient & updating the weight and biases , = 1 σ=1 (ෝ , ) • Chain rule • Partial derivative of the cost function respect to and • Calculate the updated and = = ≔ − ∗ ≔ − ∗
18. Universitas Komputer Indonesia
19. Universitas Komputer Indonesia Backpropagation Focus on the cat Cat’s ear Cat’s mouth Dog’s mouth Dog’s ear = (0 ∗ 0 + ⋯ + −1 ∗ −1 + ) Increase the weight In proportion to activation function, “Neurons that fire together, wire together” 0.67 is a Cat Increase the bias Change activation function In proportion to weight, But we cannot change the activation function
20. Universitas Komputer Indonesia Backpropagation Zooming out Cat’s ear Cat’s mouth Dog’s mouth Dog’s ear 0.67 is a Cat 0.33 is a Dog 0 +0.56 -0.73 -0.09 1 +0.26 -0.10 +0.08 2 -0.41 +0.21 +0.31 +0.72 +0.43 +058 … …. … …
21. Universitas Komputer Indonesia Gradient Descent Optimizing gradient descent +1 = − ∗ ෝ ෝ + ෝ = 1 − 1 ෝ = 1 − 2 = 1 − 1 + 1 ∗ −1 = 1 − 2 2 + 2 ∗ −1 Adam Optimizer learning rate (α), β1, β2, dan Epsilon.
22. What is Neural Network?
23. Let’s create a model!
24. Universitas Komputer Indonesia Installing TensorFlow
25. Universitas Komputer Indonesia Installing TensorFlow
26. Universitas Komputer Indonesia Preparing Dataset Training Data & Validation Data Training Dataset: The sample of data used to fit the model. Validation Dataset: The sample of data that have never been seen by the model and will be used to validate the model while the model still in training
27. Universitas Komputer Indonesia Preparing Dataset Training Data & Validation Data Training accuracy Validation accuracy OVERFIT
28. Universitas Komputer Indonesia Overfit & Underfit src: www.geeksforgeeks.org
29. Universitas Komputer Indonesia Create the Neural Network • 1 input layer that recieves images with the size of 160x160 and 3 channels with relu activation • 2 hidden layers, that consist of 200 and 60 neurons with relu activation • 1 output layer with sigmoid activation will give a probabality of binary classification model
30. Universitas Komputer Indonesia Create the Neural Network model.summary()
31. Universitas Komputer Indonesia Run the Training model.fit()
32. Universitas Komputer Indonesia Plot some charts • Validation accuracy is ~60% • Not overfitting • But underfitting
33. Universitas Komputer Indonesia Result
34. Universitas Komputer Indonesia How to Improve 1. More data? 2. Data Augmentation? 3. Better models? 4. Hyperparameter Tuning (Epoch, hidden units, layers, … )?
35. Learn more about: • Gradient descent, here • Visualization of Neural Net, here • In depth to Neural Net, here • Types of Loss function, here Books: • Deep Learning, Ian Goodfellow • Hands-On Machine Learning with Scikit-Learn and TensorFlow, Aurélien Géron techdevguide.withgoogle.com/paths/ machine-learning/ • ML crash course • Kaggle
36. Universitas Komputer Indonesia
37. Feel free to connect DSC Universitas Komputer Indonesia lexmanuel.com linkedin.com/in/lexms twitter.com/lexms_ Resources of this slides: • speakerdeck.com/lexmanuel • Notebook
38. Introduction Neural Network Machine Learning