Intro to Artificial Neural Network

Many parts of this slides are taken from DataCamp's introduction to deep learning course.

Transcript

1. Artificial Neural Network INTERNAL CLASS DATA SCIENCE RISTEK 2021 FATHINAH ASMA IZZATI 2021
2. Introduction Feedforward Activation Function Gradient Descent Backpropagation Content 1. 2. 3. 4. 5.
3. Neurons: The Brain Building Blocks
4. How the code looks? Dense implements the operation: output = activation(dot(input, kernel) + bias Sequential model is a linear stack of layers.
5. How Neuron Fires Up?
7. Activation Function TANH (e^x – e^-x) / (e^x + e^-x) SIGMOID 1.0 / (1.0 + e^-x) RELU max(0.0, x)
8. Activation Function for Output Layers Logistic (Sigmoid) : 0 or 1 Softmax Linear: No Activation https://www.deeplearningwizard.com/deep_learning/practical_pytorch/pytorch_logistic_regression/
9. How to choose?
10. Feed-Forward Network
11. Practice 1 Notebook: https://colab.research.google.com/drive/1keGn1ZDik5ANAOAotx-rvWp4Cqi8g4Ot? usp=sharing
12. The Need for Optimization
14. Loss Function
19. Gradient Descent Steps
23. Gradient Descent Example
24. Backpropagation Chain Rule "if a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man."
26. Backpropagation
29. SUMMARY
30. STOCHASTIC GRADIENT DESCENT
31. OPTIMIZER LOSS OPTIMIZATION
32. Practice 2 Notebook: https://colab.research.google.com/drive/14ID0XH96xNyWZbB3_GKFfwXZG9d2k0vR? usp=sharing
34. Key Advantages of ANN Learn from non-linear and complex relationship No input restriction, can better model heteroscedasticity data Application Image Recognition Paved a way to Deep Learning Forecasting 3 types of NN: ANN, CNN, RNN
35. Homework https://colab.research.google.com/drive/1sFOsJBV_9rB6h DCCJczxeeBBQtcR0XTM?usp=sharing
36. Sources Data Camp Introduction to Deep Learning Slides Slides MK Sistem Cerdas 2020 https://www.kdnuggets.com/2017/04/simple-understand-gradient-descent-algorithm.html https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/ https://www.analyticsvidhya.com/blog/2020/02/cnn-vs-rnn-vs-mlp-analyzing-3-types-of-neural-networks-in-deep- learning/ https://towardsdatascience.com/introduction-to-neural-networks-advantages-and-applications-96851bd1a207 https://www.bogotobogo.com/python/scikit-learn/scikit-learn_batch-gradient-descent-versus-stochastic-gradient- descent.php