Deep Learning in Neural Networks

Emerging Technologies for Higher Education Post COVID-19 Webinar organized by CJU

Transcript

1. Shadab Hussain https://shadabhussain.com Deep Learning In Neural Networks 1
2. Shadab Hussain https://shadabhussain.com 2 About Me
3. Shadab Hussain https://shadabhussain.com 3 Agenda • Introduction to Deep Learning • Artificial Neural Networks • Activation function • CNN • Confusion matrix • RNN • LSTM • Applications • Tensor-flow playground • Deep Learning Frameworks • Learning ML/DL • Research Labs and Funding • AI Fellowships
4. Shadab Hussain https://shadabhussain.com 4 Deep Learning
5. Shadab Hussain https://shadabhussain.com 5 The neuron collects signals from input channels named dendrites, processes information in its nucleus, and then generates an output in a long thin branch called axon. NUCLEUS AXON DENDRITES X1 X2 X3 NEURON W1 W2 W3 ANN
6. Shadab Hussain https://shadabhussain.com 6 INPUTS/INDEPENDENT VARIABLES 1 F b = (1 1 + 2 2 + 3 3 + ) 2 3 Activation Function
7. Shadab Hussain https://shadabhussain.com 7 Sigmoid
8. Shadab Hussain https://shadabhussain.com 8 ReLU
9. Shadab Hussain https://shadabhussain.com 9 tanh
10. Shadab Hussain https://shadabhussain.com 10 Convolutional Neural Network WHAT ARE CONVOLUTIONAL NEURAL NETWORKS (CNNS) AND HOW DO THEY LEARN? T-SHIRT/TOP TROUSER PULLOVER DRESS COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING
11. Shadab Hussain https://shadabhussain.com 11 Feature Detectors KERNELS/ FEATURE DETECTORS FEATURE MAPS
12. Shadab Hussain https://shadabhussain.com 12 CNN 0 1 1 0 1 1 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 1 0 1 1 0 0 0 1 0 0 1 0 FEATURE DETECTOR IMAGE 1 1 1 3 1 1 2 3 1 FEATURE MAP Live Convolution: http://setosa.io/ev/image-kernels/
13. Shadab Hussain https://shadabhussain.com 13 Applying ReLU T-SHIRT/TOP TROUSER PULLOVER DRESS COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING
14. Shadab Hussain https://shadabhussain.com 14 Pooling • Live illustration : http://scs.ryerson.ca/~aharley/vis/conv/flat.html 1 1 3 4 3 6 2 8 3 9 1 0 1 3 3 4 6 8 9 4 MAX POOLING 2x2 STRIDE = 2 6 8 9 4 FLATTENING
15. Shadab Hussain https://shadabhussain.com 15 HOW TO IMPROVE CNN KERNELS/ FEATURE DETECTORS 64 INSTEAD OF 32
16. Shadab Hussain https://shadabhussain.com 16 Confusion Matrix PREDICTIONS TRUE CLASS TRUE + TRUE - + - + - FALSE + FALSE - TYPE II ERROR TYPE I ERROR
17. Shadab Hussain https://shadabhussain.com 17 Recurrent Neural Network
18. Shadab Hussain https://shadabhussain.com 18 RNN Architecture TIME
19. Shadab Hussain https://shadabhussain.com 19 LSTM NETWORKS RNN PERFORMS WELL SINCE THE GAP BETWEEN THE PREDICTION “GREEN” AND THE NECESSARY CONTEXT INFORMATION “TREE” IS SMALL 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 The tree color is “green” 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 I live in Quebec in Northern Canada……where I live, the weather is generally “cold” most of the year +1 +2 ℎ ℎ+1 ℎ+2 RNN PERFORMS POORLY WHEN THE GAP BETWEEN THE PREDICTION “COLD” AND THE NECESSARY CONTEXT INFORMATION “CANADA” IS LARGE
20. Shadab Hussain https://shadabhussain.com 20 LSTM Intuition VANILLA RECURRENT NEURAL NETWORK LONG SHORT TERM MEMORY NETWORK THIS HORIZONTAL LINE (MEMORY) OR CELL STATE ENABLES LSTM TO REMEMBER VERY OLD INFORMTION
21. Shadab Hussain https://shadabhussain.com 21 Applications of DL Face Recognition
22. Shadab Hussain https://shadabhussain.com 22 Applications of DL Fake News
23. Shadab Hussain https://shadabhussain.com 23 Applications of DL StyleGAN
24. Shadab Hussain https://shadabhussain.com 24 Applications of DL Natural Language Processing
25. Shadab Hussain https://shadabhussain.com 25 Applications of DL Voice Assistant
26. Shadab Hussain https://shadabhussain.com 26 Applications of DL Self Driving Cars
27. Shadab Hussain https://shadabhussain.com 27 TensorFlow Playground •
28. Shadab Hussain https://shadabhussain.com 28 Deep Learning Frameworks
29. Shadab Hussain https://shadabhussain.com 29 DL Frameworks https://colab.research.google.com/ https://www.kaggle.com/ https://gradient.paperspace.com/free-gpu
30. Shadab Hussain https://shadabhussain.com 30 Learning ML/DL • Faculty Awards to Support Machine Learning Courses, Diversity, and Inclusion • Deep Learning Specialization • Fast.ai • Deep Learning A-Z • Deep Learning Nanodegree Program • MIT Deep Learning and Self Driving Cars
31. Shadab Hussain https://shadabhussain.com 31 Research Labs & Funding • MARIE SKŁODOWSKA-CURIE ACTIONS • NATIONAL CENTER FOR SUPERCOMPUTING APPLICATIONS • Salesforce Research Deep Learning Grant • AWS Machine Learning Research Awards • AI and deep learning Skilling and Research • Berkeley Artificial Intelligence Research (BAIR) • USC Information Sciences Institute • USC Center for AI in Society • NASA Jet Propulsion Laboratory • MIT CSAIL • The Alan Turing Institute • Fraunhofer Heinrich Hertz Institute • Oxford ML Research Reference: https://medium.com/swlh/what-are-a-few-ai-research-labs-on-the-west-coast-c0434996ad64 https://www.mariecuriealumni.eu/newsletter/10-websites-you-need-know-european-funding-opportunities
32. Shadab Hussain https://shadabhussain.com 32 AI Fellowship Programs OpenAI 2020 Spring Scholars Google AI Residency Program Microsoft AI Residency Program Facebook AI Residency For more fellowships https://github.com/dangkhoasdc/awesome-ai-residency
33. Shadab Hussain https://shadabhussain.com 33 • https://commons.wikimedia.org/wiki/File:Logistic-curve.svg • https://colah.github.io/posts/2015-08-Understanding-LSTMs/ • https://fr.wikipedia.org/wiki/Fichier:Recurrent_neural_network_unfold.svg • http://karpathy.github.io/2015/05/21/rnn-effectiveness/ • https://commons.wikimedia.org/wiki/File:RecurrentLayerNeuralNetwork_english.png • https://fr.m.wikipedia.org/wiki/Fichier:MultiLayerNeuralNetworkBigger_english.png • https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg • https://commons.wikimedia.org/wiki/File:Hyperbolic_Tangent.svg • https://commons.wikimedia.org/wiki/File:ReLU_and_Nonnegative_Soft_Thresholding_Functions.svg • https://commons.wikimedia.org/wiki/File:Sigmoid-function.svg • https://developer.nvidia.com/deep-learning-frameworks • https://towardsdatascience.com/top-10-best-deep-learning-frameworks-in-2019-5ccb90ea6de • https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/ References
34. Shadab Hussain https://shadabhussain.com 34 Thank You ☺ Questions?