Tweet
Tweet

Slide 1

Slide 1 text

Shadab Hussain https://shadabhussain.com Deep Learning In Neural Networks 1

Slide 2

Slide 2 text

Shadab Hussain https://shadabhussain.com 2 About Me

Slide 3

Slide 3 text

Shadab Hussain https://shadabhussain.com 3 Agenda • Introduction to Deep Learning • Artificial Neural Networks • Activation function • CNN • Confusion matrix • RNN • LSTM • Applications • Tensor-flow playground • Deep Learning Frameworks • Learning ML/DL • Research Labs and Funding • AI Fellowships

Slide 4

Slide 4 text

Shadab Hussain https://shadabhussain.com 4 Deep Learning

Slide 5

Slide 5 text

Shadab Hussain https://shadabhussain.com 5 The neuron collects signals from input channels named dendrites, processes information in its nucleus, and then generates an output in a long thin branch called axon. NUCLEUS AXON DENDRITES X1 X2 X3 NEURON W1 W2 W3 ANN

Slide 6

Slide 6 text

Shadab Hussain https://shadabhussain.com 6 INPUTS/INDEPENDENT VARIABLES 1 F b = (1 1 + 2 2 + 3 3 + ) 2 3 Activation Function

Slide 7

Slide 7 text

Shadab Hussain https://shadabhussain.com 7 Sigmoid

Slide 8

Slide 8 text

Shadab Hussain https://shadabhussain.com 8 ReLU

Slide 9

Slide 9 text

Shadab Hussain https://shadabhussain.com 9 tanh

Slide 10

Slide 10 text

Shadab Hussain https://shadabhussain.com 10 Convolutional Neural Network WHAT ARE CONVOLUTIONAL NEURAL NETWORKS (CNNS) AND HOW DO THEY LEARN? T-SHIRT/TOP TROUSER PULLOVER DRESS COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING

Slide 11

Slide 11 text

Shadab Hussain https://shadabhussain.com 11 Feature Detectors KERNELS/ FEATURE DETECTORS FEATURE MAPS

Slide 12

Slide 12 text

Shadab Hussain https://shadabhussain.com 12 CNN 0 1 1 0 1 1 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 1 0 1 1 0 0 0 1 0 0 1 0 FEATURE DETECTOR IMAGE 1 1 1 3 1 1 2 3 1 FEATURE MAP Live Convolution: http://setosa.io/ev/image-kernels/

Slide 13

Slide 13 text

Shadab Hussain https://shadabhussain.com 13 Applying ReLU T-SHIRT/TOP TROUSER PULLOVER DRESS COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING

Slide 14

Slide 14 text

Shadab Hussain https://shadabhussain.com 14 Pooling • Live illustration : http://scs.ryerson.ca/~aharley/vis/conv/flat.html 1 1 3 4 3 6 2 8 3 9 1 0 1 3 3 4 6 8 9 4 MAX POOLING 2x2 STRIDE = 2 6 8 9 4 FLATTENING

Slide 15

Slide 15 text

Shadab Hussain https://shadabhussain.com 15 HOW TO IMPROVE CNN KERNELS/ FEATURE DETECTORS 64 INSTEAD OF 32

Slide 16

Slide 16 text

Shadab Hussain https://shadabhussain.com 16 Confusion Matrix PREDICTIONS TRUE CLASS TRUE + TRUE - + - + - FALSE + FALSE - TYPE II ERROR TYPE I ERROR

Slide 17

Slide 17 text

Shadab Hussain https://shadabhussain.com 17 Recurrent Neural Network

Slide 18

Slide 18 text

Shadab Hussain https://shadabhussain.com 18 RNN Architecture TIME

Slide 19

Slide 19 text

Shadab Hussain https://shadabhussain.com 19 LSTM NETWORKS RNN PERFORMS WELL SINCE THE GAP BETWEEN THE PREDICTION “GREEN” AND THE NECESSARY CONTEXT INFORMATION “TREE” IS SMALL 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 The tree color is “green” 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 I live in Quebec in Northern Canada……where I live, the weather is generally “cold” most of the year +1 +2 ℎ ℎ+1 ℎ+2 RNN PERFORMS POORLY WHEN THE GAP BETWEEN THE PREDICTION “COLD” AND THE NECESSARY CONTEXT INFORMATION “CANADA” IS LARGE

Slide 20

Slide 20 text

Shadab Hussain https://shadabhussain.com 20 LSTM Intuition VANILLA RECURRENT NEURAL NETWORK LONG SHORT TERM MEMORY NETWORK THIS HORIZONTAL LINE (MEMORY) OR CELL STATE ENABLES LSTM TO REMEMBER VERY OLD INFORMTION

Slide 21

Slide 21 text

Shadab Hussain https://shadabhussain.com 21 Applications of DL Face Recognition

Slide 22

Slide 22 text

Shadab Hussain https://shadabhussain.com 22 Applications of DL Fake News

Slide 23

Slide 23 text

Shadab Hussain https://shadabhussain.com 23 Applications of DL StyleGAN

Slide 24

Slide 24 text

Shadab Hussain https://shadabhussain.com 24 Applications of DL Natural Language Processing

Slide 25

Slide 25 text

Shadab Hussain https://shadabhussain.com 25 Applications of DL Voice Assistant

Slide 26

Slide 26 text

Shadab Hussain https://shadabhussain.com 26 Applications of DL Self Driving Cars

Slide 27

Slide 27 text

Shadab Hussain https://shadabhussain.com 27 TensorFlow Playground •

Slide 28

Slide 28 text

Shadab Hussain https://shadabhussain.com 28 Deep Learning Frameworks

Slide 29

Slide 29 text

Shadab Hussain https://shadabhussain.com 29 DL Frameworks https://colab.research.google.com/ https://www.kaggle.com/ https://gradient.paperspace.com/free-gpu

Slide 30

Slide 30 text

Shadab Hussain https://shadabhussain.com 30 Learning ML/DL • Faculty Awards to Support Machine Learning Courses, Diversity, and Inclusion • Deep Learning Specialization • Fast.ai • Deep Learning A-Z • Deep Learning Nanodegree Program • MIT Deep Learning and Self Driving Cars

Slide 31

Slide 31 text

Shadab Hussain https://shadabhussain.com 31 Research Labs & Funding • MARIE SKŁODOWSKA-CURIE ACTIONS • NATIONAL CENTER FOR SUPERCOMPUTING APPLICATIONS • Salesforce Research Deep Learning Grant • AWS Machine Learning Research Awards • AI and deep learning Skilling and Research • Berkeley Artificial Intelligence Research (BAIR) • USC Information Sciences Institute • USC Center for AI in Society • NASA Jet Propulsion Laboratory • MIT CSAIL • The Alan Turing Institute • Fraunhofer Heinrich Hertz Institute • Oxford ML Research Reference: https://medium.com/swlh/what-are-a-few-ai-research-labs-on-the-west-coast-c0434996ad64 https://www.mariecuriealumni.eu/newsletter/10-websites-you-need-know-european-funding-opportunities

Slide 32

Slide 32 text

Shadab Hussain https://shadabhussain.com 32 AI Fellowship Programs OpenAI 2020 Spring Scholars Google AI Residency Program Microsoft AI Residency Program Facebook AI Residency For more fellowships https://github.com/dangkhoasdc/awesome-ai-residency

Slide 33

Slide 33 text

Shadab Hussain https://shadabhussain.com 33 • https://commons.wikimedia.org/wiki/File:Logistic-curve.svg • https://colah.github.io/posts/2015-08-Understanding-LSTMs/ • https://fr.wikipedia.org/wiki/Fichier:Recurrent_neural_network_unfold.svg • http://karpathy.github.io/2015/05/21/rnn-effectiveness/ • https://commons.wikimedia.org/wiki/File:RecurrentLayerNeuralNetwork_english.png • https://fr.m.wikipedia.org/wiki/Fichier:MultiLayerNeuralNetworkBigger_english.png • https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg • https://commons.wikimedia.org/wiki/File:Hyperbolic_Tangent.svg • https://commons.wikimedia.org/wiki/File:ReLU_and_Nonnegative_Soft_Thresholding_Functions.svg • https://commons.wikimedia.org/wiki/File:Sigmoid-function.svg • https://developer.nvidia.com/deep-learning-frameworks • https://towardsdatascience.com/top-10-best-deep-learning-frameworks-in-2019-5ccb90ea6de • https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/ References

Slide 34

Slide 34 text

Shadab Hussain https://shadabhussain.com 34 Thank You ☺ Questions?