Convolution Neural Network 

Hello! I am Mayank Mishra I am really excited about AI | ML You can find me at: @mayank_skb 2 

Line Up 1. What is Neural Network and Artificial Neural Network? 2. What is Convolution Neural Network? a. Convolution Neural Network b. What is an Image? c. Design of a Convnet d. Convolution Layer e. Activation Function f. Pooling Layer g. Fully Connected Layer 3. Let’s Summarize 4. Demo 3 

1. What is Neural Network and Artificial Neural Network ? 4 

Neural Network A neural circuit, is a population of neurons interconnected by synapses to carry out a specific function when activated. Neural Circuits interconnect to one another to form large scale brain networks. 5 

Artificial Neural Network A computer system modelled on the human brain and nervous system. What does it means?? An ANN is based on a collection of connected units or nodes called artificial neurons which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal from one artificial neuron to another. An artificial neuron that receives a signal can process it and then signal additional artificial neurons connected to it. 6 

Artificial Neural Network Ref : Course from Stanford University - CS231n (Computer Vision) 

Design of a Artificial Neuron A typical Artificial Neuron consist of two parts : 1. Input Dendrites 2. Cell Body a. Affine Transformation b. Activation Function 3. Output Axon 

Affine Transformation Design of a Artificial Neuron 

Design of Artificial Neural Network Basically, there are 3 different layers in a neural network :- 1. Input Layer (All the inputs are fed in the model through this layer) 2. Hidden Layers (There can be more than one hidden layers which are used for processing the inputs received from the input layers) 3. Output Layer (The data after processing is made available at the output layer) 

Artificial Neural Network 

2. Convolution Neural Network 12 

Convolution Neural Network Convolution Neural Network or CNN are similar to Fully Connected Neural Network. In terms like : 1. Neurons are made up of ;learnable weights and biases 2. Taking Input 3. Performs Affine Transformation 4. Passing it through Activation Function 5. Responds with output So How Convolutional Neural Network are different from other Networks?? 13 

Convolution Neural Network ● ConvNets makes the explicit assumption that inputs are images. ● They are not fully connected and are sparsely connected in architecture. ● The architecture of a CNN is designed to take advantage of the structure of an input image 

What is an Image? Image is a matrix of pixels 

Design of a ConvNet Layers in a ConvNet: 1. Convolution Layer 2. Activation Function 3. Pooling Layer 4. Fully Connected Layer 

Convolution Layer ● It is the core building block of the ConvNets ● Consist of set of learnable parameters ● One set is called as a filter ● Filters are small spatially but goes in full depth 

Convolution Layer Input Image Filter 

Convolution Layer 

Convolution Layer So there are three things : 1. Input Size (W) 2. Filter Size (F) 3. Stride Size (S) 4. Padding Size (P) How to compute the Size of the new Image Size = ((W - F + 2P) / S ) + 1 

Activation Function There are number of Activation function like: 1. Sigmoid Activation 2. Tanh Activation 3. ReLU 4. Leaky ReLU, etc. For Convolutional Neural Network mostly ReLU is used in practice. But Why?? 1. Accelerate the convergence of Gradient Descent by a factor of 6x in comparison to Sigmoid and tanh nonlinearity. 2. Compared to Sigmoid and tanh, ReLU is computationally efficient. 

Activation Function ReLU Nonlinearity ● Stands for Rectified Linear unit ● Computes the function f(x) = max(0, x) 

Pooling Layer ● Common to use periodically in-between successive conv layer ● Reduces the spatial size of the representation ● Reduces the amount of computation and weights in the network ● Operates on every depth individually Common Practices ● Use a max - pool layer of size 2 x 2 with stride size of 2 ● Downsamples the image by a factor of 2 both in width and height ● Decreased 75% of activations 

Pooling Layer 

Fully Connected Layer ● Neuron in FC layer have full connection to all activation in the previous layer ● The network is similar to a Regular Neural Network ● The purpose of the Fully Connected layer is to use high-level features for classifying the input image into various classes based on the training dataset. ● Their activation can hence be computed with a matrix multiplication followed by a bias addition Output = W * x + B Where , W = Matrix of weights for FC layer x = Input from previous layer B = Bias Offset 

Let’s Summarize 26 

Convolution Layer ● Accepts a volume of size W1 × H1 × D1 ● Requires four hyperparameters: ○ Number of filters K, ○ their spatial extent F, ○ the stride S, ○ the amount of zero padding P. ● Produces a volume of size W2 × H2 × D2 where: ○ W2 = ( W1 − F + 2P ) / S + 1 ○ H2 = ( H1 − F + 2P ) / S + 1 ○ D2 = K ● A common setting of the hyperparameters is F=3,S=1,P=1F=3,S=1,P=1. 

Pooling Layer ● Accepts a volume of size W1 × H1 × D1 ● Requires two hyperparameters: ○ their spatial extent F, ○ the stride S, ● Produces a volume of size W2 × H2 × D2 where: ○ W2 = ( W1 − F ) / S + 1 ○ H2 = ( H1 − F ) / S + 1 ○ D2 = D1 ● Note that it is not common to use zero-padding for Pooling layers ● A pooling layer with F = 3, S = 2 (also called overlapping pooling), and more commonly F = 2, S = 2. 

Demo 29 

Thanks! Any questions? You can find me at: @mayank_skb 30