Types of Convolution Layer

Transcript

1. W H C KH KW C K kernels K OW OH Number of parameters in Basic Convolution → KH * KW * C * K Basic Convolution take sum
2. 1 W H C 1 C K OW OH Number of parameters in Basic Convolution → KH * KW * C * K Number of parameters in Pointwiase Convolution → 1 * 1 * C * K Pointwise Convolution is a type of convolution that uses a 1x1 kernel: a kernel that iterates through every single point. This kernel has a depth of however many channels the input image has Pointwise Convolution K kernels
3. C W H KH KW C kernels C OW OH Depthwise Convolution Depthwise (channelwise) Convolution is a type of convolution where we apply a single convolutional filter for each input channel.
4. C W H KH KW C kernels C OW’ OH’ Depthwise Separable Convolution • number of parameters in Basic Convolution (input: H*W*8, output: OH*OW*16, kerel size: 3*3) KH*KW*C*K = 3*3*8*16 = 1152 • number of parameters when performing the same conversion as above using Depthwise Separable Convolution KH*KW*C + C*K = 3*3*8 + 8*16 = 200 K kernels 1 1 C K OW OH Pointwise Convolution Depthwise Convolution
5. C W H 1 1 Vector of c dimension SE Module bottleneck FC layer (c → c/r → c) W H C Average Pooling channel weight vector of c dimension