Introduction To Deep Learning Joshua Ndemenge @Jcardif254

Agenda • Concepts of Deep Learning • Deep Learning Frameworks • Building blocks of a Deep Neural Network • Deep Learning on Azure • Demo on Azure ML Studio • Building a Deep Neural Network in azure • Questions & Discussions

Concepts of Deep Learning

AI is the field of inducing intelligence into a machine or system artificially, with or without explicit programming. Machine learning can be defined as the process of inducing intelligence into a system or machine without explicit programming it. DL is a subfield of machine learning (ML) in artificial intelligence (AI) that deals with algorithms inspired from the biological structure and functioning of a brain to aid machines with intelligence.

Deep Learning Models Designed using neural network architecture A neural net is a hierarchical organization of neurons with connections to other neurons.

Deep Learning Frameworks

Deep Learning Frameworks Low Level DL Frameworks • Theano • Torch • PyTorch • MxNet • Tensorflow • Microsoft CNTK • Caffe High Level DL Frameworks • Keras • Gluon • Lasagne

Building Blocks of a Deep Neural Network

Input Data • A DL model understands data as tensors. • Tensors are generic form for vectors (a simple n- Dimension matrix) • A DL model can only interpret numeric data

Neurons

Activation Function ,f(z) An activation function is the function that takes the combined input z applies a function on it, and passes the output value, thus trying to mimic the activate/deactivate function. The activation function, therefore, determines the state of a neuron by computing f(z) on z. Without the Activation function : • The range of the output would be -∞ ≤ output ≤ ∞ • The network will not really learn hence rendered useless f(z)

Types of Activation Functions 1. Sigmoid Activation Function 2.ReLU Activation Function f(z) = 1 1+− f(z) = +1

Types of Activation Functions 3. Leaky ReLU 2.Tanh Squashing Activation Function f(z) =µz Where µ is a parameter defined as a small constant e.g. 0.005 f(z) =1−−2 1+−2

Types of Activation Functions 3. Sigmoid Squashing Function Other Activation Function include: • Tanh • Swish Activation Function • eLU (exponential Linear Unit) • SeLU (Scaled eLU) f(z) = 1+

Model The overall structure of a DNN is developed using the model object in Keras. This provides a simple way to create a stack of layers by adding new layers one after the other. The easiest way to define a model is by using the sequential model, which allows easy creation of a linear stack of layers.

Layers A layer in the DNN is defined as a group of neurons or a logically separated group in a hierarchical network structure. Some important Layers include: 1. Dense Layer 2. Dropout Layer 3. Convolution Layers 4. Recurrent Layers

The Loss Function This is a metric that helps the network understand whether it is learning in the right direction. Based on the type of data outcome, we have several standard loss functions defined in ML and DL. For regression use cases • Mean Square Error • Mean Absolute Error • Mean Absolute Percentage Error • Mean Square Logarithmic Error For classification use cases • Binary Loss Entropy • Categorical loss Entropy

No content

Demo on Neural Net Playground

No content

Deep Learning on Azure

Machine Learning/AI Stack at Microsoft Microsoft Cognitive Services Azure Machine Learning Service Machine Learning Studio SaaS PaaS Code First, Hybrid execution Model

Demo on Azure ML Studio

Building a Deep Neural Network in azure

Questions & Discussions

Thank You