Slide 1

Slide 1 text

Categorization using Neural Networks by Fatai Salami (Phatye)

Slide 2

Slide 2 text

● Fatai Salami ● Software Developer – Lecturer – Researcher ● [email protected] or @phatye Profile

Slide 3

Slide 3 text

● Sometimes referred to as classification ● categorization is assigning a category ci to document di What is Categorization

Slide 4

Slide 4 text

● text categorization (e.g., spam filtering) ● fraud detection ● optical character recognition ● machine vision (e.g., face detection) ● natural-language processing (e.g., spoken language understanding) ● market segmentation (e.g.: predict if customer will respond to promotion) ● Bioinformatics (e.g., classify proteins according to their function) Categorization Tasks

Slide 5

Slide 5 text

● IT DEPENDS ● How large is your training set?, How large is your feature space? Data Set linearly separable? ● Quality data trumps everything What Algorithm should I use?

Slide 6

Slide 6 text

Case Study: Will s/he run away with my money?

Slide 7

Slide 7 text

● Clean data ● Identify possible features ● Separate data in the ratio 60 (train) : 20 (validate) : 20 (test) Getting data

Slide 8

Slide 8 text

● Organization ● State of residence ● Area of residence ● Monthly income ● Gender ● Marital status ● No of dependants ● How much borrowed? ● Account balance at middle of the month ● Credit score etc… Identify features

Slide 9

Slide 9 text

Good to have all features in the range -1 to +1 ● numeric x-data can be Gaussian normalized (really depends on your data) ● binary x-data is (-1 +1) encoded ● categorical x-data is 1-of-(C-1) effects-coded ( ex: [0,1] or [1,0] or [-1,-1] ) Prepare features for Neural Network

Slide 10

Slide 10 text

● numeric y-data is left alone ● binary y-data is 1-of-C dummy-coded ● categorical y-data is 1-of-C dummy-coded

Slide 11

Slide 11 text

Design Neural Network Model

Slide 12

Slide 12 text

● Input layer – only one layer. Neurons determined by number of features to be used ● Output layer – only one layer. Usually one neuron except for multiclass classification (run away: 0, return money: 1) Introducing “maybe” = 3 neurons (run away: {1,0,0}, maybe: {0,1,0}, return money:{0,0,1}

Slide 13

Slide 13 text

● Hidden layers – linearly separable data requires no hidden layer. Rare problems requiring more than two hidden layer. One is usually enough. There are many rule-of-thumb methods: ● The number of hidden neurons should be between the size of the input layer and the size of the output layer. ● The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer. ● The number of hidden neurons should be less than twice the size of the input layer.

Slide 14

Slide 14 text

● Too few hidden neurons result in under-fitting (model is not able to get good results) ● Too many hidden neurons results in over-fitting (model fits well with training data but does not perform well with test data)

Slide 15

Slide 15 text

Always better to use existing library/services ● Azure machine learning services ● Weka ● C# - Aforge.Net, NeuronDotNet ● Java – Neuroph ● Python – PyBrain, NeuroLab ● Php - Neural Mesh ● R - nnet Coding the Neural Network

Slide 16

Slide 16 text

● Epoch – number of iterations (too much can result to model over-fitting) ● Learning rate – how much an updating step influences the current value of the weights (too small – runs for long; too large – can go past optimum weights) ● Momentum – used to diminish the fluctuations in weight changes over consecutive iterations Network Configuration Terms

Slide 17

Slide 17 text

Training using Aforge.Net

Slide 18

Slide 18 text

Predicting using Aforge.Net

Slide 19

Slide 19 text

● Analysing the result

Slide 20

Slide 20 text

Where: ● Vi = Original output of data ● Vo = Output from neural net ● Vl = length of data ● TP = True Positive – accurately predicted valid for a valid transaction. ● FP = False Positive – predicted valid for a fraudulent transaction. ● FN = False Negative – predicted fraudulent for a valid transaction.

Slide 21

Slide 21 text

Thank you