Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning End to End at Surat

Machine Learning End to End at Surat

This is presented in GDG Surat Devfest. It starts with basic introduction of Machine Learning and ends with the deployment of model using flask.

B1a1cc3d71600c6e47c33c65fa08f71f?s=128

Krunal Kapadiya

November 18, 2018
Tweet

More Decks by Krunal Kapadiya

Other Decks in Technology

Transcript

  1. Krunal Kapadiya, Volansys @krunal3kapadiya Surat Machine Learning End to End

  2. None
  3. None
  4. 4

  5. Applications of Machine Learning

  6. Two ways Google can help us! Surat

  7. What is Machine Learning? Input Output Magic

  8. What is Machine Learning? "A computer program is said to

    learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.” - Tom. M. Mitchell
  9. Proprietary + Confidential

  10. None
  11. Set Goal Split Data Train Model Results Run an ML

    Experiment Steps to follow Test Model Surat
  12. Training and splitting data with validations Steps to follow 20%

    Test case 80% Training set Total numbers of training set
  13. Training and splitting data with validations Steps to follow 15%

    Test set 70% Training set Total numbers of training set 15% Validation
  14. Find apple or orange problem Traditional approach to solve problem

  15. Find apple or orange problem Traditional approach to solve problem

  16. Find apple or orange problem Traditional approach to solve problem

  17. Weight Texture Label 150g Bumpy Orange 170g Bumpy Orange 140g

    Smooth Apple 130g Smooth Apple Feature Feature Find apple or orange problem Training Data
  18. Weight Texture Label 150g Bumpy Orange 170g Bumpy Orange 140g

    Smooth Apple 130g Smooth Apple Feature Feature Examples Find apple or orange problem Training Data
  19. Orange Apple Weight = 150 G Yes No Yes No

    Texture = bumpy ? ... Decision Tree Find apple or orange problem
  20. None
  21. Types of Machine Learning Surat

  22. Types of Machine learning Based on Human supervision - Supervised

    - Unsupervised - Reinforcement
  23. Other types of Machine learning Based on learning Online learning

    Offline learning Based on Data patterns Instance based Model based
  24. Supervised model

  25. Algorithms in supervised learning - K Nearest Neighbours - Linear

    regressions - Logistic regressions - Support vector machines - Decision tree and random forests
  26. Show me code Surat

  27. # Select a linear model model = sklearn.linear_model.LinearRegression() # Train

    the model model.fit(X, y) # Make a prediction for Cyprus X_new = [[22587]] # Cyprus' GDP per capita print(model.predict(X_new)) # outputs [[ 5.96242338]]
  28. Unsupervised model

  29. Algorithms in unsupervised learning - Clustering - k-Means - Hierarchical

    cluster analysis (HCA) - Association rule learning - Apriori - Eclat
  30. Problems: Customer segmentation We have data, it is in csv

    format having rows - Customer Id - Gender - Age - Annual Income - Spending score
  31. kmeans = KMeans(n_clusters=5, init='k-means++', random_state=0) y_kmeans = kmeans.fit_predict(X) # will

    return the predicted class
  32. Problems: Customer segmentation

  33. Reinforcement learning

  34. Algorithms in Reinforcement learning - Q-learning - SARSA - DQN

    - DDPG - Multi arm bandit problem
  35. Neural network

  36. Convolutional Neural Network

  37. - Pooling

  38. Show me code Surat

  39. x = tf.reshape(x, shape=[-1, 28, 28, 1]) # Convolution Layer

    with 32 filters and a kernel size of 5 conv1 = tf.layers.conv2d(x, 32, 5, activation=tf.nn.relu) # Max Pooling (down-sampling) with strides of 2 and kernel size of 2 conv1 = tf.layers.max_pooling2d(conv1, 2, 2) # Convolution Layer with 64 filters and a kernel size of 3 conv2 = tf.layers.conv2d(conv1, 64, 3, activation=tf.nn.relu) # Max Pooling (down-sampling) with strides of 2 and kernel size of 2 conv2 = tf.layers.max_pooling2d(conv2, 2, 2)
  40. # Flatten the data to a 1-D vector for the

    fully connected layer fc1 = tf.contrib.layers.flatten(conv2) # Fully connected layer (in tf contrib folder for now) fc1 = tf.layers.dense(fc1, 1024) # Apply Dropout (if is_training is False, dropout is not applied) fc1 = tf.layers.dropout(fc1, rate=dropout, training=is_training) # Output layer, class prediction out = tf.layers.dense(fc1, n_classes)
  41. input_fn = tf.estimator.inputs.numpy_input_fn( x={'images': mnist.train.images}, y=mnist.train.labels, batch_size=batch_size, num_epochs=None, shuffle=True) #

    Train the Model model.train(input_fn, steps=num_steps)
  42. But, real problems are... - Insufficient quantity of training data

    - Non representative training data - Poor quality data - Irrelevant features - Overfitting - Underfitting
  43. Lastly, how to publish model in production Surat

  44. Add ML model in production using

  45. header = {'Content-Type': 'application/json', \ 'Accept': 'application/json'} """Reading test batch

    """ df = pd.read_csv('../data/test.csv', encoding="utf-8-sig") df = df.head() """Converting Pandas Dataframe to json """ data = df.to_json(orient='records') Add ML model in production using flask Source: https://www.analyticsvidhya.com/blog/2017/09/machine-learning-models-as-apis-using-flask/
  46. //OUTPUT [ { "Loan_ID": "LP001015", "Gender": "Male", "Married": "Yes", "Dependents":

    "0", "Education": "Graduate", "Self_Employed": "No", "ApplicantIncome": 5720, "CoapplicantIncome": 0, "LoanAmount": 110, "Loan_Amount_Term": 360, "Credit_History": 1, "Property_Area": "Urban" } ] Add ML model in production using flask Source: https://www.analyticsvidhya.com/blog/2017/09/machine-learning-models-as-apis-using-flask/
  47. How can I start it - Look at the dataset

    - Write down columns and it’s correlation - Make questions derived from the dataset - Explanatory Analysis with visualization - Frame problem - Create solution by creating model
  48. Where to start learning ML • Data analytics vidhya •

    KDnuggets • Coursera and Udacity course
  49. Start learning by own • https://developers.google.com/machine-learning/crash-course/ml-intro • https://www.kaggle.com/learn/overview • https://www.tensorflow.org/tutorials/

    • https://pandas.pydata.org/pandas-docs/stable/10min.html
  50. ML Wordtoit is available in Assistant Link:- http://bit.ly/mlwordtoit

  51. Krunal Kapadiya, Volansys @krunal3kapadiya Surat Thank you! Q&A #IndiaMLCC

  52. None