Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning for developers

Lee Boonstra
November 02, 2018

Machine Learning for developers

Lee Boonstra

November 02, 2018
Tweet

More Decks by Lee Boonstra

Other Decks in Technology

Transcript

  1. Machine Learning for
    JavaScript developers
    Lee Boonstra
    Customer Engineer, Google
    Twitter: @ladysign

    View full-size slide

  2. 2
    Lee Boonstra
    @ladysign

    View full-size slide

  3. 4
    What’s Machine Learning

    View full-size slide

  4. Why now?

    Amount of data Better Models More Computing Power

    View full-size slide

  5. Machine
    Learning to
    classify
    things
    Dog vs. Mop
    Oh, that’s easy...

    View full-size slide

  6. We would need
    machine learning
    to give us results
    Confidence level

    View full-size slide

  7. It’s inspired by
    how our
    brains work
    Neural Networks

    View full-size slide

  8. It’s easier to make
    computers learn
    than to build
    smarter
    computers
    Artificial Intelligence
    Process of building smarter
    computers
    Machine Learning
    Process of making
    computers learn

    View full-size slide

  9. Why Google?
    Data Scientists use Tensorflow
    Tensorflow is what we use for our own internal machine
    learning projects, and now it’s available to you!
    Google made it open source.
    ● More than 480 contributions
    ● 10,000 commits in a year
    ● 53k star rating
    http://www.tensorflow.org

    View full-size slide

  10. Google is an AI company
    Used across products:
    Unique project directories
    Time

    View full-size slide

  11. Machine Learning use cases
    • Predictive maintenance or condition
    monitoring
    • Warranty reserve estimation
    • Propensity to buy
    • Demand forecasting
    • Process optimization
    Manufacturing
    • Predictive inventory planning
    • Recommendation engines
    • Upsell and cross-channel marketing
    • Market segmentation and targeting
    • Customer ROI and lifetime value
    Retail
    • Alerts and diagnostics from real-time
    patient data
    • Disease identification and risk satisfaction
    • Patient triage optimization
    • Proactive health management
    • Healthcare provider sentiment analysis
    Healthcare and Life Sciences
    • Aircraft scheduling
    • Dynamic pricing
    • Social media – consumer feedback and
    interaction analysis
    • Customer complaint resolution
    • Traffic patterns and congestion
    management
    Travel and Hospitality
    • Risk analytics and regulation
    • Customer Segmentation
    • Cross-selling and upselling
    • Sales and marketing campaign
    management
    • Creditworthiness evaluation
    Financial Services
    • Power usage analytics
    • Seismic data processing
    • Carbon emissions and trading
    • Customer-specific pricing
    • Smart grid management
    • Energy demand and supply optimization
    Energy, Feedstock and Utilities

    View full-size slide

  12. There’s a lack of ML expertise
    in the industry
    1000’s
    Deep Learning Researchers
    21M
    Developers
    <1M
    Data Scientists

    View full-size slide

  13. To create your own machine learning model
    it takes a complex & time intensive process
    UPDATE
    DEPLOY
    EVALUATE
    TUNE ML MODEL
    PARAMETERS
    ML MODEL DESIGN
    DATA
    PREPROCESSING
    Large computational resource . Machine learning expertise . Manual data labeling

    View full-size slide

  14. So how can Google
    Cloud help?
    Pre-Trained Models
    Train your own Conversation Solutions

    View full-size slide

  15. Introducing Cloud AI
    Less ML
    expertise
    More ML
    expertise
    Cloud AI solutions
    Cloud
    Job Discovery
    Contact
    Center
    Document
    Understanding
    Cloud AI building blocks
    Cloud AI platform
    Cloud ML
    Engine
    Cloud Video
    Intelligence
    ML professionals & service partners
    ASL
    Professional Services
    Organization
    Cloud
    AutoML Vision
    Vision
    Cloud
    Vision
    Language Conversation
    Cloud Natural
    Language
    Cloud
    AutoML NL
    Dialogflow
    Enterprise
    Cloud
    Translation
    Cloud
    Speech-to-Text
    Cloud
    Text-to-Speech
    Cloud AutoML
    Translation
    New
    New
    New
    Cloud
    GPU
    Cloud
    TPU
    Cloud
    Dataflow
    Cloud
    Dataproc
    Machine & Deep
    Learning
    ML accelerators ML libraries
    Tensorflow Kubeflow
    Kaggle/datasets
    Datasets

    View full-size slide

  16. Train your own:
    Tensorflow on Cloud ML Engine
    ● Fully managed service
    ● Train using a custom TensorFlow graph for any ML use cases
    with CPUs/GPUs
    ● Training at scale to shorten dev cycle
    ● Automatically maximize predictive accuracy with HyperTune
    ● High throughput batch predictions
    ● Low latency online predictions (Beta)
    ● Integrated Datalab experience
    Train your own

    View full-size slide

  17. Train your own:
    Use your own data to train
    models
    Train your own
    Cloud Datalab
    Cloud Machine Learning
    Cloud Storage Google BigQuery Develop/Model/Test

    View full-size slide

  18. Pre-trained models
    ML APIs
    ● Developer Machine Learning APIs. Google trained the model
    for you. Your developers just need to call an URL endpoint.
    ○ Understanding Images
    ○ Understanding Videos
    ○ Understanding Speech
    ○ Converting Text to Speech
    ○ Understanding Text
    ○ Translating Text

    Pre-Trained Models

    View full-size slide

  19. Vision API
    Detect broad sets of
    categories within an image,
    ranging from modes of
    transportation to animals.
    Analyze facial features to
    detect emotions: joy,
    sorrow, anger.
    Detect logos.
    Detect and extract text
    within an image, with
    support for a broad range of
    languages, along with
    support for automatic
    language identification.
    Extract text
    Detect different types of
    inappropriate content from
    adult to violent content.
    Powered by Google Safe
    Search
    Detect inappropriate content
    Object Recognition Facial sentiment & logos
    Pre-Trained Models

    View full-size slide

  20. Video Intelligence API
    Detect entities within the
    video, such as "dog",
    "flower" or "car".
    You can now search your
    video catalog the same way
    you search text
    documents..
    Extract actionable insights
    from video files without
    requiring any machine
    learning or computer vision
    knowledge.
    Enable Video Search
    Label Detection Insights From Videos
    Pre-Trained Models

    View full-size slide

  21. Speech API
    Powered by deep
    learning neural
    networking to power
    your applications..
    No need for signal
    processing or noise
    cancellation before
    calling API. Can
    handle noisy audio
    from a variety of
    environments.
    Noise Robustness
    Can provide context
    hints for improved
    accuracy. Especially
    useful for device and
    app use cases.
    Word Hints
    Speech Recognition
    Recognizes over 80
    languages & variants.
    Can also filter
    inappropriate content
    in text results
    Over 80 languages
    Can stream text
    results, returning
    partial recognition
    results as they
    become available.
    Can also be run on
    buffered or archived
    audio files.
    Real-time results
    Pre-Trained Models

    View full-size slide

  22. Natural Language API
    Identify entities and label by
    types such as person,
    organization, location,
    events, products and media.
    Enables you to easily
    analyze text in multiple
    languages including
    English, Spanish and
    Japanese.
    Extract tokens and
    sentences, identify parts of
    speech (PoS) and create
    dependency parse trees for
    each sentence.
    Syntax analysis
    Entity Recognition Multi-Language Support
    Understand the overall
    sentiment expressed in a
    block of text.
    Sentiment Analysis
    Pre-Trained Models

    View full-size slide

  23. Translation API
    Supports more than 100
    languages and thousands
    of language pairs.
    Behind the scenes,
    Translation API is learning
    from logs analysis and
    human translation
    examples. Existing
    language pairs improve and
    new language pairs come
    online at no additional cost.
    Sometimes you don’t know
    your source text language in
    advance. Can automatically
    identify languages with high
    accuracy.
    Automatic language
    detection
    The Premium edition is
    tailored for users who need
    precise, long-form
    translation services (e.g.
    livestream translations, high
    volume of emails, detailed
    articles and documents)
    Premium edition BETA
    Text Translation Continuous Updates
    Pre-Trained Models

    View full-size slide

  24. Translation API
    Pre-Trained Models

    View full-size slide

  25. Auto ML
    Custom task
    Generic task
    Someone else has
    solved this before
    Trained on common
    classes
    Specific to
    your dataset
    ML APIs TensorFlow
    “cat”
    “bob”
    AUTO ML
    Developer
    Data Scientist
    Developer or
    Data scientist

    View full-size slide

  26. How it works
    AutoML Vision
    Photo dataset
    Train Deploy Serve
    Generate predictions
    with a REST API
    Auto ML

    View full-size slide

  27. Via a web interface you will
    go through these steps
    Auto ML
    1 3 4
    5
    Upload Labeled
    Data
    Evaluate Predict
    Iterate
    (if needed)
    2
    Train

    View full-size slide

  28. Optimize the pre-trained
    models with your own data
    Auto ML
    ● Create models for your own domain, but use the Google
    pre-trained models as a base.
    ○ Vision, Translation, or NLP
    ● Web Interface, to upload a CSV with labeled data.
    ● Your use-case is:
    ○ Not supported by pre-built APIs AND
    ○ Has sufficient labeled training data
    ● You want to get to produce a model and predict quickly
    ● You don’t want to build a model from scratch
    Auto ML

    View full-size slide

  29. 35
    50% of enterprises spend more time
    on creating bots than on mobile app
    development by 2021
    —Gartner
    Conversation Solutions

    View full-size slide

  30. 36
    AI to improve your customer experience
    Voice Activated Speakers &
    smart assistants.
    Chatbots in websites, apps
    or social media platforms.
    AI in callcenters
    Conversation Solutions

    View full-size slide

  31. Meet the Google Assistant
    It’s your own personal
    Google, always ready
    to help.

    View full-size slide

  32. 38
    ● Write the conversation - Dialogflow (Enterprise)
    ● Deploy on GA+ UX components - Actions on Google
    What you need to build
    your own action
    Conversation Solutions

    View full-size slide

  33. 39
    ● Previously known as API.AI
    ○ (Sept 2016, acquired by Google)
    ● Powered by Machine Learning:
    ○ Natural Language Understanding (NLU)
    ○ Intent Matching
    ○ Conversation Training
    ● Cross platform
    ● Build faster with the Web UI
    ● Scalable: separate your conversation text from
    code
    ● Speech / Voice Integration
    ● Multi-lingual bot support (20+ languages)
    ● Part of Google Cloud (60+ cloud services)
    Development
    suite for building
    Conversational
    UIs.
    Conversation Solutions

    View full-size slide

  34. 40
    ● Bring your agents to smart speakers (Google
    Home) or phones (Android, iOS app)
    ● Actions on Google includes:
    ○ UI toolkit,
    ○ Audio toolkit
    ○ Account Linking API
    ○ SDKs
    ○ tutorial guides
    ● UI components such as:
    ○ Buttons, Images
    ○ Cards, Carousels,
    ○ Lists
    ○ Tables
    Program for developers of
    Actions (“apps”) that run via
    Google Assistant
    Actions on Google
    Conversation Solutions

    View full-size slide

  35. 41
    Terminology
    Google Assistant — The virtual assistant of
    Google.
    Out of the box on Android 6+. For iOS available as
    app.
    Action — A third party app, running on top of the
    Google Assistant.
    Google Home — Voice-activated speaker powered
    by Voice.
    Smart Display — Voice-activated speaker with
    screen powered by Voice and Touch.
    Conversation Solutions

    View full-size slide

  36. 42
    3rd party integration
    Extend the Google
    Assistant
    with your own custom
    actions.
    Hey Google, let me talk to Babs The Banking Bot
    Welcome, how can I help you?
    I want to transfer money.
    Let’s get Babs the Banking Bot
    How much do you want to transfer?
    100 euro.
    Conversation Solutions

    View full-size slide

  37. Confidential + Proprietary
    Confidential + Proprietary
    Demo

    View full-size slide

  38. 44
    Sure!
    I’d like to transfer 100 euro
    to my wife her bank
    account.”
    A customer communicates with
    the Google Assistant

    View full-size slide

  39. 45
    What happens under the hood...

    View full-size slide

  40. Confidential + Proprietary
    Which customers are unhappy and why?

    View full-size slide

  41. Confidential + Proprietary
    How can I improve the user experience?

    View full-size slide

  42. 48
    Collect real-time chats
    from Dialogflow SDK

    View full-size slide

  43. 49
    Mask sensitive
    Information
    with DLP API

    View full-size slide

  44. 50
    Understand the text
    with NLP API

    View full-size slide

  45. 51
    Store all data in
    a data-warehouse

    View full-size slide

  46. 52
    Optimize your agent

    View full-size slide

  47. Confidential + Proprietary
    Advanced Chatflow with machine learning bot analytics
    User types to custom UI
    or channel
    Chatbot replies
    Dialogflow
    Enterprise
    Customer Client
    JS Angular 5 web front-end
    Kubernetes Engine
    Chat Server
    Dialogflow SDK / socket.io
    Kubernetes Engine
    Back-end CRM
    Python / Django
    Kubernetes Engine
    Container
    Registry
    Containers images can be
    stored in the Container Registry
    Messaging Publisher
    Pub/Sub
    Webhook
    Router
    Cloud Function
    Webhook
    Container
    Builder
    Building Dev
    Pipelines

    View full-size slide

  48. Confidential + Proprietary
    Advanced Chatflow with machine learning and bot analytics
    User types to custom UI
    or channel
    Chatbot replies
    Dialogflow
    Enterprise
    Customer Client
    JS Angular 5 web front-end
    Kubernetes Engine
    Chat Server
    Dialogflow SDK / socket.io
    Kubernetes Engine
    Back-end CRM
    Python / Django
    Kubernetes Engine
    Subscription
    Cloud Function
    Sensitivity
    Filter
    DLP API
    Sentiment
    Detector
    NLP API
    Data
    Warehouse
    BigQuery
    Messaging Publisher
    Pub/Sub
    Webhook
    Router
    Cloud Function
    Webhook

    View full-size slide

  49. Developers can use machine learning too!
    ● Tensorflow (JS or Python)
    ● With out of the box solutions for Speech, Video, Image or Text
    ● Training of custom models by uploading CSV data
    ● With solutions for call centers, smart speakers or chatbots.

    View full-size slide

  50. Lee Boonstra - @ladysign
    Thank you!

    View full-size slide