Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Having fun with Google Cloud + RasPi

Having fun with Google Cloud + RasPi

Kazunori Sato

January 30, 2016
Tweet

More Decks by Kazunori Sato

Other Decks in Programming

Transcript

  1. Having fun with
    Google Cloud + RasPi

    View Slide

  2. +Kazunori Sato
    @kazunori_279
    Kaz Sato
    Staff Developer Advocate,
    Tech Lead for Data & Analytics
    Cloud Platform, Google Inc.

    View Slide

  3. = The Datacenter as a Computer

    View Slide

  4. View Slide

  5. Enterprise

    View Slide

  6. Jupiter network
    40 G ports
    10 G x 100 K = 1 Pbps total
    CLOS topology
    Software Defined Network

    View Slide

  7. Borg
    No VMs, pure containers
    Manages 10K machines / Cell
    DC-scale proactive job sched
    (CPU, mem, disk IO, TCP ports)
    Paxos-based metadata store

    View Slide

  8. Google BigQuery

    View Slide

  9. 1 B
    1 B 100 B 900 M

    View Slide

  10. At Google, MapReduce is classic.
    We use BigQuery.
    Confidential & Proprietary
    Google Cloud Platform 10

    View Slide

  11. Google BigQuery
    Demo: RegEx + GROUP BY on 10 B rows
    response read RegEx
    10B
    ~10sec 372GB

    View Slide

  12. SELECT your_data FROM billions_of_rows
    WHERE full_disk_scan_required = true;
    Scanning 1 TB in 1 sec
    with 5,000 - 10,000 disk spindles

    View Slide

  13. BigQuery Analytic Service in the Cloud
    BigQuery
    Analyze Export
    Import
    How to use BigQuery?
    Google
    Analytics
    ETL tools
    Connectors
    Google Cloud
    BI tools and
    Visualization
    Google Cloud
    Spreadsheets, R,
    Hadoop

    View Slide

  14. “The World's Largest IoT platform” with
    RasPi + BigQuery

    View Slide

  15. Blazingly Fast
    Capable of scanning 100B rows in ~20 sec
    Low Cost
    Storage: $0.020 per GB per month
    Queries: $5 per TB
    Fully Managed
    Use thousands of servers with zero-ops
    SQL
    Simple and Intuitive SQL with JS UDF
    Benefits of BigQuery

    View Slide

  16. Confidential & Proprietary
    Google Cloud Platform 16
    Google Brain

    View Slide

  17. View Slide

  18. View Slide

  19. ● Google Now
    ● Google Photos
    ● Gmail
    ● YouTube
    ● and more
    Google Brain: The Brain of Google services

    View Slide

  20. View Slide

  21. View Slide

  22. Confidential & Proprietary
    Google Cloud Platform 22
    Google Cloud Vision API

    View Slide

  23. Demo Video

    View Slide

  24. 24
    Types of Detection
    ● Label
    ● Landmark
    ● Logo
    ● Face
    ● Text
    ● Safe search

    View Slide

  25. 25
    Types of Detection
    Face Detection
    ○ Find multiple faces
    ○ Location of eyes, nose, mouth
    ○ Detect emotions: joy, anger,
    surprise, sorrow
    Entity Detection
    ○ Find common objects and
    landmarks, and their location in
    the image
    ○ Detect explicit content

    View Slide

  26. Making a request
    {
    "requests":[
    {
    "image": {
    "content": "base64ImageString"
    },
    "features": [
    {
    "type": "LABEL_DETECTION",
    "maxResults": 10
    },
    {
    "type": "FACE_DETECTION",
    "maxResults": 10
    },
    // More feature detection types...
    ]
    }
    ]
    }

    View Slide

  27. Confidential & Proprietary
    Google Cloud Platform 27
    TensorFlow

    View Slide

  28. A new system for distributed, parallel machine learning:
    ● Based on general-purpose dataflow graphs
    ● Targeting heterogeneous devices
    ○ single PC with CPU
    ○ single PC with GPU(s)
    ○ mobile device
    ○ clusters of 100s or 1000s of CPUs and GPUs
    What is TensorFlow?

    View Slide

  29. Portable
    ● Runs on many platforms
    ○ Data Center
    ○ CPUs and GPUs
    ○ Mobile phones

    View Slide

  30. Yet another dataflow systemwith tensors
    MatMul
    Add Relu
    biases
    weights
    examples
    labels
    Xent
    Edges are N-dimensional arrays: Tensors

    View Slide

  31. Distributed

    View Slide

  32. Simple Example

    View Slide

  33. Thank you

    View Slide