Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ADDC 2019 - Dan Abdinoor: The NPU Revolution

ADDC 2019 - Dan Abdinoor: The NPU Revolution

Behind the use of artificial intelligence capabilities is a new and foundational piece of technology: the Neural Processing Unit. These AI-only processors are changing the rules for machine learning power and affordability, creating new and ideal conditions for intelligent devices. In this talk, we will explore the history, recent breakthroughs, and future impact, covering everything you need to know about the Age of the NPU.

Recordings & more: https://addconf.com/

More Decks by ADDC - App Design & Development Conference

Other Decks in Technology

Transcript

  1. The NPU
    Revolution
    Dan Abdinoor
    CEO & Co-founder, Fritz

    View Slide

  2. View Slide

  3. Newly possible with
    intelligence
    ● Vision
    ● Voice
    ● Autonomy
    Two Types of AI Solutions
    Improved with
    intelligence
    ● Translation
    ● User Input
    ● Storage & Retrieval

    View Slide

  4. View Slide

  5. Facebook AI Research Progress

    View Slide

  6. Growing AI Model Complexity

    View Slide

  7. Training Time (hours)
    Number of computer devices
    Simple to Scale in Cloud

    View Slide

  8. Not so simple
    to scale on Edge
    Limited space
    Limited power
    Limited connectivity

    View Slide

  9. Move the Intelligence not the Data

    View Slide

  10. Neural
    Processing
    Unit

    View Slide

  11. What is an
    NPU?

    View Slide

  12. Silicon
    Transistors
    Vacuum
    Tubes
    M
    icro-
    processors
    16-Bit
    32-Bit
    D
    ual-cores
    4-16
    Cores
    1950 1960 1970 1980 1990 2000 2010
    CPU
    GPU
    NPU
    Video
    signal
    output
    Perfectly parallel
    operations
    M
    atrix and
    vector
    operations
    Floating point
    operations
    Tensor cores
    H
    igh-volum
    e
    Low
    -precision
    Chronology of Processing Units

    View Slide

  13. CPU
    GPU
    NPU
    Increase Volume

    View Slide

  14. Lower Precision

    View Slide

  15. Limit Intermediate Data Fetching
    Tesla

    View Slide

  16. NPU Chips

    View Slide

  17. Apple Neural Engine
    8 Cores
    5 TFLOPS

    View Slide

  18. Huawei / HiSilicon Kirin 980

    View Slide

  19. Samsung Exynos

    View Slide

  20. View Slide

  21. Intel Mobileye

    View Slide

  22. Qualcomm Cloud AI 100

    View Slide

  23. Google Edge TPU

    View Slide

  24. Google Edge TPU Dev Board
    NVIDIA Jetson Nano

    View Slide

  25. Tesla FSD Hardware

    View Slide

  26. NPU
    Performance

    View Slide

  27. Real-world Performance Relative to iPhone X NPU

    View Slide

  28. View Slide

  29. NPU-powered
    Solutions

    View Slide

  30. iPhone Machine Learning Timeline
    Performance
    2015 2016 2017 2018
    A9
    iPhone 6S
    PowerVR GPU
    +”Hey Siri”
    + People in Photos
    A10 Fusion
    iPhone 7
    Apple GPU
    6 core
    50% faster
    + Portrait Mode
    A11 Bionic
    iPhone 8 + X
    Apple Neural Engine
    2 core
    0.6 teraflops
    + Face ID
    + Raise to wake
    A12 Bionic
    iPhone XS
    Apple Neural Engine
    8 core
    5 teraflops
    + Computational Photography
    + Developer Access to ANE

    View Slide

  31. Tesla

    View Slide

  32. Try-on a Bike
    Try-on Fashion

    View Slide

  33. Context-Aware Smart Home

    View Slide

  34. Healthcare Retail
    Photography
    Sports + Fitness

    View Slide

  35. The Future
    NPU

    View Slide

  36. More
    Efficient

    View Slide

  37. Mythic Low Power NPU

    View Slide

  38. Hailo 8

    View Slide

  39. Xnor Solar Camera

    View Slide

  40. More
    Powerful

    View Slide

  41. Graphcore IPU

    View Slide

  42. Quadric Edge Supercomputer

    View Slide

  43. Quadric Edge Supercomputer

    View Slide

  44. NPU
    Implications

    View Slide

  45. NPU
    Moore’s Law May Yet Continue

    View Slide

  46. Parkinson's Law
    Project workloads expand to fill the
    time allotted
    Abdinoor’s Law
    Software workloads expand to fill the
    computing resources available

    View Slide

  47. The NPU Revolution?
    1. AI solutions
    2. Here today
    3. More designs on the way
    4. Transformational

    View Slide

  48. Thank You
    Dan Abdinoor
    [email protected]
    Graphcore Visualization of Alexnet

    View Slide