Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ADDC 2019 - Dan Abdinoor: The NPU Revolution

ADDC 2019 - Dan Abdinoor: The NPU Revolution

Behind the use of artificial intelligence capabilities is a new and foundational piece of technology: the Neural Processing Unit. These AI-only processors are changing the rules for machine learning power and affordability, creating new and ideal conditions for intelligent devices. In this talk, we will explore the history, recent breakthroughs, and future impact, covering everything you need to know about the Age of the NPU.

Recordings & more: https://addconf.com/

More Decks by ADDC - App Design & Development Conference

Other Decks in Technology

Transcript

  1. The NPU
    Revolution
    Dan Abdinoor
    CEO & Co-founder, Fritz

    View full-size slide

  2. Newly possible with
    intelligence
    ● Vision
    ● Voice
    ● Autonomy
    Two Types of AI Solutions
    Improved with
    intelligence
    ● Translation
    ● User Input
    ● Storage & Retrieval

    View full-size slide

  3. Facebook AI Research Progress

    View full-size slide

  4. Growing AI Model Complexity

    View full-size slide

  5. Training Time (hours)
    Number of computer devices
    Simple to Scale in Cloud

    View full-size slide

  6. Not so simple
    to scale on Edge
    Limited space
    Limited power
    Limited connectivity

    View full-size slide

  7. Move the Intelligence not the Data

    View full-size slide

  8. Neural
    Processing
    Unit

    View full-size slide

  9. What is an
    NPU?

    View full-size slide

  10. Silicon
    Transistors
    Vacuum
    Tubes
    M
    icro-
    processors
    16-Bit
    32-Bit
    D
    ual-cores
    4-16
    Cores
    1950 1960 1970 1980 1990 2000 2010
    CPU
    GPU
    NPU
    Video
    signal
    output
    Perfectly parallel
    operations
    M
    atrix and
    vector
    operations
    Floating point
    operations
    Tensor cores
    H
    igh-volum
    e
    Low
    -precision
    Chronology of Processing Units

    View full-size slide

  11. CPU
    GPU
    NPU
    Increase Volume

    View full-size slide

  12. Lower Precision

    View full-size slide

  13. Limit Intermediate Data Fetching
    Tesla

    View full-size slide

  14. Apple Neural Engine
    8 Cores
    5 TFLOPS

    View full-size slide

  15. Huawei / HiSilicon Kirin 980

    View full-size slide

  16. Samsung Exynos

    View full-size slide

  17. Intel Mobileye

    View full-size slide

  18. Qualcomm Cloud AI 100

    View full-size slide

  19. Google Edge TPU

    View full-size slide

  20. Google Edge TPU Dev Board
    NVIDIA Jetson Nano

    View full-size slide

  21. Tesla FSD Hardware

    View full-size slide

  22. NPU
    Performance

    View full-size slide

  23. Real-world Performance Relative to iPhone X NPU

    View full-size slide

  24. NPU-powered
    Solutions

    View full-size slide

  25. iPhone Machine Learning Timeline
    Performance
    2015 2016 2017 2018
    A9
    iPhone 6S
    PowerVR GPU
    +”Hey Siri”
    + People in Photos
    A10 Fusion
    iPhone 7
    Apple GPU
    6 core
    50% faster
    + Portrait Mode
    A11 Bionic
    iPhone 8 + X
    Apple Neural Engine
    2 core
    0.6 teraflops
    + Face ID
    + Raise to wake
    A12 Bionic
    iPhone XS
    Apple Neural Engine
    8 core
    5 teraflops
    + Computational Photography
    + Developer Access to ANE

    View full-size slide

  26. Try-on a Bike
    Try-on Fashion

    View full-size slide

  27. Context-Aware Smart Home

    View full-size slide

  28. Healthcare Retail
    Photography
    Sports + Fitness

    View full-size slide

  29. The Future
    NPU

    View full-size slide

  30. More
    Efficient

    View full-size slide

  31. Mythic Low Power NPU

    View full-size slide

  32. Xnor Solar Camera

    View full-size slide

  33. More
    Powerful

    View full-size slide

  34. Graphcore IPU

    View full-size slide

  35. Quadric Edge Supercomputer

    View full-size slide

  36. Quadric Edge Supercomputer

    View full-size slide

  37. NPU
    Implications

    View full-size slide

  38. NPU
    Moore’s Law May Yet Continue

    View full-size slide

  39. Parkinson's Law
    Project workloads expand to fill the
    time allotted
    Abdinoor’s Law
    Software workloads expand to fill the
    computing resources available

    View full-size slide

  40. The NPU Revolution?
    1. AI solutions
    2. Here today
    3. More designs on the way
    4. Transformational

    View full-size slide

  41. Thank You
    Dan Abdinoor
    [email protected]
    Graphcore Visualization of Alexnet

    View full-size slide