ADDC 2019 - Dan Abdinoor: The NPU Revolution

Behind the use of artificial intelligence capabilities is a new and foundational piece of technology: the Neural Processing Unit. These AI-only processors are changing the rules for machine learning power and affordability, creating new and ideal conditions for intelligent devices. In this talk, we will explore the history, recent breakthroughs, and future impact, covering everything you need to know about the Age of the NPU.

Recordings & more: https://addconf.com/

Transcript

1. The NPU
   Revolution
   Dan Abdinoor
   CEO & Co-founder, Fritz
   

   View Slide

2. View Slide

3. Newly possible with
   intelligence
   ● Vision
   ● Voice
   ● Autonomy
   Two Types of AI Solutions
   Improved with
   intelligence
   ● Translation
   ● User Input
   ● Storage & Retrieval
   

   View Slide

4. View Slide

5. Facebook AI Research Progress
   

   View Slide

6. Growing AI Model Complexity
   

   View Slide

7. Training Time (hours)
   Number of computer devices
   Simple to Scale in Cloud
   

   View Slide

8. Not so simple
   to scale on Edge
   Limited space
   Limited power
   Limited connectivity
   

   View Slide

9. Move the Intelligence not the Data
   

   View Slide

10. Neural
    Processing
    Unit
    

    View Slide

11. What is an
    NPU?
    

    View Slide

12. Silicon
    Transistors
    Vacuum
    Tubes
    M
    icro-
    processors
    16-Bit
    32-Bit
    D
    ual-cores
    4-16
    Cores
    1950 1960 1970 1980 1990 2000 2010
    CPU
    GPU
    NPU
    Video
    signal
    output
    Perfectly parallel
    operations
    M
    atrix and
    vector
    operations
    Floating point
    operations
    Tensor cores
    H
    igh-volum
    e
    Low
    -precision
    Chronology of Processing Units
    

    View Slide

13. CPU
    GPU
    NPU
    Increase Volume
    

    View Slide

14. Lower Precision
    

    View Slide

15. Limit Intermediate Data Fetching
    Tesla
    

    View Slide

16. NPU Chips
    

    View Slide

17. Apple Neural Engine
    8 Cores
    5 TFLOPS
    

    View Slide

18. Huawei / HiSilicon Kirin 980
    

    View Slide

19. Samsung Exynos
    

    View Slide

20. View Slide

21. Intel Mobileye
    

    View Slide

22. Qualcomm Cloud AI 100
    

    View Slide

23. Google Edge TPU
    

    View Slide

24. Google Edge TPU Dev Board
    NVIDIA Jetson Nano
    

    View Slide

25. Tesla FSD Hardware
    

    View Slide

26. NPU
    Performance
    

    View Slide

27. Real-world Performance Relative to iPhone X NPU
    

    View Slide

28. View Slide

29. NPU-powered
    Solutions
    

    View Slide

30. iPhone Machine Learning Timeline
    Performance
    2015 2016 2017 2018
    A9
    iPhone 6S
    PowerVR GPU
    +”Hey Siri”
    + People in Photos
    A10 Fusion
    iPhone 7
    Apple GPU
    6 core
    50% faster
    + Portrait Mode
    A11 Bionic
    iPhone 8 + X
    Apple Neural Engine
    2 core
    0.6 teraflops
    + Face ID
    + Raise to wake
    A12 Bionic
    iPhone XS
    Apple Neural Engine
    8 core
    5 teraflops
    + Computational Photography
    + Developer Access to ANE
    

    View Slide

31. Tesla
    

    View Slide

32. Try-on a Bike
    Try-on Fashion
    

    View Slide

33. Context-Aware Smart Home
    

    View Slide

34. Healthcare Retail
    Photography
    Sports + Fitness
    

    View Slide

35. The Future
    NPU
    

    View Slide

36. More
    Efficient
    

    View Slide

37. Mythic Low Power NPU
    

    View Slide

38. Hailo 8
    

    View Slide

39. Xnor Solar Camera
    

    View Slide

40. More
    Powerful
    

    View Slide

41. Graphcore IPU
    

    View Slide

42. Quadric Edge Supercomputer
    

    View Slide

43. Quadric Edge Supercomputer
    

    View Slide

44. NPU
    Implications
    

    View Slide

45. NPU
    Moore’s Law May Yet Continue
    

    View Slide

46. Parkinson's Law
    Project workloads expand to fill the
    time allotted
    Abdinoor’s Law
    Software workloads expand to fill the
    computing resources available
    

    View Slide

47. The NPU Revolution?
    1. AI solutions
    2. Here today
    3. More designs on the way
    4. Transformational
    

    View Slide

48. Thank You
    Dan Abdinoor
    dan@fritz.ai
    Graphcore Visualization of Alexnet
    

    View Slide