ADDC 2019 - Dan Abdinoor: The NPU Revolution

Behind the use of artificial intelligence capabilities is a new and foundational piece of technology: the Neural Processing Unit. These AI-only processors are changing the rules for machine learning power and affordability, creating new and ideal conditions for intelligent devices. In this talk, we will explore the history, recent breakthroughs, and future impact, covering everything you need to know about the Age of the NPU.

Recordings & more: https://addconf.com/

Transcript

1. The NPU Revolution Dan Abdinoor CEO & Co-founder, Fritz
3. Newly possible with intelligence • Vision • Voice • Autonomy Two Types of AI Solutions Improved with intelligence • Translation • User Input • Storage & Retrieval
5. Facebook AI Research Progress
6. Growing AI Model Complexity
7. Training Time (hours) Number of computer devices Simple to Scale in Cloud
8. Not so simple to scale on Edge Limited space Limited power Limited connectivity
9. Move the Intelligence not the Data
10. Neural Processing Unit
11. What is an NPU?
12. Silicon Transistors Vacuum Tubes M icro- processors 16-Bit 32-Bit D ual-cores 4-16 Cores 1950 1960 1970 1980 1990 2000 2010 CPU GPU NPU Video signal output Perfectly parallel operations M atrix and vector operations Floating point operations Tensor cores H igh-volum e Low -precision Chronology of Processing Units
13. CPU GPU NPU Increase Volume
14. Lower Precision
15. Limit Intermediate Data Fetching Tesla
16. NPU Chips
17. Apple Neural Engine 8 Cores 5 TFLOPS
18. Huawei / HiSilicon Kirin 980
19. Samsung Exynos
21. Intel Mobileye
22. Qualcomm Cloud AI 100
23. Google Edge TPU
24. Google Edge TPU Dev Board NVIDIA Jetson Nano
25. Tesla FSD Hardware
26. NPU Performance
27. Real-world Performance Relative to iPhone X NPU
29. NPU-powered Solutions
30. iPhone Machine Learning Timeline Performance 2015 2016 2017 2018 A9 iPhone 6S PowerVR GPU +”Hey Siri” + People in Photos A10 Fusion iPhone 7 Apple GPU 6 core 50% faster + Portrait Mode A11 Bionic iPhone 8 + X Apple Neural Engine 2 core 0.6 teraflops + Face ID + Raise to wake A12 Bionic iPhone XS Apple Neural Engine 8 core 5 teraflops + Computational Photography + Developer Access to ANE
31. Tesla
32. Try-on a Bike Try-on Fashion
33. Context-Aware Smart Home
34. Healthcare Retail Photography Sports + Fitness
35. The Future NPU
36. More Efficient
37. Mythic Low Power NPU
38. Hailo 8
39. Xnor Solar Camera
40. More Powerful
41. Graphcore IPU
42. Quadric Edge Supercomputer
43. Quadric Edge Supercomputer
44. NPU Implications
45. NPU Moore’s Law May Yet Continue
46. Parkinson's Law Project workloads expand to fill the time allotted Abdinoor’s Law Software workloads expand to fill the computing resources available
47. The NPU Revolution? 1. AI solutions 2. Here today 3. More designs on the way 4. Transformational
48. Thank You Dan Abdinoor dan@fritz.ai Graphcore Visualization of Alexnet