SambaNova - IT Press Tour 42 March 2022

Transcript

1. IT Press Tour AI is Here March 14, 2022 1
2. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Agenda Timing Topic Presenter Take Away 11:00 - 11:05 Introductions 11:05 - 11:15 Welcome Rodrigo Liang A Well Positioned Company 11:15 - 11:30 Company Narrative Amy Love Market Dynamics 11:30 - 11:45 SambaNova Overview Marshall Choy Thoughtful Product Leadership 11:45 - 12:00 Q&A 12:00 - 12:10 Lunch/Break 12:10 - 12:25 Platform Overview Arvind Sujeeth Purpose Built for AI 12:25 - 12:40 Use Cases PD Singh Use Cases to meet market demand 12:40 - 12:55 Customer Implementations Marshall Choy Wide Range of Customers 12:55 - 1:00 Q&A Close
3. Welcome Rodrigo Liang March 14, 2022 3
4. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. SambaNova Systems 4 Ushering in the AI enabled era today. Our customers move faster. They deploy the power of AI and Deep Learning to discover new opportunities, unlock new revenue, and boost efficiency. Industry leading AI-as-a-Service Pretrained Models Next Generation Architecture Chris Ré Professor CS Stanford University Kunle Olukotun Professor EE/CS Stanford University Rodrigo Liang CEO AI is here. SambaNova was founded to solve challenges in applying AI technology to drive impact in unprecedented ways. We are democratizing AI, by providing purpose built Deep Learning solutions, delivered as a service, to enable the future of AI today.
5. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. SambaNova Snapshot 5 GLOBAL PRESENCE HEADQUARTERS IN PALO ALTO, CA AI/ML SOFTWARE-DEFINED HARDWARE 2017 ESTABLISHED THE COMPANY AI PROCESSOR (RDU) OPTIMIZED FOR DATAFLOW 500+ EMPLOYEES CLOUD-READY HIGH-VALUE MISSION-CRITICAL ENTERPRISE WORKLOADS SERIES D FUNDED $1.1B RAISED $5B VALUATION INTEGRATED AI PLATFORM OFFERED AS A SUBSCRIPTION
6. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. AI is a Generational Shift 6 AI TODAY MOBILE 2008 WEB 1998 SCALE-OUT COMPUTING EMBEDDED DEVICES DATAFLOW COMPUTING DEVELOPER DRIVEN CONSUMER DRIVEN DATA DRIVEN
7. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. 7 Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Purpose-built for AI from the Chip to the Platform
8. Market Leadership Amy Love 8
9. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Bold. Old. The difference is AI 9 Copyright © 2022 SambaNova Systems, Inc. All rights reserved.
10. 10 If you’re not winning with AI, you’re being beat by AI
11. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. OF CEOS BELIEVE AI WILL HAVE A LARGER IMPACT THAN THE INTERNET 63% ADDED TO THE GLOBAL ECONOMY IN THE NEXT DECADE BY AI TRILLION $15 18 BY 2025, EVERY CONNECTED PERSON WILL HAVE A DIGITIAL INTERACTION EVERY… SECONDS IDC, The Digitization of the World PwC’s Annual Global CEO Survey, McKinsey Report: The Executive’s AI Playbook PwC’s Annual Global CEO Survey, McKinsey Report: The Executive’s AI Playbook Artificial intelligence in Business by McKinsey & Co
12. 12 2. Pace of Technology Innovation 3. Rate of Adoption and Deployment at Scale 1. Massive Industry Shift
13. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Comissioned Study: Key Learnings 13 plan to allocate more than $100 million of IT budget toward strategic technology goals. 70% Main reason why your organization is investing in AI.ML? 40% 25% 22% Powers innovation Operational efficiency Keep up with competitors 75% say improving access to deep learning is very important for fostering competition and innovation in their industry. What are your organization’s biggest challenges in scaling your AI/ML efforts? 50% Restrictive computing architectures 35% Lack of trained talent 28% Difficulty customizing models
14. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. ACCELERAITION is here. AI is here. 28% of enterprises are deploying AI at Scale 72% are being left behind. What are you going to do next? Reach out to us, the leader in AI deep learning to jump start your journey today.
15. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. TRANSFORMAITION is here. AI is here. The state of the art AI is increasing at 10x per year Is your organization prepared? Reach out to us, the leader in AI deep learning to jump start your journey today.
16. SambaNova Overview Marshall Choy March 14, 2022 16
17. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. AI Transformers 1: Deloitte State of AI in the enterprise, Fourth Edition AI ADOPTION IS ACCELERATING AI TECHNOLOGY IS ACCELERATING 28 OF ENTERPRISES HAVE HIGH AI DEPLOYMENT AND OUTCOMES1 2: Based on datapoint that compute demands of top models doubled every 3.4 months between 2012-2018 from AI and Computer research paper from Georgetown Center for Security and Emerging Technology, January 2022 3: Based on the increase of the number of parameters of large language models, which have increased at a pace of roughly 10X per year % 10X ANNUAL GROWTH IN MODEL SIZE2 AND COMPUTE REQUIREMENTS3
18. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The Dawn of the Deep Learning Era 18 • “Industrialization of Computer Vision” • Highly Flexible, General-Purpose Models • Surpassing Human Level Performance at Some Tasks Dawn of the Deep Learning Era AI 2.0 Digital Transformation & Machine Learning AI 1.0
19. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. 19 CHALLENGE Organizations looking to adopt deep learning are struggling with technical challenges and competing for limited expertise. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. THE DEEP LEARNING Outdated GPU/CPU-based AI infrastructure 1 Competition for limited top AI Talent 2 Deep learning advancements are outpacing the ability to deploy them 3
20. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The Deep Learning Deployment Gap AI transformation: Time to hire a team, build infrastructure, train and deploy model Increase in compute requirements 2 32X Increase The Deep Learning Deployment Gap Increase in model size3 15X Increase 1: “Most AI transformations take 18-36 months to complete” from “Building the AI-Powered Organization”, Harvard Business Review, 2019 2: Based on datapoint that compute demands of top models doubled every 3.4 months between 2012-2018 from AI and Computer research paper from Georgetown Center for Security and Emerging Technology, January 2022 3: Based on the increase of the number of parameters of large language models, which have increased at a pace of roughly 10X per year 18 months¹ Increase 10X every 12 months Increase 2X every 3.4 months Compounding Compounding
21. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The SambaNova Difference: AI is here. 21 Advanced Deep Learning Capabilities • We solve the hard problems • We solve the large problems • We solve the scale problems Faster Time to Value • Deploy in weeks, not years • Pre-trained models reduce the need for heavy data-science effort Reduced Complexity • Simplified and ease of use Purpose Built for Innovation • Ability to handle the pace of AI innovations: 10X increase in compute requirements and model size every year
22. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The Industry’s Most Powerful Deep Learning Platform Dataflow-as- a-ServiceTM High Resolution Image Processing Complex 3D Image Analysis Real Time Video Analysis Sentiment Analysis Document Classification Named Entity Recognition COMPUTER VISION AS-A-SERVICE NATURAL LANGUAGE PROCESSING AS-A-SERVICE Public Sector BFSI Manufacturing Energy Healthcare Customer Intent Product Recommendation Personalization RECOMMENDATION SYSTEMS AS-A-SERVICE Pre-trained models | Domain Specific Fine Tuning Integrated Hardware/Software AI Platform Deep Learning Models-as-a-Service, Deployed in Weeks Industries Powered by Reconfigurable Dataflow Unit (RDU)
23. Questions 23
24. Platform Overview Arvind Sujeeth 24
25. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. “In the weeks since its arrival, GPT-3 has spawned dozens of other experiments that raise the eyebrows in much the same way. It generates tweets, pens poetry, summarizes emails, answers trivia questions, translates languages and even writes its own computer programs, all with very little prompting.” State-of-the-Art AI is the Province of the Elite Few NYTimes, 11/24/2020 ML & systems PhDs Integration & deployment Massive data center resources
26. 26 To unlock the world’s hardest problems, first we need a new approach
27. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The RDU and SambaFlow Are the New Engine for AI 27 The old way: Kernel-by-kernel Bottlenecked by memory bandwidth and host overhead Memory Data & Weights Result Memory Data & Weights Result Memory Data & Weights Result Memory Data & Weights Result Memory Batch & Weights Memory Result Conv 1 Cache Cores Conv 2 Cache Cores Pool Cache Cores Norm Cache Cores Sum Cache Cores Content Switching Content Switching Content Switching Content Switching
28. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The RDU and SambaFlow Are the New Engine for AI 28 Pool Conv 1 Conv 2 Pool Sum Norm Norm Pool Conv Conv Norm Sum Weights PMU Conv Weights PMU PMU PCU PMU PCU PCU PMU PCU PMU PCU PCU PMU PMU PCU PCU PMU PMU PCU PCU PMU PMU PCU PCU PMU PMU PCU PCU PMU PCU PMU PMU PCU PMU PMU PCU PCU PMU PCU PMU PCU PMU PMU PCU Sample 1 PCU PMU PMU PCU PCU PMU PMU PCU PMU PCU PCU PMU PCU PMU PCU PCU Sample 2 Sample 3 Sample 4 Sample 5 Sample 6 The Dataflow way: Spatial programming Eliminates memory traffic and overhead, increases parallelism
29. 29 And we need to meet programmers where they are to make it seamless
30. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The RDU and SambaFlow Are the New Engine for AI 30 The Journey of a PyTorch App With SambaFlow Ordinary PyTorch ML developers are familiar with
31. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The RDU and SambaFlow Are the New Engine for AI 31 Weights Weights Sample Conv 1 Pool Conv 2 Norm Sum The Journey of a PyTorch App With SambaFlow SambaFlow Spatial Compilation Whole program optimization reconfigures the RDU to be optimal for this model…instantly
32. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The RDU and SambaFlow Are the New Engine for AI The compiler handles partitioning so no changes are required to go from 1 socket to 1000 The Journey of a PyTorch App With SambaFlow
33. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Vertically-integrated, full-stack state-of-the-art AI deployable instantly 33 GPT3 Conv3d Rescale ... Dataflow-as-a- Service On-prem Hosted Cloud BFSI Solution PubSec/ Govt solution Healthcare solution Oil and Gas Solution Manufacturing Solution No/low code access to latest models on scalable infrastructure Vertical Solutions use common Software platform components Platform includes enterprise AI functions like Mlops, manageability, Observability, BI, labelling mgmt. Base Models, compiler and compute functions Hardware
34. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The AI world is constrained in the Habitable Zone Fine grained (Too Hot) Large Models (Too Cold) Performance Habitable Zone GPT-3 Large embeddings True-Res CV BERT-Large ResNet RNNs Sparsity AI4S
35. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. SambaNova has expanded the Habitable Zone Fine grained Large Models GPU-optimized GPT-3 Large embeddings True-Res CV BERT-Large ResNet RNNs Sparsity AI4S 1x GPU Performance
36. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Example: True-Res Computer Vision 36 TILES ARE STREAMED THROUGH MODEL PIPELINE ON CHIP Copyright © 2021 SambaNova Systems, Inc. All rights reserved. conv 3x3, ReLU conv 1x1 copy and crop max pool 2x2 up-conv 2x2 SAMBAFLOW AUTOMATICALLY TILES THE INPUT IMAGE FOR DEEP LEARNING OPERATIONS AND HANDLES OVERLAPS BETWEEN TILES Model is exposed through a simple API that works at any resolution
37. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Results on the TCGA Cancer Detection Workload 37 Copyright © 2021 SambaNova Systems, Inc. All rights reserved. Only supported by RescaleNet50 5760X5760 AS INPUT - ORIGINAL RESOLUTION 89.4% RescaleNet50 512X512 AS INPUT - ORIGINAL RESOLUTION 83.5% RescaleNet50 512X512 AS INPUT - LOWER RESOLUTION 82% ResNet50 224X224 AS INPUT - ORIGINAL RESOLUTION 77%
38. Use Cases PD Singh 38
39. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. The SambaNova Solutions: Enterprise grade AI 39 Banking, Financial Services and Insurance • McKinsey estimates that the potential annual value unlocked by Advanced AI is $683.7 Billion. • Disruption from fintech, DeFi and crypto. • Large scale NLP problems for customer service, fraud prevention, anti-money laundering, document processing. Healthcare • McKinsey estimates that the potential annual value unlocked by Advanced AI is $306.9 Billion. • Disruption from COVID, cheap sequencing, pervasive data sharing. • CV and NLP for cancer screening, drug discovery, Point of care diagnoses. Manufacturing • McKinsey estimates potential annual value unlocked by Advanced AI is $491B Billion • Disruption from great resignation, supply chain disruptions, remote work and robotics. • CV for defect detection, time series / recommendation systems for supply chain, predictive maintenance, and Agritech. Energy and Sustainability • BCG claims applying AI to corporate sustainability will add $1.3T -$2.6T of value. • Disruption from Increasing Energy demand, climate change, net zero pledge. • RL for efficient datacenters, fusion reactors. 3Dconv for efficient oil extraction.
40. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Enterprise grade AI Use cases in Banking, Financial Services and Insurance • Customer fulfilment and retention - Chatbots for faster MTTR in customer service, customer churn prediction and prevention. • NLP for insurance cost reduction - Can take into account data points from vast amounts of data that humans simply can't. • AI based credit scoring - Increases speed, accuracy of credit risk models and gives quick access to credit for BNPL. • Algorithmic trading - Reduce compliance risks, analyze more signals, aid in alpha generation strategies, • Roboadvisers - $8T AUM in wealth management is slowly being moved to roboadvisers. • AI based fraud detection - 5% of global GDP transactions are money laundering and $80-100B lost in fraud transactions. KYC scenarios • AI based Regtech - Respond to regulatory requirement quickly and effectively. To become an AI-first institution, a bank must streamline its capability stack for value creation Reimagined engagement AI-powered decision making Core technology and data Operating model
41. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Enterprise grade AI Use cases in Healthcare • Disease detection – Data analysis using NLP for diagnosis, high-res imaging analysis for disease diagnosis in radiology, pathology image analytics. • Drug Discovery – ML analysis of patient data, clinical trials, publications for new drug applications. Protein folding prediction. • Targeting and detection of evolving pathogens – detect and reduce antibiotic resistance • Real time patient monitoring • Healthcare supply chain management • Dynamic pricing of health insurance • Supplementing Underequipped point of care in developing/underdeveloped/disaster stuck places
42. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Enterprise grade AI Use cases in Manufacturing • Defect detection – High-res imaging to detect defect on assembly and QA lines. • Predictive maintenance – time series models to predict machine failures in advance. • Yield optimization/enhancement - RL used for robotic control, extruder operations and chemical process optimization. • Autonomous machine control – various control inputs can be optimized in a closed loop system for reduced energy, better quality, sustained reactions. • Generative design – Use generative models to design better than traditional topology optimization. • Inventory management – predictive models for inventory replenishment Share of respondents
43. Customer Implementations Marshall Choy 43
44. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Driving Research that Solves Great Challenges Copyright © 2022 SambaNova Systems, Inc. All rights reserved.
45. Copyright © 2022 SambaNova Systems, Inc. All rights reserved. Banking Innovators are moving fast, and they’re not on Wall St. 45 Copyright © 2022 SambaNova Systems, Inc. All rights reserved. • OTP - one of the largest independent financial service providers in Central and Eastern Europe ◦ Over 40,000 employees ◦ 17 million customers • Deployed and in production in under 3 months • Changing the banking market ◦ Delivering a customer centric experience • Sees SambaNova as a partner, not a vendor • Building the fastest SuperComputer for AI in Europe Copyright © 2022 SambaNova Systems, Inc. All rights reserved.
46. Questions 46
47. Thank You 47