Pavilion Data - IT Press Tour #40 Sep./Oct. 2021

Transcript

1. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. IT Press Tour OCTOBER 6, 2021
2. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. Achieving Success and Acceleration Dario Zamarian Chief Executive Officer
3. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. The Next Stage Excitement About Pavilion: • Current technology is unmatched • Ability to accelerate compute, enabling customers to get to answers faster • Secured wins in Life Science, Media & Entertainment, Financial Services, and other high-performance computing applications • We have an ambitious vision and we continue to execute on our roadmap 3
4. Accelerating Modern Data Driven Applications Costa Hasapopoulos Chief Field Technology Officer and WW VP of Business Development
5. Product and Solution Update Our Focus Areas Why is our technology unique and different? • Our Software… • Our enabling hardware…. The industry is taking notice Summary Our Focus Areas 5
6. Our Industry & Solution Focus LIFE SCIENCES HIGH PERFORMANCE DATABASES FINANCIAL MEDIA & ENTERTAINMENT HIGH PERFORMANCE VIRTUALIZATION FEDERAL 6
7. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. The MARKET is driving a new set of HyperParallel data centric applications … Applications Infrastructure For traditional SCALE UP apps and/or SCALE OUT applications Consistent, Predictable, Scalable High Performance plus Low Latency Performance of DAS + all the benefits of SHARED Start small grow big… Multiple Controllers High Bandwidth IP or InfiniBand Ports Modern Protocols – NVMe & NVMe-oF Multi Protocol – Block, File, Object Scale-out HyperParallel Scale-out HyperParallel Servers Scale-out HyperParallel SCALE OUT? Shouldn’t Storage also be HyperParallel?
8. Product and Solution Update Our Focus Areas Why is our technology unique and different? • Our Software… • Our enabling hardware…. The industry is taking notice Summary Why is our technology unique and different? • Our Software… 8
9. 9 The Foundation & Journey – Pavilion HyperOSTM Current System 2018 - HyperOS 1.x • High Performance Block Storage • Ethernet • NVMe-TCP and NVMe-RoCE 2019 - HyperOS 2.x • + Standalone File and Object • + NVMe-RDMA - Infiniband • + iSCSI, NFS, S3 2021 - HyperOS 3.x • + Containerization • + Internal Dynamic / Configurable Functions • + Global Namespace for File & Object • + Multi-chassis scale-out • + NFS RDMA, PHFS Client, Plug-ins • + Tiering HyperOS 3.x HW as Value SW as Enabler HW as Enabler SW as Value
10. The Most Flexible High Performance Storage Platform Block Solutions Block … Object … Fast Object Solutions … Block … File External File Systems via Block High Performance File Solutions File Protocol Layer File Protocol Layer Object Protocol Layer kd b High Speed Sensor / Device Ingest Not just Performance Density But Flexible Performance Density All at market competitive pricing
11. Pavilion HyperOS version 3.x – Enhancement to Block, File and Object  Enhanced QoS support • Volume Level • Cap based QoS • 4 Tiers including “ludicrous mode”  2.2PB Raw  High Availability Enhancements • N+X Controller Availability • Data Assurance Enhancements  Windows NVMe-oF Driver  Snapshot/Clone  Encryption  Thin Provisioning  Volume Expansion Block File  Multi-controller / multi-chassis distributed file system  Single NODE Performance (scales linearly per chassis) • 78 GB/s Read • 56 GB/s Write  Single or Multiple Namespaces  Flexible Client Connectivity • NFS V3/V4 • NFS RDMA • SMB • Gluster Native Client • Hadoop & Spark plug-ins  Tiering  Replication  Snapshot & Clones  Security  Encryption Object  Multi-controller / multi-chassis distributed file system  Single NODE Performance (scales linearly per chassis) • 52 GB/s Read • 27 GB/s Write  Cloud Native Object Storage  Single or Multiple Namespaces  File Access Available  Worm  Tiering  Replication  Snapshots & Clones  Application Integrations / plug-ins  Security  Encryption
12. Flexibility of Deployment with Best -in-Class Performance Density Bring YOUR OWN file system Use OUR file system Use OUR block Use OUR object NFSv3 NFSv4 NFS RDMA PHFS Client Plug-in Plug-in SMB & SMB Direct File S3 Plug-ins Object NVMe-TCP NVMe-RoCE NVMe-RDMA iSCSI Block
13. Version 3.0 - High Performance File Systems (NFS and S3) NFS NFS S3 S3 NFS NFS NFS S3 NFS S3 S3 … … NFS … S3 NFS Single Pavilion System Multiple Pavilion Systems Cluster
14. Block, File & Object – The Ultimate Flexibility in Deployment Single Pavilion System B B B B NFS B B S3 NVMe-RoCE iSCSI Multi Pavilion System Cluster … NFS NVMe-RoCE NFS S3 Bucket S3 Bucket NVMe-oF to External File System
15. Multi -Client OPTIONS and Interoperability – Ultimate Flexibility … Pavilion HyperParallel File System Single Mount Point //PavilionPHFS NOTE: All “client side” software natively bundled into most all distributions of LINUX NFSv3 NFSv3 NFSv3 NFSv4 NFSv4 NFSv4 PHFS Client PHFS Client PHFS Client NFS RDMA NFS RDMA NFS RDMA Spark Client Spark Client Spark Client Hadoop Client Hadoop Client Hadoop Client NFSv3 NFSv4 pNFS Gluster Client Hadoop Client Client Side Protocols: • NFSv3 • NFSv4 • NFS RDMA • PHFS Client • Spark Client • Hadoop Client 1. All clients can be the same protocol 2. Potential to intermix client protocols The most FLEXIBLE HyperParallel File System in the world!
16. 16 A Scalable Architecture – Manageability as well as Scalability Data Plane Control Plane
17. A Peek into the Software – Its power and simplicity… DASHBOARD
18. System Health
19. Pavilion Platform – an API First approach 19
20. 20 Management & Other Ecosystem Integrations NVMe-TCP Driver NVMe-RoCE Driver VMWare plug-in NVMe-RoCE NFS iSCSI
21. Product and Solution Update Our Focus Areas Why is our technology unique and different? • Our Software… • Our enabling hardware…. The industry is taking notice Summary Why is our technology unique and different? • Our enabling hardware… 21
22. 22 Why Are We Different? SC = Storage Controller NVMe = NVMe Drive Ultra Low Latency, High BW Network SC SC Distributed Shared Persistent Memory Ethernet/IB SC SC SC NVM e NVM e NVM e NVM e NVM e HyperParallel Platform A Network Centric Design: N x N Connectivity
23. How Pavilion is Different PCIe Switch based architecture (6.1Tb/s) 20 Independent Storage Controllers (CPU, Memory & Network) Unique Architecture: • Built like a network switch – PCIe Backplane • Built from the ground up to support NVMe & NVMe-oF • Multi/Many Controllers & Network Ports • Ethernet or InfiniBand 4RU Chassis 40 x 100 GbE/EDR or 10 x 200 HDR InfiniBand ports 72 x 2.5” U.2 NVMe Drives Unrivaled Performance Density • Any controller to Any Drive Connectivity • Cache-less and Tier-less • DMA from any drive and any controller (nanoseconds) • RDMA from hosts to Pavilion (microseconds) • Software patents that take “unfair advantage of our hardware” • Ability to TIER outside of our platform to spinning disk
24. = Block File Object or 1 2 3 4 5 11 12 13 14 15 6 7 8 9 10 16 17 18 19 20 S1 S2 Internal Cross Controller ETHERNET network NVMe-TCP NVMe-RoCE NVMe-RDMA iSCSI NFSv3 NFSv4 Gluster NFS RDMA S3 PavilionOS — Built from the Ground up To Support NVMe & NVMe-oF Multiple Independent Controllers enable linear scalability
25. What Would Someone Do with All That Performance? A flexible (scale in place)platform that can grow as your business grows Block Block File File Object Object Independently scale performance and capacity Linearly scalable – Consistent Predictable High Performance with Low Latency – Block, File and/or Object Performance Capacity START with 2 line cards and 1 RAID group
26. Unique High Availability Features Controller Failover Flexibility Data Assurance T10 DIF Metadata Versioning Pavilion Patent Pending Protects against drive firmware “dropped writes” Active/Passive Controllers Each controller has active workload Spare Controller “Controller RAID” Spare warm controller with NO workload
27. Pavilion HyperOSTM 3.0 Performance Expectations 27 File Object Block Single Storage Platform Unit = 4 Rack Units up to 2.2PB Flexibility to support multiple STORAGE types on a single platform File and Object Shared Global Namespace can be clustered with multiple systems to linearly scale performance and capacity 20M Read IOPS 5M Write IOPS 100 µs Read Latency 25 µs Write Latency 120 GB/s Read 90 GB/s Write 78 GB/s Read 56 GB/s Write 52 GB/s Read 28 GB/s Write Shared Global Namespace
28. Single Mount Point //PavilionPHFS FILE - Per Chassis: Read BW = 78 GB/s Write BW = 56 GB/s ( 72% of Read) Linear Scale Up and Scale Out 1 2 3 4 19.5 READ 14 WRITE 2 500TB 19.5 READ 14 WRITE 3 500TB 19.5+ READ 14 WRITE 4 500TB 19.5 GB/s READ 14 GB/s WRITE .5PB 39 GB/s READ 28 GB/s WRITE 1PB 78 Read 56 Write 1 2PB 878 GB/s READ 856 GB/s WRITE 2PB 156 GB/s READ 112 GB/s WRITE 4PB 234 GB/s READ 168 GB/s WRITE 6PB 312 GB/s READ 224 GB/s WRITE 8PB 390 GB/s READ 280 GB/s WRITE 10PB 546 GB/s READ 392 GB/s WRITE 14PB 468 GB/s READ 336 GB/s WRITE 12PB 78 Read 56 Write 2 2PB 19.5 READ 14 WRITE 1 500TB 78 Read 56 Write 3 2PB 78 Read 56 Write 4 2PB 78 Read 56 Write 5 2PB 78 Read 56 Write 6 2PB 78 Read 56 Write 7 2PB 78 Read 56 Write 8 2PB 624 GB/s READ 448 GB/s WRITE 16PB Single Mount Point //PavilionPHFS UNLMITED scalability = Single Mount Point //PavilionPHFS Scale Up Scale Out (example 8 chassis – scale is unlimited) 58.5 GB/s READ 42 GB/s WRITE 1.5PB 78 GB/s READ 56 GB/s WRITE 2PB Best in Every Class Performance & Performance Density 624 GB/s READ 448 GB/s WRITE 16PB Usable 32 Rack Units OBJECT - Per Chassis: Read BW = 52 GB/s Write BW = 28 GB/s ( 60% of Read) Object Performance 416 GB/s READ 224 GB/s WRITE 16PB Usable 32 Rack Units
29. The real power of NVMe-oF for VMware - RDMA NVMe-RoCE 29
30. Pavilion Enables VMware Environment to Deliver Consistent Predictable Scalable High Performance with Low Latency HyperParallel Data PlatformTM NVIDIA/Mellanox Ethernet 20 x 100GbE 4 Rack Units Containers Virtual Machines Read 120 GB/s 20M IOPS @100 µs Write 90 GB/s 5M IOPS @ 25 µs VM VM VM VM VM VM VM VM VMware ESXi VMware ESXi APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS
31. Pavilion provides a MATERIAL Performance Boost for VMWare! 143% 206% 47% 109% 155% 61% 0% 50% 100% 150% 200% 250% Read BW (1MB) GBps Random Read (4k) IOPS Random Read Latency (4k) uSec Write BW (1MB) GBps Random Write (4k) IOPS Random Write Latency (4k) uSec NVMe-RoCE to iSCSI Same HARDWARE! 2x the READ IOPS at 47% the Latency 1.5x the WRITE IOPS at 61% the Latency Compared to the SAME ESXi hardware (servers and storage) 2x the Virtual Machines! 50% the COSTS Run MORE virtual machines at ½ the latency Do ~2x the I/O’s at ½ the latency on the SAME number of VM’s 143% the BW for AI/ML/DL/Analytics workloads 31
32. NVMe-TCP NVMe-RoCE SMB Direct 32 Unleashing the power of WINDOWS NVMe-TCP or NVMe-RoCE Windows Certified Drivers SMB SMB & SMB Direct File Sharing BLOCK FILE M&E Applications M&E Industry Life Sciences Industry Super High Bandwidth per Host with Ultra Low Latency – Large and Small Block sizes!! For BLOCK or FILE
33. SQL Server Acceleration LOW latency HIGH IOPs SQL Server Solution SQL Server 2 SQL 2 SQL 2 SQL Server 3 SQL 3 SQL 3 SQL Server 4 SQL 4 SQL 4 Driven by need to SCALE SQL Server – consistently and predictably • Enabled by Windows NVMe-oF (low latency – high bandwidth) • Legacy DAS (performance reasons) • Move to SAN for flexibility and consistency • Enabled by predictable consistent performance – ULTRA LOW LATENCY Multiple Customer Segments & Use Cases Enabled by Flexibility 1. 5 Controllers + 1 zone 2. 2nd zone 3. 5 Controllers + 3rd zone 4. 4th zone Start small grow big… SQL Server 1 SQL 1 SQL 1 Pavilion storage underneath my critical SQL Servers is the first SAN to completely move the primary bottleneck to database performance away and above the storage device itself. Now, my bottleneck to performance is where it should be - database design and application code! David Klee Heraflux Technologies
34. SQL Server Acceleration – Partnered with VMware LOW latency HIGH IOPs SQL Server Solution VMWare 2 VMWar e 2 VMWare 3 VMWar e 3 VMWare 4 VMWar e 4 Similar or Same Performance as Bare-Metal SQL Server!! • Enabled by Windows NVMe-RoCE (low latency – high bandwidth) • All the benefits of VMWare with all the same functionality and performance of Bare Metal • Enabled by predictable consistent performance – ULTRA LOW LATENCY Start small grow big… VMWare 1 VMWar e 1 VMware ESXi NVMe-RoCE SQL Server on VMware comprises the bulk of the servers that we see in the wild. The unmatched performance that Pavilion offers allows me to reduce my SQL Server memory footprint, pack more SQL Servers on the host while improving performance, and squeezing more out of my expensive SQL server licensing. Everybody wins! David Klee Heraflux Technologies
35. What can you do with all that performance in FILE ? 117% Read 169% Write 73% Latency 67% Rack Units 110% Read 366% Write 9% Latency 40% Rack Units Note: Pavilion also supports NVIDIA GPUDirect Storage for BLOCK (others do NOT) GPUDirect Storage Life Sciences – Cryo-EM High Speed Ingest High Speed Analysis High Speed Processing Enables… Improved • Data workflow concurrency • Microscope utilization • Cycle time between tests • Processing time Pavilion’s performance makes “shared storage possible” Competitor 2 Competitor 1
36. What can you do with all that performance in BLOCK? 16 Competitor arrays High Performance Low Latency VMWare ecosystem Large European Research Organization 1.5 Pavilion Pavilion customer for over 2 years with multiple purchases and multiple systems now in production. From VDI, to SQL/Server, to GreenPlum, to custom applications Now adding VMWare Tanzu and CloudFoundry NVMe-RoCE & Pavilion with VMWare provides: 2x the VM Density at 50% the Latency = 50% the Cost >10X faster >6X smaller >20X less costly X
37. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. The challenge – How to compare storage vendors? What tool to generate a LOAD (FIO to GDSIO) Do we pick just READ or WRITE? Should we focus on SMALL or LARGE (block size)? Should vendors get away with just publishing “one or two numbers” What about “how many rack units it takes to deliver the #’s? Are we comparing APPLE to APPLES (block to block, file to file, Object to Object)? 350GBps – 15M IOPS 2 FULL racks… (88 RU) But BLOCK only Storage Vendor Y 300GBps – 48 RU Storage Vendor X 120GB/s Read, 90 GB/s Write 20M Read IOPS, 9M Write IOPS 100 µs to 25 µs 4 RU (rack UNITs) – less than 7” Block / File / Object
38. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. An answer…. And…. More than ONE or TWO numbers!! Performance Density Capacity Density https://pavilion.io/blog/normalizing-data-storage-performance-reporting-across-vendors-for-customer-clarity/
39. Configuration Flexibility providing - Start SMART – Grow As Needed Capacity Optimized Performance and/or Capacity Optimized Performance & Capacity Optimized Option 6 Max Performance Large Capacity Option 3 Performance Large Capacity Option 5 Max Performance Medium Capacity Option 4 Max Performance Medium Capacity Option 2 Performance Medium Capacity Option 1 Performance Small Capacity 20 Controllers 40x100Gb ports 231TB to 2.2PB Raw 8 Controllers 16x100Gb ports 231TB to 2.2PB Raw 10 Controllers 20x100Gb Ports 116TB to 1.1PB Raw 10 Controllers 20x100Gb Ports 58TB to 553TB Raw 4 Controllers 8x100Gb ports 116TB to 1.1PB Raw 2 Controllers 4x100Gb ports 58TB to 553TB Raw BLOCK 120 GB/s Read / 90 GB/s Write 20M Read IOPS / 5M Write IOPS 100uSec to 25uSec Latency 48 GB/s Read / 36 GB/s Write 8M Read IOPS / 2M Write IOPS 100uSec to 25uSec Latency 60 GB/s Read/45 GB/s Write 10M Read IOPS / 2.5M Write IOPS 100uSec to 25uSec Latency 30 GB/s Read / 22.5 GB/s Write 5M Read IOPS / 1.25M Write IOPS 100uSec to 25uSec Latency 24 GB/s Read /18 GB/s Write 4M Read IOPS / 1M Write IOPS 100uSec to 25uSec Latency 12 GB/s Read / 9 GB/s Write 2M Read IOPS / 0.5M Write IOPS 100uSec to 25uSec Latency FILE 78 GB/s Read 56 GB/s Write 31.2 GB/s Read 22.4 GB/s Write 39 GB/s Read 28 GB/s Write 19.5 GB/s Read 14 GB/s Write 15.6 GB/s Read 11.2 GB/s Write 7.8 GB/s Read 5.6 GB/s Write OBJECT 52 GB/s Read 28 GB/s Write 20.8 GB/s Read 11.2GB/s Write 26 GB/s Read 14 GB/s Write 13 GB/s Read 7 GB/s Write 10.4 GB/s Read 5.6 GB/s Write 5.2 GB/s Read 2.8 GB/s Write
40. Product and Solution Update Our Focus Areas Why is our technology unique and different? • Our Software… • Our enabling hardware…. The industry is taking notice Summary The industry is taking notice 40
41. Gartner Quote “Next-generation, data-centric workloads like artificial intelligence (AI), modeling and simulation require bandwidth, latency and input/output operations per second (IOPS)performance beyond the capabilities of NVMe technology-enabled, dual- controller scale-up of all-flashstorage arraysat scale.” Roger Cox Competitive Landscape: Innovative All-Flash Array Offerings Architected for the Data-Centric Era Published 13 October 2020 - ID G00733623 Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. 41
42. Michael Kagan – CTO and SC20 “High performance computing requires high performance IO” “The rule is simple, the higher processing power the computer element the more data it can process hence faster data delivery is required” “Storage access is yet another bottleneck that needs to be resolved…” Pavilion – “Performance improvement for file storage of almost 4x by using GPUDirect Storage”
43. Pavilion + DGX-A100 –A Powerful Combination – FILE or Block 43 70 GiB/s (75 GB/s) 178 GiB/s (191 GB/s) 110 GiB/s (118 GB/s) 69GiB/s (74 GB/s) 9.0ms 1.75ms 4.4ms 5.6ms 26% 16% 28% 11% BLOCK 69 GiB/s (74 GB/s) 170 GiB/s (182 GB/s) 139 GiB/s (149 GB/s) 70GiB/s (75 GB/s) 8.96ms 3.87ms 9.0ms 4.51ms 14% 4.4% 51% 4% Supporting NVIDIA GPUDirect Storage FILEor Block Dense Form Factor – 8 RU (4RU per Chassis) Ultra LOW Latency Impressive WRITE performance Notes: • All IO tested using GDSIO and loads directly to the GPU (GPU Direct) or to the GPU’s via the CPU (NON GPU Direct) – this is NOT an FIO test • % in YELLOW = CPU Utilization • ms = Latency • All numbers were tested with ONE Pavilion chassis and 4 GPU’s and extrapolated to TWO Pavilion chassis (8RU) and 8 GPUS solution is expected to LINEARLY scale) • Two chassis were optimized for READ throughput to saturate the DGX-A100. We could add more chassis to saturate the WRITE throughput (not a focus on these tests) NO host agents required FILE 2.5x greater BW 2.5x lower Latency 1.8x lower CPU util 2.5x greater BW 2.3x lower Latency 11.5x lower CPU util GPU Direct NON GPU Direct GPU Direct NON GPU Direct
44. Pavilion’s Performance Advantage to the Competition in GPUDirect Storage Read Bandwidth Write Bandwidth Rack Units Support for Block 40% to 67% 110% to 117% 366% to 169% Read Latency Write Latency 9% to 73% 47% to Not Published No and No
45. Product and Solution Update Our Focus Areas Why is our technology unique and different? • Our Software… • Our enabling hardware…. The industry is taking notice Summary Summary 45
46. Where we stand out from the pack…. All who have the Need for Speed! 46 Flexibility Performance Density Scalability VMware 7.x – NVMe-RoCE Storage for Parallel File Systems like SpectrumScale, Lustre, BeeGFS Microsoft SQL Server High Performance Object Solutions High Performance File System (NFS etc) Back to Shared Storage (from DAS)… AI / ML / DL Analytics (plus NVIDIA GPUDirect Storage ) High Speed Ingest High Performance Virtualized Environments
47. Fast FILE, Fast OBJECT & Fast BLOCK – Many Uses Cases 47 Applications run faster, perform more consistently and scale larger To provide solutions for… And deliver cost effective outcomes to the business… High speed sensor ingest High frequency trading Machine Learning Artificial Intelligence Data Analytics Virtual Infrastructure Log analysis Product Development / Engineering File sharing Machine Learning Enable Applications like…
48. Go To Market Success Chris McBride – Chief Revenue Officer
49. FEDERAL LIFE SCIENCES FINANCIAL MEDIA & ENTERTAINMENT ANALYTICS Significant New Footprints Across Segments and Use Cases High Data Ingest Artificial Intelligence Real-Time Decision Support on Large Data Sets 49
50. Acceleration of Federal Footprint Two of Five Eyes Four members of the Intelligence Community United States Geological Survey 50
51. Where we are winning Analytics workloads • Financial Services • Cybersecurity • Facial Recognition • Signals Processing • Extreme Science Discovery 51
52. Where we are winning Media & Entertainment • Volumetric Capture • Post-Production 52
53. Where we are winning Life Sciences • Cryo-EM • Genomics • Radiology 53
54. Where we are winning Virtualized Infrastructure 54
55. Pavilion Data Systems. ©2021 Proprietary & Confidential. All rights reserved. Questions