2018 Ceph Day: "NVMe Over Fabrics and Composable Infrastructure"

Presentation at Ceph Day 2018: "NVMe Over Fabrics and Composable Infrastructure - What Do They Mean for Software-Defined Storage?"

Transcript

1. NVMe over Fabrics and Composable Infrastructure – What do they mean for SDS? Tom Lyon DriveScale, Inc.
2. Who Is Tom Lyon? §  Co-Founder and Chief Scientist, DriveScale §  Distinguished Engineer, Cisco §  Co-Founder and Chief Scientist, Nuova Systems [-> Cisco UCS] §  CTO, Nokia Internet Communications §  Founder & CTO, Ipsilon Networks [-> Nokia IP Routing & Security] §  Distinguished Engineer, Sun Microsystems [employee #8] §  Systems Architect, Amdahl Corp. §  BSE in EECS, Princeton Univ. §  Hacking on UNIX since 1975 §  UNIX, UTS, SunOS, NFS, SPARC, SunLink, IP Switching, ATM, UCS, VFIO … 2 ©2018 DriveScale Inc. All Rights Reserved.
3. Why Software-Defined Storage? §  SANs originally provided separation of compute & storage resources –  This is a very good thing – consolidation, simpler servers, … §  But SANs now are $$$$$! and don’t scale §  Centralized storage creates network bottlenecks §  SAN hardware evolved to just being servers anyways §  But: –  there is no right size for an SDS server/storage combo –  Every cluster needs different cpu/storage ratios –  CPUs and drives have different “natural” life-cycles 3 ©2017 DriveScale Inc. All Rights Reserved.
4. NVMe enabled SSDs §  Crazy fast SSDs: –  500K IOPS –  3GB/s §  But: –  Bottleneck shifts to software –  Hard to buy small & fast SSDs –  2TB and rising for good performance §  NVMe itself is just an interface protocol enabling deep & parallel queues 4 ©2017 DriveScale Inc. All Rights Reserved.
5. NVMe Over Fabrics §  Substitute network for PCIe – keep NVMe basics §  Fibre Channel or RDMA networks –  Ethernet RoCE is leader; iWarp has lost out; IB still niche –  NVMe over TCP coming because RDMA is “very difficult” –  Typically for 25Gb Ethernet and beyond §  NVMe-o-F aspects: –  Data Plane – highly standardized –  Control Plane - some standardization –  Manageability aspects – don’t hold your breath 5 ©2017 DriveScale Inc. All Rights Reserved.
6. NVMe-o-F Use Cases §  1. Just another protocol between servers & SAN –  Esp NVMe on FC §  2. Protocol embedded in backend network of storage system –  NVMe devices not visible to general network §  3. Protocol for direct dis-aggregation of drives from servers –  Target devices are real network citizens –  Discovery, security, monitoring, … are very important 6 ©2017 DriveScale Inc. All Rights Reserved.
7. NVMe-o-F Targets §  Of course, x86 servers can be NVMe targets §  But lots of new SOC chips for more efficient hardware: –  Broadcom Stingray –  Mellanox Bluefield –  Kazan – pure hardware data path §  Redundant controllers needed to avoid new SPOFs §  So software needs to live on the server, not on the target §  Servers become stateless, homogeneous, re-deployable 7 ©2017 DriveScale Inc. All Rights Reserved.
8. Server Disaggregation §  If SSDs outside the server make sense, why not –  Hard Drives –  GPUs? –  DRAM? §  Disaggregation requires re-Composition 8 ©2017 DriveScale Inc. All Rights Reserved.
9. Composable Infrastructure: Definition 9 ©2018 DriveScale Inc. All Rights Reserved.
10. Composable Infrastructure: Large System Vendors §  Cisco started it, but dropped the ball §  HPE is promoting it with Synergy blade system §  DellEMC entering market –  PowerEdge MX composable system –  “Kinetic Infrastructure” §  WDC – OpenFlex Composable Infrastructure 10 ©2018 DriveScale Inc. All Rights Reserved.
11. Composable Infrastructure: Startups §  DriveScale: “Software Composable Infrastructure for Your Data Center” –  HDD, SSD, NVMe - Ethernet §  Liqid: “On-Demand Composable Infrastructure” –  NVMe & GPU – External PCIe §  H3Platform: NVMe and GPU disaggregation –  NVMe & GPU – External PCIe §  Other NVMe over Fabrics: –  Apeiron, Attala, Excelero, E8 Storage, Pavilion Data, … 11 ©2018 DriveScale Inc. All Rights Reserved.
12. ©2018 DriveScale Inc. All Rights Reserved. What is DriveScale? 12 DAS Composable DriveScale was born to radically transform how modern applications are deployed in the data center. Seeing how hyperscale companies were reaping the benefits of analytics and AI with massive data sets on their scale-out infrastructure, DriveScale set out to make it viable for any enterprise to deploy this infrastructure on- premises and reap the same reward. DriveScale designed a software platform that creates a way to run these same data-intensive applications unchanged with cloud scale and efficiency at a significantly lower cost.
13. DriveScale Composability Platform 13 Bare Metal DriveScale Composability Platform PCIe Ethernet iSCSI RoCE v2 NVME/TCP Rack Servers HDD SSD NVMe Blade Servers Modular Servers GenZ NoSQL