Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DDN "At Scale" - IT Press Tour December 2019

DDN "At Scale" - IT Press Tour December 2019

The IT Press Tour

December 13, 2019
Tweet

More Decks by The IT Press Tour

Other Decks in Technology

Transcript

  1. DDN | ©2019 DDN Confidential IT Press Tour Update 2019:

    A Year of Further Change and Expansion December 9, 2019
  2. DDN Confidential DDN | ©2019 Agenda •Introductions •DDN Reintroduction [At-Scale

    and Enterprise] •DDN at Scale •DDN Enterprise •Q&A
  3. DDN Confidential DDN | ©2019 E N T E R

    P R I S E A T S C A L E ► DDN now 1000 people worldwide! • Organic growth, R&D Investment • Acquisition ► Strategic Directions into AI, Multicloud, Exascale, Data Management ► Continued Commitment to AI and HPC marketplace • Strongest Portfolio for Data at Scale ► New Opportunities in Enterprise ► Customers are demanding more storage solutions from fewer suppliers ► Many start ups in the market and everyone is the fastest, but immaturity is risky in storage at scale ► No companies like DDN engage as a dedicated partner for the toughest data challenges.
  4. DDN Confidential DDN | ©2019 E N T E R

    P R I S E A T S C A L E Why DDN, Why Now? Emergence of AI Applications At-scale – Intersection of Enterprise and HPC AIOps Driven by Intelligent Infrastructure
  5. DDN Confidential DDN | ©2019 E N T E R

    P R I S E A T S C A L E YOUR STRATEGIC PARTNER FOR MODERN DATA WORKLOADS Virtualized Enterprise Software Defined Unified Storage High Performance Unified Storage E N T E R P R I S E AI Data Management HPC & Big Data Storage A T S C A L E
  6. DDN Confidential DDN | ©2019 E N T E R

    P R I S E A T S C A L E VMs Containers Databases & Analytics Objects GPU CPU VMWare, HyperV, NAS, VDI, VMWare, Openstack, SAS GRID, ANALYTICS, HPC, AI AI, Big Data & High-Performance Computing Our Expanded Ecosystem for End to End AI
  7. DDN Confidential DDN | ©2019 E N T E R

    P R I S E A T S C A L E At Scale Data Services for AI, Big Data and HPC Optimized AI Platforms For Every Use Case DDN BY High Performance Flash & Hybrid Unified Storage SW Defined Storage for Telco 5G, IoT and the Enterprise Simplicity and Control for Virtualized Environments Unlimited Performance for your most Demanding Workloads
  8. DDN Confidential DDN | ©2019 Realizing the Potential of the

    Data Centric Enterprise ► As data volumes, complexity, diversity and value increase, our customers look for new ways to simplify operations and extract that value ► Our new combined IP will deliver on the promise of intelligent multicloud analytics that: • Simplifies, Predict and Automates • Puts the Right Data in the Right Place at the Right Time • Exposes deeper Value from All Data ► Beyond Storage: Unified analytics resolves broader business challenges across the entire ecosystem – On premises and multicloud
  9. DDN Confidential DDN | ©2019 ► #1 for worldwide HPC

    Storage market share ► Well over 10 EXAbytes of shipped capacity ► 150+ patents in data management ► “Top 5 AI vendors to watch” for choice and end to end ► Only company with > 16% of revenues invested back into AI and HPC R&D ► Fastest and Safest choice for Storage at Scale A T S C A L E >150 PATENTS #1 HPC Storage Vendor DDN >10 EXABYTES DELIVERED >7000 CUSTOMERS Intersect360 Research, 2019
  10. DDN Confidential DDN | ©2019 ► DDN’s Research Group engages

    with our partners and customers to build cutting edge solutions ► Tackling I/O optimization across HPC and Commercial ► Production-grade tools for I/O optimization (e.g. kubernetes, kafka) for autonomous ► Accelerating processing of Seismic data with Extreme- Scale Parallel I/O for Seismic Workflows ► Better efficiency and productivity of numerical weather and climate simulation A T S C A L E Advanced storage Monitoring Energy Oriented Center of Excellence I/O Optimization Energy Research Weather Modelling I/O Analytics AI and Big Data in HPC Research
  11. AI-HPC Traditional HPC Large Scale Numerical Simulation Scalable Data Analytics

    Deep Learning Beyond Deep Learning, Beyond HPC Consumer Retail • High resolution • Realtime analysis • Simple deployment • Personalization & Intelligent Consumer Interaction Financial Services • Fraud detection • Risk mitigation • Massive historical chart analysis • Revenue enhancement and client experience improvement Life Sciences and Healthcare • Enhance clinical care • Drive operational efficiency • Speed up drug discovery • Improve patient outcomes Autonomous Vehicles and Robotics • Global data transfer and management • Connected vehicle and robot safety • Predictive maintenance forecasting • Demand & capacity planning
  12. DDN Confidential DDN | ©2019 Preprocess Supercomputing Multi-Physics Workflows AMR

    Checkpoint Classify Manage Train Tune Infer User & Service Management Data Management Multi-Tenanted Security More Value from All your Data In One Place ► Your data in the right place at the right time ► Multicloud Ready ► The right user and service management to aid your environment’s efficiencies ► Comprehensive Security for Containers, Cloud and On Prem Transparent Tiering NFS SMB HDFS S3
  13. DDN Confidential DDN | ©2019 Preprocess Supercomputing Multi-Physics Workflows AMR

    Checkpoint Classify Manage Train Tune Infer Security GPU Network Protocol AI Optimizations HPC Optimizations Security GPU Network Protocol AI Optimizations Concurrency I/O Pattern Network Protocol HPC Optimizations Optimized across the board for both AI and HPC workloads and environments ► Deep Optimizations for both AI and HPC delivers for the highest efficiencies and the right capabilities ► AI at Scale: Container Aware, Direct GPU Integration, MultiRail Networking, Accelerated Protocols for AI Frameworks ► HPC: Any thread count, the toughest IO patterns, Dynamic Data placement, Leanest Datapath
  14. DDN Confidential DDN | ©2019 Preprocess Supercomputing Multi-Physics Workflows AMR

    Checkpoint Classify Manage Train Tune Infer Security GPU Network Protocol AI Optimizations HPC Optimizations User & Service Management Data Management Multi-Tenanted Security Security GPU Network Protocol AI Optimizations Concurrency I/O Pattern Network Protocol HPC Optimizations The Intelligent, Optimized Environment for AI and HPC ► Deep Optimizations for both AI and HPC delivers for the highest efficiencies and the right capabilities ► Your data in the right place at the right time Transparent Tiering NFS CIFS HDFS S3
  15. DDN Confidential DDN | ©2019 IME IME IME Client Real

    time monitoring Comprehensive At Scale Data Management IME and EXA5 ► Ground-Up developed EXAScaler NVMe platform that delivers the best throughput and IOPs per NVMe ► Tight Integration with the most scalable HDD Platform with EXA5 data management ► Fully Automated Advanced Data Tiering designed for AI and HPC Super Fast Distributed Rebuild NVMe Optimized Data Layout and Flash Management Comprehensive Systems Management Dial In Erasure Coding Dynamic Data Placement Optimizes continuously EXAScale Efficiency that copes with real world at scale challenges
  16. DDN Confidential DDN | ©2019 EXA5 Stratagem Data Management Architecture

    Stratagem Services State of jobs kept here with some metadata around stratagem state and history Parallel Actions 2019 2020+ Stratagem Scanning Engine Parallel scanning Stratagem State Stratagem Policy Engine Hot Pools Auto Purge Search Multicloud & Enterprise Data Management EXA Scale Change logs REST and UI MGMT …
  17. DDN Confidential DDN | ©2019 Preprocess Supercomputing Multi-Physics Workflows AMR

    Checkpoint Classify Manage Train Tune Infer Security GPU Network Protocol AI Optimizations HPC Optimizations User & Service Management Data Management Multi-Tenanted Security Security GPU Network Protocol AI Optimizations Concurrency I/O Pattern Network Protocol HPC Optimizations The Intelligent Multicloud always optimizing your data transfers to and from the cloud Multicloud cache Embargoed until March 23, 2020
  18. DDN Confidential DDN | ©2019 Optimized AI Platforms For Every

    Use Case Accelerate applications by achieving full GPU saturation on DGX Streamline concurrent and continuous deep learning workflows Flexible configuration with best technology and economics Seamless scaling to match evolving workflow needs Optimized for DGX platforms and NGC containers for DL and HPC Easy to deploy and manage with turnkey support from DDN and partners ®
  19. DDN Confidential DDN | ©2019 DDN A3I SOLUTIONS – NVIDIA

    SUPERPOD REFERENCE ARCHITECTURE ► AI400 + DGX-2 SuperPOD at-scale testing and validation published by NVIDIA: • The AI400 All-flash appliance delivers incredible sequential and random read performance, as required by the heaviest DL workloads. • Metadata performance scales well from 1 to 96 nodes, with no degradation as the number of nodes and threads increases. • The AI400 is a fully-integrated platform that’s easy to deploy. DDN provides excellent technical deployment and support services. ► RA document available from NVIDIA website NVIDIA DGX-2 SUPERPOD REFERENCE ARCHITECTURE
  20. DDN Confidential DDN | ©2019 ACCELERATE YOUR SUPERPOD WITH ACHIEVE

    FULL GPU PERFORMANCE GROW SEAMLESSLY TO EXASCALE BOOST WORKLOADS WITH GPUDIRECT ENABLE END-TO-END AI WORKFLOWS RELY ON THE AT-SCALE EXPERTS
  21. DDN Confidential DDN | ©2019 5 DGX-2 10 AI400 400

    GB/s 0 GB/s 50 GB/s 100 GB/s 150 GB/s 200 GB/s 250 GB/s 300 GB/s 350 GB/s 400 GB/s 1 DGX-2 2 DGX-2 3 DGX-2 4 DGX-2 5 DGX-2 GPU READ THROUGHPUT SCALING WITH TEN AI400s ® DDN DELIVERS LINEAR PERFORMANCE SCALING EXA5 GPU integration delivers up to 80 GB/s of throughput per DGX-2 Performance scales linearly and provides maximum at-scale application acceleration
  22. DDN Confidential DDN | ©2019 DDN A3I SOLUTIONS - GPUDIRECT

    TO STORAGE ► Direct Memory Access from EXAScaler to GPU memory ► 2x-8x higher BW data transfers between Storage and GPU. ► 3.8x lower latency with no faulting and bounce buffers ► Stable and consistent latency as concurrency increases ► Lower consumption of host CPU or memory subsystem ► The GPU is the computing element with the highest IO bandwidth, e.g. 215 GB/s vs. the CPU’s 50 GB/s. ► Very fast access to petabytes of remote storage faster than even the page cache in CPU memory. PCIe SW CPU RAM GPU Network Bounce Buffer PCIe GPUDirect PCIe SW CPU RAM GPU Network GPUDirect Today
  23. ddn.com ©2019 DDN STORAGE *Other names and brands may be

    claimed as the property of others. Any statements or representations around future events are subject to change. Thank You! Keep in touch with us. 9351 Deering Avenue Chatsworth, CA 91311 1.800.837.2298 1.818.700.4000 company/datadirect-networks @ddn_limitless [email protected]
  24. DDN Confidential DDN | ©2019 SWITCH TO EXA5 MIGRATE ALL

    YOUR DATA RAPIDLY, EASILY, AND WITH CONFIDENCE • Data migration for massive data sets • Move data, maintain permissions and ACLs • Migrate your data safely • Automated migration • Track migration through an intuitive GUI
  25. DDN Confidential DDN | ©2019 Advanced Security and Data Safety

    for a Distributed at Scale World Group A Group C Group B AUDIT ► Rogue clients, VMs and containers cannot gain root on filesystem ► Secure Shared Key or Kerberized Auth. of Clients/Containers ► Multi-Tenancy exports only specific datasets to users ► Secure Audit Logging ► Data at Rest Encryption for Drives and NVMe* Encryption *NVMe SED available Q1 2020 End to end Data Protection
  26. DDN Confidential DDN | ©2019 Advancing the Containerization of AI

    and HPC ► Containers bring security challenges adding higher chance of malicious root access from clients ► Containers multiply the challenge of managing concurrency and load balancing across systems ► Capability class GPU systems such as DGX require fine grained managed of IO path across multiple networks. *NVMe SED available Q1 2020 Compromised containers get restricted privileges Parallel Management of Mounts across Network Path Simple Deployment. Completely Native Container load Balancing Multirail gets higher single system performance