Volumez - IT Press Tour #60 Jan. 2025

1 © 2025 Volumez, Inc. All rights reserved. © 2025
Volumez, Inc. All rights reserved. 1 Title IT Press Tour 2025, Silicon Valley January 29, 2025 Maximize GPU Yield and Automate AI/ML Data Pipelines with Volumez © 2025 Volumez, Inc. All rights reserved.

2 © 2025 Volumez, Inc. All rights reserved. 2 ©
2025 Volumez, Inc. All rights reserved. Featured & Guest Speakers Jean Banko Director Revenue Marketing & Sales Ops John Blumenthal Chief Product & Business Officer Dianne Gonzalez Sr. Director Product Marketing & BD Dr. Eli David CTO & Co-founder DeepCube / Volumez Advisor Dr. Roman Vainshtein Head of GenAI Fujitsu Research Yarden Maymon AI/ML Engineering Lead

2025 Volumez, Inc. All rights reserved. Agenda • Introductions • AI Market Overview • Volumez Solution: Data Infrastructure as a Service • MLCommons: MLPerf Storage 1.0 Benchmark • Hardware Systems Design and Volumez • Value Propositions: Performance and Automation • Dr. Eli David: AI Market View of Training Performance and Automation • Demo: Managed Datasets/Notebook Integration • Dr. Roman Vainshtein: Using Volumez at Fujitsu • Q&As • 6:30pm Dinner

2025 Volumez, Inc. All rights reserved. Artificial Intelligence: Cornerstone of Modern Business Strategy Enterprises are investing heavily in AI Global AI spending expected to grow at a staggering 25.6% CAGR, reaching $301B by 2026, and surpassing $500B by 2027 (IDC) Key drivers of AI investments • Business Transformation • Competitive Advantage • Generative AI Applications Generative AI is the fastest- growing segment Enterprises want to leverage GenAI creativity, beyond prediction and classification of predictive modeling and classification 2x GenAI growth rate versus overall AI (IDC) 78% Enterprises are increasing investments in AI (Deloitte) 5% v 80% GenAI usage in the enterprise, 2023 versus 2026 prediction (Gartner) ~70% Enterprises plan on Hybrid/Multi-Cloud for GenAI strategy (IDC)

2025 Volumez, Inc. All rights reserved. GenAI is unlocking new possibilities across industries by enabling creativity, hyper-personalization, and automation at an unprecedented scale. Hyper -Personalized marketing content, virtual stylists, synthetic customer feedback Retail Synthetic financial data for model testing, conversational AI for personalized banking Finance Patient specific treatment plans, synthesizing medical images for training models Healthcare Optimized designs for new products, creating simulations of manufacturing to improve efficiencies Manufacturing Creating synthetic voices, virtual hosts, influencers Media GenAI builds on traditional AI by not just processing data but creating new, unique outputs tailored to specific needs, driving innovation across industries. The value of Creativity vs. Prediction GenAI in Action: Enterprises Desire to Unlock New Possibilities

2025 Volumez, Inc. All rights reserved. The AI/GenAI Investment Paradox: Success Remains Elusive >80% Enterprises are Investing in AI1 10% AI projects moved to production2 60% IDC prediction on underperformance of GenAI initiatives3 1 IDC (2024, January). IDC Future Enterprise Resiliency & Spending, Wave 1. 2 Forbes (2024, January 8). Reasons Why Generative AI Pilots Fail To Move Into Production. https://www.forbes.com/sites/peterbendorsamuel/2024/01/08/reasons-why-generative-ai-pilots-fail-to-move-into-production/. 3 IDC prediction that "60% of generative AI initiatives will underperform due to challenges in integrating data, AI models, and business processes."(Computerworld, How Generative AI Will Drive a Foundational Shift in Your Company, link)

2025 Volumez, Inc. All rights reserved. The Silent Saboteur: Unbalanced Systems Undermine Data Pipelines Data scientist time/focus Fragmented, manual workflows Infrastructure resources Operational time I/O Bottlenecks Storage Inefficiencies Underutilized GPU’s = Overprovisioning Cost Escalations / Waste Workload Sizing Root Cause Analysis Skills Gaps = Poor Time to Value Drained Team Bandwidth Complex Management Versioning Issues Poor Integrated Tooling = Delayed Pipelines Increased Errors Lack of automation Debugging Infrastructure DevOps Drain = Poor Model Accuracy Stalled Innovation

2025 Volumez, Inc. All rights reserved. AWS io2 10K Volumez on AWS 10K AWS io2 30K Volumez on AWS 30K AWS io2 50K Volumez on AWS 50K Storage $837.50 $608.40 $2,137.50 $855.00 $3,437.50 $1,026.60 Compute $185.07 $92.53 $740.28 $92.53 $1,110.41 $92.53 $- $500 $1,000 $1,500 $2,000 $2,500 $3,000 $3,500 $4,000 $4,500 Volumez vs. AWS io2 Block Express • AWS io2 10K uses r7g.2XL (8 vCPUs) compute and io2 storage. • AWS io2 30K uses r7g.8XL (32 vCPUs) compute and io2 storage. • AWS io2 50K uses r7g.12XL (48 vCPUs) compute and io2 storage. • Volumez on AWS 10K, 30K & 50K uses r7g.XL (4 vCPUs) compute and 2 x i4i.2XL storage. Total costs -31% Storage – 27% Compute -50% Total costs -75% Storage -70% Compute -92% Total costs -67% Storage – 60% Compute -88% Confidential and Proprietary. Impact Analysis: Unbalanced Systems Performance

2025 Volumez, Inc. All rights reserved. AWS io2 10K Volumez on AWS 10K AWS io2 30K Volumez on AWS 30K AWS io2 50K Volumez on AWS 50K Storage $837.50 $608.40 $2,137.50 $855.00 $3,437.50 $1,026.60 Compute $185.07 $92.53 $740.28 $92.53 $1,110.41 $92.53 $- $500 $1,000 $1,500 $2,000 $2,500 $3,000 $3,500 $4,000 $4,500 Volumez vs. AWS io2 Block Express Economic benefits • Compute cost savings up to 92% • Storage cost savings up to 70% • Total cloud cost savings of 31% to 75% • Optimization of compute instances – less expensive CPUs with no hidden EBS IOPS limits • Guaranteed IOPS, latency, and bandwidth with Volumez • AWS io2 10K uses r7g.2XL (8 vCPUs) compute and io2 storage. • AWS io2 30K uses r7g.8XL (32 vCPUs) compute and io2 storage. • AWS io2 50K uses r7g.12XL (48 vCPUs) compute and io2 storage. • Volumez on AWS 10K, 30K & 50K uses r7g.XL (4 vCPUs) compute and 2 x i4i.2XL storage. Total costs -31% Storage – 27% Compute -50% Total costs -75% Storage -70% Compute -92% Total costs -67% Storage – 60% Compute -88% Confidential and Proprietary. Impact Analysis: Unbalanced Systems Performance

2025 Volumez, Inc. All rights reserved. Balance is Key to Meet AI/GenAI’s Unprecedented Demands Key Requirements Needed for Success Cloud-Aware, Precision-Tuned Data Infrastructure for AI/GenAI • AI and GenAI workloads require cloud-aware, precision-tuned systems that meticulously address IaaS constraints, aligning compute, memory, and storage resources to meet the high- performance demands of AI/GenAI. Maximizing Yield and ROI with Optimized Resources • AI/GenAI demands infrastructure and operational efficiency, where every GPU, storage, and human resource operates at full potential to deliver peak performance with zero waste, resulting in faster training times, reduced costs, and the scalability needed to unlock the maximum yield of cutting-edge models. Self-Service AI Infrastructure • Data scientists need tools that empower them to manage ML pipeline requirements without engaging in infrastructure complexity, freeing scientists to focus on innovation and experimentation while ensuring optimal infrastructure performance Extreme Performance in a Compact Footprint • AI/GenAI workloads demand concentrated compute and storage power in a dense infrastructure to achieve sustainability, cost efficiency, and energy optimization while reducing physical hardware sprawl.

16 © 2025 Volumez, Inc. All rights reserved. Volumez Data
Infrastructure as a Service (“DIaaS”) Foundation Technology, Multiple Services/Workload Specific Block Infrastructure Services NFSv3/4 Servers Database Engines Streaming Engines Batch Processors AI/ML Workloads Video Processors Data Infrastructure as a Service differentiated infrastructure scale and economics tailored to data intensive workloads cloud native cloud neutral cloud aware capacity instance (HDD) hybrid instance performance managed disk capacity managed disk performance instance (SSD) AI Infrastructure Services DIaaS DIaaS for AI Training

17 © 2025 Volumez, Inc. All rights reserved. Cloud Awareness:
the Foundation of DIaaS Balanced Systems Created From Deep Understanding of IaaS Capabilities and Constraints 17 Data Infrastructure as a Service Volumez Analysis Identify capacity Lock device read/write ratio Pressure tests to determine limits Measure real performance Identify physical location/topology (node, zone, region) Store classification in catalog Cloud Provider IaaS Volumez IaaS Catalog/ Dynamic Accounting Capabilities Constraints Compatibilities Costs State Tags Cloud Provider IaaS workload Awareness discovers, creates and maintains balanced data infrastructure for the workload performance cost

18 © 2025 Volumez, Inc. All rights reserved. DIaaS High
Level Architecture Flash Stack NVMe/TCP Instance Storage Applications Data Infrastructure as a Service Application Stack Application Stack Application Customer VPC Linux Data Paths • Intelligent “cloud aware” data path composition • Policy-driven optimizations • Secure control plane

19 © 2025 Volumez, Inc. All rights reserved. DIaaS Leverages
Data Infrastructure as Code (“DIaC”) 19 { "name":: "production-database", "performance": { "iops": 1000000, "latency_usec": 300 }, "encryption": true, "resilience": { "zones": 1 } } Declarative policy Declare 100% Linux Data Path No Storage Controller Compose Data Infrastructure as a Service

21 © 2025 Volumez, Inc. All rights reserved. Data Infrastructure
as a Service for AI/ML AI Infrastructure Capabilities with Volumez: Faster Time to Model Deployment with New Economics Data Ingestion Data Pre-Processing, Shuffle Model Training Deployment/ Inference Checkpointing Weights Data Loading Block Service (high capacity nodes) Block Service (performance nodes) Block Service (performance nodes) DIaaS for AI/ML Training (XFS, NFS) Block Service (capacity nodes) Block Service (performance nodes) RAG (Vector DBs) Batch Inference Today’s focus

22 © 2025 Volumez, Inc. All rights reserved. Purpose of
the Product Remove storage related GPU waste Reduce storage costs Bridge operational gap between data scientists and infrastructure Simplify infrastructure workflows to increase data scientist efficiency

23 © 2025 Volumez, Inc. All rights reserved. RDMA x8
x8 x8 Ethernet 80Gbps 80Gbps 80Gbps 24Gbps 24Gbps 24Gbps *Azure H100 Network *Azure A100 Network Media GPUs GPU Machines Dataset Dataset Dataset Automated Elastic Pipelines: JIT Infrastructure Data Pipelines Tailored to the Workload Data Infrastructure as a Service Object Storage Datasets Repository Volumez Automation • Terraform/IaC • APIs • PyTorch library

24 © 2025 Volumez, Inc. All rights reserved. DIaaS for
Training and Fine-Tuning: 2 Configurations Foundation Volumez Technology Simplifying and Scaling AI Data Pipelines DIaaS for AI: Hyperconverged DIaaS for AI: Flex • Simple integration with GPU server instances • Utilizes local SSDs • High performance • Target market • static, long-lasting clusters (experiments, fine-tuning) • moderate dataset capacities (1TB-100TB) • Utilizes SSDs outside the GPU cluster • Automated provisioning • Ultra high performance/scale (see MLPerf results) • Parallel training • Higher resilience • Target market • dynamic clusters • large dataset capacities >100TB) Simplicity Scale

25 © 2025 Volumez, Inc. All rights reserved. Data Infrastructure
as Code for AI AI Infrastructure from the Data Scientist’s Notebook Volumez PyTorch library

26 © 2025 Volumez, Inc. All rights reserved. Volumez Managed
Datasets Let the PhDs Be The PhDs Data Scientist • 16 A100 • 3DUnet • 100K CT Scans • s3://my-data Auto-Calculate storage/infra requirements Deploy optimal infrastucture Clone data from S3 Customer VPC Attach volume to notebook Benefits Generated Directly From Data Scientist Notebook • Removes storage related GPU waste • Increases storage efficiencies and performance, eliminating infra waste • Eliminates storage expertise requirements • Dynamically automates infrastructure provisioning • Simplifies integration into existing infrastructure and workflows model/dataset submitted PyTorch library

27 © 2025 Volumez, Inc. All rights reserved. Example Workflow
Integration with Azure ML Simplicity: Automated, Dynamic AI Data Infrastructure Lifecycle import torch import torch.nn as nn import torch.optim as optim from torch.utils.data import DataLoader import numpy as np import volumez as vlz # Define the mount point mount = "/mnt/data" # Create volumez data infrastructure dataset = vlz.Dataset(name, source=“s3://path/to/data”, mount=mount, mode=Edit) # Load the data dataloader = DataLoader(dataset, batch_size=5, shuffle=True) Locate and configure dataset access 1 Azure Blob Azure Data Fabric Snowflake Databricks . . . Auto-provision Dataset Volume 3 DIaaS for Training Run job, send telemetry to AML portal 4 Intelligent, dynamic configuration: sizing, capacity, perf Call AML scheduler to assign job 2 Complete job, logging/audit, destroy infra 5

29 © 2025 Volumez, Inc. All rights reserved. What is
MLPerf ? • MLCommons • “Aims to accelerate AI innovation to benefit everyone.” • An Industry baseline to measure AI workloads performance. • Simulate GPU workload on CPU to create storage load. • Benchmark storage metrics while maintaining at least 90% GPU utilization. • Throughput • Samples/sec • Training time • GPU Utilization > 90% • 3DUnet • Resnet50 • Cosmoflow Primary Metrics Models

30 © 2025 Volumez, Inc. All rights reserved. Volumez Configuration:
MLPerf Storage 1.0 • 137 App Nodes (c5n.18xlarge) • Each simulates x3 H100 running unet3d • 128 Media Nodes (i3en.24xlarge) – 60 TB per node • Open Submission • Custom Data loader • Change of data format • Benchmark saturation – “Barrier Removal”

2025 Volumez, Inc. All rights reserved. MLPerf Storage 1.0 AI/ML Training Benchmark 1 9.5M IOPS 1.10 TB/Sec Throughput 92% GPU Utilization 411 Simulated GPUs 1 Results verified by MLCommons Association. Retrieved from https://mlcommons.org/benchmarks/storage/.

2025 Volumez, Inc. All rights reserved. Volumez Shatters MLPerf Storage 1.0 AI/ML Training Benchmark 1 Demonstrates DIaaS for AI/ML Superiority on Cloud Infrastructure and On-Prem 32 99.7GB/Sec 93.5% GPU Utilization 1.14 TB/Sec 9.9M IOPS 92% GPU Utilization 1 Results verified by MLCommons Association. Retrieved from https://mlcommons.org/benchmarks/storage/. 273GB/Sec 91.6% GPU Utilization 34.6GB/Sec 91.5% GPU Utilization 40.9 GB/Sec 90.8% GPU Utilization 695GB/Sec 91.7% GPU Utilization 48.4GB/Sec 92.12% GPU Utilization

33 © 2025 Volumez, Inc. All rights reserved. There Is
No Performance Without Cost • Comparing vendors running in AWS • Normalized against Volumez DIaaS AI Training • Price/Throughput == How much you pay for the same performance • Price/Capacity == How much you pay for available capacity under this performance *Estimated cost based on submitted data and us-east-1 prices +27% +202% +479% +127%

35 © 2025 Volumez, Inc. All rights reserved. AI Workloads
Require New Design Center for Storage Systems Is There a Singular Workload? High Capacity • Data volumes are much larger than standard enterprise storage • Customer question: “How can I store 500 PB of file in a single namespace?” • Customer problems: • File storage systems often break down > 100 PB • Lack of DC space/power High Bandwidth • Bandwidth required for reasonable training speed are much higher than before. • Customer question: “How can I reach 1 TB/s of bandwidth from file storage to my GPU cluster?” • Customer problems: • Reaching these speeds requires overprovisioning storage by as much as 300%. High IOPS • Some AI workloads require very high speed access to very small blocks (<1 KB) • Customer question: “How can I reach >>1 MIOPs to each GPU of 512 B each?” • Customer problems: • Reaching these speeds requires RDMA These problems statements are not all from the same customer/use case

36 © 2025 Volumez, Inc. All rights reserved. AI Data
Infrastructure And Performance Density System Balance Between Performance and Capacity: Cloud and On-Prem Fall Short Requirement: 50 PB of data processed every 8 hours Storage server form factor: 2 GB/s bandwidth per server with 100 TB capacity How many servers needed to support capacity? • 50 PB/100 TB servers ~ 500 servers How many servers needed to support bandwidth? • 50 PB/8 hours ~ 174 GB/s overall bandwidth required • 174 GB/s / 2 GB/s ~87 servers required for performance Conclusion • 500 servers required, which is >500% of the capacity required Approach • To reduce cost, put 1/5 capacity in each server, down to 20TB A Balancing Act Challenge

37 © 2025 Volumez, Inc. All rights reserved. AI Data
Infrastructure Requires Performance/Density Design Center for Storage System/Volumez Integration 24 x SSDs RAID and HBAs, PCIe Switc hes, Retimers 2 x CPUs PCIe switch GPU, FPGA, NIC 16 x SSDs 1 x CPUs Up to 4 NIC … … Typical 2U x 24 SSD General Purpose Server Performance Density System: 1U x 16 SSD Storage Server Gen purpose server requires many slots for different options Large number of slots requires PCIe switches with possible bottlenecks Dual CPUs remain standard, but take higher amounts of power Connecting large number of SSDs to CPUs requires a maze of components Performance of server is limited and SSDs saturate the server with only 4-6 servers. NICs connected directly to CPU. Each slot maps to a single CCD in the CPU, allowing for pipelining of data paths. Single CPU with 128 PCIe Gen 5 lanes sufficient to map 16 SSDs directly to 4 high speed NICs. 64 PCIe lanes to SSDs, 64 lanes to NICs creates a balanced system. 16 total SSDs. Every 4 SSDs map to single CCD in the CPU. 64 Lanes 64 Lanes Form factor design in cloud and on-prem systems • High performance SSDs…but performance limits at the system level • Fixed capacity limited by form factor…more servers required for capacity but wasted performance Performance density design • Volumez enables system balance – apportioning components to meet performance/capacity requirement • Tailored configurations prevent waste, dramatic reductions in cost

38 © 2025 Volumez, Inc. All rights reserved. Use Cases
and System Performance/Density Metric A New Measure for Data Infrastructure: Bandwidth/Capacity Ratio Use Case Representative Server Capacity (TB) Bandwidth (GB/s) Bandwidth/ Capacity Ratio Archive 1000 1 0.001 SW development 1000 20 0.020 Relational DB 500 10 0.020 VDI 250 6 0.024 Containers 500 12 0.024 AI (File) 1000 50 0.050 Component Level Performance/Density HDD 24 0.300 0.013 Raw PCIe Gen 4 SSD 8 7 0.875 Raw PCIe Gen 5 SSD 8 14 1.750 HDD Variety of use cases served by SSDs SSDs have sufficient performance for AI – the question is the ability to extract that performance by the storage server and balancing software (Volumez)

39 © 2025 Volumez, Inc. All rights reserved. System Level
Conclusions for AI Data Infrastructure Volumez Key to Extracting Maximum Performance/Density •There is no singular workload •Different workload characteristics comprise the data pipeline •General purpose systems cannot deliver performance/density New data infrastructure demands •Business context for AI infrastructure matters at scale and speed •System level/component economics combined with Volumez balancing creates superior performance/density Economics are paramount •Maximizing storage capacity to minimize space and OpEx cost •Simultaneously maximizing performance minimizes cost •Maximized performance/density can only be achieved with resource balance (Volumez) Balancing act is the key

41 © 2025 Volumez, Inc. All rights reserved. Scale Produces
Yield Key Metric for Efficiency, Quality, Cost Hard manufacturing process measurement: proportion of defect-free goods produced compared to total number of products initially started or attempted Yield (%) = (effective output during training / total computational capacity ) * 100 Yield (%) = (7500 TFLOPS / 10000 TFLOPS ) * 100 = 75% Factors affecting GPU yield DIaaS for AI Training! • GPU utilization • Training efficiency – how well the AI model and data pipeline are optimized • Bottlenecks – data loading, memory access, other system components • Hardware limitation Yield (%) = (number of good units produced / total number of units started) * 100 GPU yield example

42 © 2025 Volumez, Inc. All rights reserved. Volumez Increases
Cloud Data Pipeline Yield Today GPU utilization <80% Average Training Time = 10 hours* Cloud Provider Global GPU Availability Customer/User Job Count With Volumez GPU utilization 99% Average Training Time = 5 hours* Cloud Provider Global GPU Availability Volumez Max GPU Utilization: Decreasing Training Time, Increases Job Count for Same Time on Same GPU Availability Customer/User Job Count *estimated

43 © 2025 Volumez, Inc. All rights reserved. Volumez Maximizes
Data Pipeline Yield Unique Infrastructure Balancing Tailored to Training Workload Maximizes GPU Utilization Infrastructure Balance GPU Utilization Data Pipeline Yield 100 % Volumez Balanced Infrastructure: Key to GPU ROI • Leverages AI/ML to create precise training infrastructure for model and dataset • Produces minimized infrastructure cost while maximizing GPU utilization • Increases research efficiency for higher quality models

44 © 2025 Volumez, Inc. All rights reserved. Markets Containing
Unbalanced AI/ML Infrastructure Data Type Market Segment Use Cases Examples MRI / CT scans Medical Imaging Automated scan analysis AIDoc Experiment data Pharmaceuticals Drug discovery, clinical trails monitoring, success prediction Pfizer Genome scans Genomics Protein design, genomic data analysis Illumina Video, Sensors Autonomous vehicles Object detection, navigation, decision making Mobileye, Wayve, Gatik High resolution videos Media & Entertainment Content creation, recomendation, enhancement Netflix, Zoom Video / 3D Images Gaming & VR Content creation, game design, voice synthesis Nintendo, Activision, Epic games Video Security & Defense Video surveillance, sattelite images, object tracking, drones Honeywell, Bosch Security Audio Customer Support Automated call centers, sentiment analysis AT&TT, RingCentral Time Series Algo-trading Market prediction, automated trading Citadel, IG Group Time Series Financial services Fraud detection, risk managment Large banks, credit companies Video, Audio, Sensors Robotics Movement, navigation, performing tasks, Boston Dynamics, Tesla, iRobot Video Agriculture Field analytics, climate prediction John Deere, Agco Unbalanced AI Infrastructure Prevalence • Large data samples • Medical, video, audio, time series • Multi node training • Capacity varies with GPU count • Not so much: transformer models

45 © 2025 Volumez, Inc. All rights reserved. 2 Perspectives
of Cloud GPU Yield Customer Model Complexity Training Dataset Size Time to Complete GPUs Allocated Cost 1 Medium Medium 1 week 2 High Large 3 days 3 High X-Large 2 weeks “I don’t need this model for a while” “We’re running many experiments, and I need this model this week” “We changed cameras on our cars and need a new foundation model from the telemetry” Volumez delivers 98% utilization for all jobs Customer Perspective “I get the maximum value of what I paid for” “Scale enables me to control my cost based on model/dataset requirements” Cloud Provider Perspective “Scale maximizes the value of my GPU inventory” ”I increase my profit margins by eliminating waste"

2025 Volumez, Inc. All rights reserved. Next Steps • Send a collateral package • Record podcast with Neil, John, and Dianne • Post blog on Coldago Research 2025 List of Gems • Preview your draft articles • Promote your published articles

Volumez - IT Press Tour #60 Jan. 2025

Volumez - IT Press Tour #60 Jan. 2025

More Decks by The IT Press Tour

Other Decks in Technology

Featured

Transcript