PremDay #3 - The Road to 250,000 Drives

CERN presents how its storage servers evolved over the years.

Transcript

2. HL-LHC: The Road to 250,000 Drives Prem'Day 2026 Hervé Rousseau HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 2
3. Outline Introduction Storage platforms evolution Upcoming challenges Comments and questions HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 3
4. Agenda Introduction Storage platforms evolution Upcoming challenges Comments and questions HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 4
5. CERN • Established in 1954, based on the France-Switzerland border • Annual budget ≈ 1 200 MCHF 1 • More than 13 000 people work at CERN, around 3 500 employees • Acts as an “infrastructure provider” for high-energy physics research 1CERN Quick Facts 2023 HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 5
6. CERN: infrastructure
7. CERN: Computing infrastructure Disk storage • 1.4 EB (90 % HDD, 10% SSD) • 100k HDDs, 10k SSDs • Wide range of capacities Tape Storage • 115000 slots • 1.6 EB • 330 drivesa aOne tape drive per 10 PB / 500 cartridges HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 7
8. Agenda Introduction Storage platforms evolution Upcoming challenges Comments and questions HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 8
9. Legacy setup Server • 2U4N platform • x86 Dual-CPU • Up to 512 GB of RAM • 100 Gbps uplink Storage enclosure • 24 bays • 48 Gbps • Up to 432 TB per JBODa adepending on target service
10. Legacy setup • Each server is connected to 2 enclosuresa • 3.456 PB per rack • 2.8 kW power draw (out of 5.5 kW available) • 0.81 W/TB adepending on target service HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 10
11. Current setup Server • 2U4N platform • x86 Dual-CPU • Up to 384 GB of RAM • 100 Gbps uplink Storage enclosure • 60 bays • 2x48 Gbps • Up to 1440 TB per JBODa aWith 24 TB drives
12. Current setup • Each server is connected to 4 enclosuresa • 11.520 PB per rack • 5.1 kW power draw (out of 5.5 kW available) • 0.44 W/TB adepending on target service HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 12
13. Current setup: software HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 13
14. Current setup: software
15. Agenda Introduction Storage platforms evolution Upcoming challenges Comments and questions HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 15
16. Usable space increase, does not consider data redundancy HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 16
17. Usable space increase, does not consider data redundancy HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 17
18. Ongoing work Hardware • Single-socket platforms (ARM ?) • 400+ Gbps uplink • 100+ drives enclosures • SMR drives • Additional flash (caching) Operations • Rewrite storage device monitoring/alerting • Implement “lazy” repair workflow • Add-in cards inventory/monitoring
19. Expected outcome Legacy Current Expected 0 2 4 6 8 10 12 2.47 4.55 9.1 Storage (EB) Storage Capacity Legacy Current Expected 0 0.2 0.4 0.6 0.8 1 1.2 0.81 0.44 0.22 Power Efficiency (W/TB) Power Efficiency
20. Conclusion • Optimize for TB/W and USD/TB • Investigate OCP Datacenter SAS-SATA Device Specification • Streamline operations and pave way for 250k+ HDDs HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 20
21. Agenda Introduction Storage platforms evolution Upcoming challenges Comments and questions HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 21
22. Discussion Thank you for your attention! HL-LHC: The Road to 250,000 Drives Wednesday 6th May, 2026 22