VMworld 2011: Running the Hands On Labs (HOL)

Presentation from OpenStorage Summit 2011 (OSS2011)

Transcript

1. VMworld  2011: Running  the  Hands  On  Labs  (HOL) and How  We  Got  There Andy  BenneC,  Craig  Morgan Nexenta  Systems
2. VMware  Trial  Cloud  -­‐  SuperRack – Background • Approached  February  2011   • Co-­‐design  for  a  commodity  white  box  soluPon • To  Be  Comparable  to  Vpod  and  Vblock – Architecture  Team • Pulled  together  with  VMware  IntegraPon  Engineering   Team  (IE) • Responsible  for  bleeding  edge  deployments  and  PoCs • Brains  behind  the  HoL  as  a  testbed  for  early   deployments  of  VMware  and  partner  technologies
3. VMware  Trial  Cloud  -­‐  SuperRack
4. VMware  Trial  Cloud  -­‐  SuperRack • Requirements – run  High  Density  Generic  Cloud  Workloads – Cost  per  TB  primary  design  consideraPon  but  good   IOP’s  performance  important – commodity  x86  hardware  via  technology  partners – Nexenta  HA  design  and  scale  out  for  future  with   NFS  referrals – Classic  POD  Cloud  Design
5. SuperRack
6. Nexenta  Storage  Module Part Descrip+on Total  Qty System 2u  6  HDD  Chassis 4 Motherboard X8DAH+F 4 Processors X5670  2.93G 8 Memory 8GB  NETLIST 72 HDD Seagate  2TB  SAS  7.2k 360 JBODs DataOn  60x  bay 6 Controllers LSI  SAS 8 10GbE  NIC STD  Dual-­‐port  w.  SFP+ 4 SATA  DOM Innodisk  64GB 4 License 720TB  raw   1 SSD 2x  STEC  ZeusRAM  8GB 8 SSD STEC  ZeusIOPS  200GB 8
7. Nexenta  Storage  Module • Nexenta  Layout – mulPple  pools  (acPve/acPve  cluster) – 2x  data  pools • Per  Pool – vdev  layout  is  6x  RAID  Z2 – 30x  vdevs  per  pool – 2x  SLOG  devices  (STEC  ZeusRAM  mirrored) • Totals  (Rack) – 360  drives,  720TB  raw,  480TB  usable
8. Other  Storage  Service  Providers 2x  FAS  3270  Appliances 144  x  15K  Hard  Disks 4  PAM  Cards  per  Appliance 2x  EMC  VNX  7500  plaforms 3x  File  Blades  per  VNX 1  VNX  was  all  SSD  96  SSD’s  in  total 1  VNX  140+  15K  Disks  FAST  Cache,  FAST  VP  SSD  Tier  
9. Hands-­‐on-­‐Labs  Workload – Some  highlights • each  lab  provision – 26x  disPnct  labs – 4x  to  25x  VMs  per  lab – max  VM  size  was  26GB – create,  deploy,  destroy  per  login – lab  choice  random  (student  choice),  no  pre-­‐pop  possible! • nested  VMs  workload,  highly  latency  sensiPve – Takeaways • 148,103  VM’s  created  during  Vegas  show • 1x  VM  created  every  1.215  seconds
10. HoL  StaPsPcs Element Las  Vegas Copenhagen Total Lab  Time 50 38 100  (12@Sydney) Labs  Taken 13415 4797 18912  (700@Sydney) Lab  Hours 12388 ? Lab  Seats  available 487 280 767 VMs  per  Lab 4-­‐25x 4-­‐22x Unique  ACendees 5042 1045 6207  (120@Sydney) VMs  Created 148103 53358 201461
11. Nexenta  StaPsPcs • Ran  4  of  8  VMware  verPcals  in  Vegas • 10.3  billion  NFS  IOPs  served – 7.9  billion  in  Vegas,  2.4  billion  in  Copenhagen • 3  billion  NFS  IOPs  from  one  head  in  Vegas   • Peak  controller  load   – 154,000x  4K  NFS  ops/sec  at  sub  1  ms  latency – 38,590x  16K  NFS  ops/sec  on  a  single  controller
12. Nexenta  StaPsPcs  (cont.) – Highest  Bandwidth  (single  head,  16K  average  I/O) • 1,305  MB/sec  total • 928MB/sec  read • 376MB/sec  write • ...  less  than  2ms  latency  throughout  above!
13. Nexenta  OperaPonal  Issues – DRAM  failure • DRAM  failure  in  one  head  triggered  HA  failover • partner  head  ran  the  workload  of  “both”  for  6hrs  unPl   evening  maintenance  window • NS  called  out  failed  DIMM  serial  number • DIMM  replaced  head  back  in  service  12mins – High  Availability  RSF-­‐1  HA  plugin  worked  flawlessly • VMware  saw  no  loss  of  service • monitoring  informed  NOC  before  they  aCributed  issue • head  over  provisioning  in  design  meant  soluPon  didn’t   glitch  with  extra  workload
14. Nexenta  VMworld  Custom  AnalyPcs – Tech  demo  (give  us  feedback!) • Sketch  concept  to  delivered  in  1  1/2  days • Custom  wriCen  with  distance  visualisaPon  in  mind • 30o  screen  (jumbotron)  at  100o  distance • Convey  as  much  useful  info  as  densely  as  possible • High  level,  useful  staPsPcs • NFS  IOPs  (not  raw  I/O  or  bytes) • VM  &  VMDK  latency  and  OPS  (appropriate  objects) • Response  to  VMworld  NOC  team  input • Deliver • bandwidth,  flow,  read,  write,  top  10x,  server,  latency,   alignment
15. Nexenta  VMworld  Custom  AnalyPcs
16. Nexenta  VMworld  Custom  AnalyPcs VMworld  Vegas  2011  Prototype Video  Clip  available  at:  h:p://bit.ly/aura-­‐vegas
17. Nexenta  VMworld  Custom  AnalyPcs VMworld  Copenhagen  2011  Prototype Video  Clip  available  at:  h:p://bit.ly/aura-­‐copenhagen
18. Thanks! QuesPon  and  Answer Andy  BenneC  (andy@nexenta.com) Craig  Morgan  (craig@nexenta.com) Richard  Elling  (richard@nexenta.com)