Energy Efficiency in Data Centers

Energy Efficiency in Data Centers

Presentation by Marina Zapater at GoingGreen workshop, organized by EESTEC (May 10th, 2013)

96e2d41af0b2b857d8ec7cd65d170e72?s=128

GreenLSI

May 10, 2013
Tweet

Transcript

  1. Energy Efficiency in Data Centers Marina Zapater   Marina  Zapater

     |    Going    Green   1   GreenLSI  –  Integrated  Systems  Lab   Electronic  Engineering  Dept   Green
  2. Green   Marina  Zapater  |    Going  Green   2

      Data Centers
  3. Green   Marina  Zapater  |    Going  Green   3

      Outline •  Why  Data  Centers  (DC)  in   this  Workshop?   •  The  DC  in  next-­‐genera?on   applica?ons   •  Energy  consump?on  at  the   Data  Center   •  Insight  on  op?miza?on   strategies   •  Conclusions  
  4. Green   Marina  Zapater  |    Going  Green   4

      Outline •  Why  Data  Centers  (DC)  in   this  Workshop?   •  The  DC  in  next-­‐genera?on   applica?ons   •  Energy  consump?on  at  the   Data  Center   •  Insight  on  op?miza?on   strategies   •  Conclusions  
  5. Green   Marina  Zapater  |    Going  Green   5

                         US  EPA  2007  Report  to  Congress  on  Server  and  Data  Center  Energy  Efficiency   Why DC in this Workshop? Motivation
  6. Green   Marina  Zapater  |    Going  Green   6

      Motivation •  Energy  consump?on  of  data  centers   –  1.3%  of  worldwide  energy  produc?on  in  2010   –  USA:  80  mill  MWh/year  in  2011  =  1,5  x  NYC   –  1  data  center  =  25  000  houses   •  More  than  43  Million  Tons  of  CO2  emissions  per   year  (2%  worldwide)   •  More  water  consump?on  than  many  industries   (paper,  automo?ve,  petrol,  wood,  or  plas?c)                      Jonathan  Koomey.  2011.  Growth  in  Data  center  electricity  use  2005  to  2010  
  7. Green   Marina  Zapater  |    Going  Green   7

      Motivation José  M.Moya  |    Madrid  (Spain),  July  27,  2012   7   •  It  is  expected  for  total  data   center  electricity  use  to   exceed  400  GWh/year  by   2015.   •  The  required  energy  for   cooling  will  con?nue  to  be  at   least  as  important  as  the   energy  required  for  the   computa?on.   •  Energy  op?miza?on  of  future   data  centers  will  require  a   global  and  mul?-­‐disciplinary   approach.   0   5000   10000   15000   20000   25000   30000   35000   2000   2005   2010   World  server  installed  base   (thousands)   High-­‐end  servers   Mid-­‐range  servers   Volume  servers   0   50   100   150   200   250   300   2000   2005   2010   Electricity  use     (billion  kWh/year)   Infrastructure   Communica?ons   Storage   High-­‐end  servers   Mid-­‐range  servers   Volume  servers   5,75  Million  new  servers  per  year   10%  unused  servers  (CO2  emissions   similar  to  6,5  million  cars)  
  8. Green   Marina  Zapater  |    Going  Green   8

      What about urban DC? •  50%  of  urban  DC  have  already  or  will  soon  reach  the   maximum  capacity  of  the  power  grid  
  9. Green   Marina  Zapater  |    Going  Green   9

      Tier 4 Data Center
  10. Green   Marina  Zapater  |    Going  Green   10

      Outline •  Why  Data  Centers  (DC)  in   this  Workshop?   •  The  DC  in  next-­‐generaNon   applicaNons   •  Energy  consump?on  at  the   Data  Center   •  Insight  on  op?miza?on   strategies   •  Our  vision  and  future  trends  
  11. Green   Marina  Zapater  |    Going  Green   11

      The DC in next generation applications •  Tradi?onal  uses  of  Data  Centers:   –  Webmail,  Web  search,  Databases,  Social  networking  or  distributed   storage,  High-­‐performance  compu?ng  (HPC),  Cloud  compu?ng   •  Next-­‐genera?on  applica?ons:   –  Popula?on  monitoring  applica?ons:  e-­‐Health,  Ambient  Assisted  Living   –  Smart  ci?es   •  Next-­‐genera?on  applica?ons  generate  huge  amounts  of  data   •  Need  to  store,  analize  and  generate  knowledge  
  12. Green   Marina  Zapater  |    Going  Green   12

      Global energy optimization •  Solu?on:  GoingGreen!     •  How:  Global  energy  op?miza?on  strategies   –  Proposal  of  a  holis?c  energy  op?miza?on  framework   –  Minimizing  overall  power  consump?on   –  Mul?-­‐level  op?miza?on:  WBSN,  Personal  Servers  and  Data  Centers  
  13. Green   Marina  Zapater  |    Going  Green   13

      Global energy optimization •  Execu?ng  part  of  the  workload  in  the  Personal  Servers     –  Classifying  tasks  depending  on  their  demand   –  Resource  management  techniques  based  on  fast  run?me  alloca?on   algorithms  executed  on  the  Personal  Servers   –  Execu?ng  some  tasks  in  Personal  Servers  instead  of  forwarding  load  to  DC.   –  Up  to  10%  in  energy  savings  and  15%  execu?on  ?me  savings  
  14. Green   Marina  Zapater  |    Going  Green   14

      Outline •  Why  Data  Centers  (DC)  in  this   Workshop?   •  The  DC  in  next-­‐genera?on   applica?ons   •  Energy  consumpNon  at  the   Data  Center   •  Insight  on  op?miza?on   strategies   •  Conclusions  
  15. Green   Marina  Zapater  |    Going  Green   15

      Energy Consumption at the DC What is really a Data Center? hjp://cesvima.upm.es   WORKLOAD   Scheduler   Resource     Manager   ExecuNon  
  16. Green   Marina  Zapater  |    Going  Green   16

      Energy Consumption at the DC How does cooling work? •  Typical  raised-­‐floor  air-­‐cooled  Data  Center:  
  17. Green   Marina  Zapater  |    Going  Green   17

      Energy Consumption at the DC Power consumption breakdown •  The  major  contributors  to  electricity  costs  are:   –  Cooling  (around  50%)   –  Servers  (around  30-­‐40%)   •  The  most  common  metric  to  measure  efficiency  in   Data  Centers  is  PUE  (Power  Usage  Effec?veness)  
  18. Green   Marina  Zapater  |    Going  Green   18

      Power Usage Effectiveness (PUE) •  Average  PUE  ≈  2   •  State  of  the  Art:    PUE  ≈  1,2   –  The  important  part  is  IT  energy  consump?on   –  Current  work  in  energy  efficient  data  centers  is  focused  in   decreasing  PUE   –  Decreasing  PIT     does  not  decrease  PUE,  but  it  has  in  impact   on  the  electricity  bill   !"# = 1 !"#$ = !!"!#$ !!" =! ! !!!!!!!!!!= !!"#$% + !!""#$%& + !!"# !!"#$% ≈ !!""#$%& + !!" !!" !
  19. Green   Marina  Zapater  |    Going  Green   19

      “Traditional” approaches What would Google do? PUE  =  1.2  
  20. Green   Marina  Zapater  |    Going  Green   20

      Research trends Abstrac?on  level   •  Higher  levels  of   abstrac?on  bring   more  benefits   •  Some  areas  have   brought  more   benefits  than   others   Solu?ons  proposed  by  the  State  of  the  Art  
  21. Green   Marina  Zapater  |    Going  Green   21

      Outline •  Why  Data  Centers  (DC)  in  this   Workshop?   •  The  DC  in  next-­‐genera?on   applica?ons   •  Energy  consump?on  at  the   Data  Center   •  Insight  on  opNmizaNon   strategies   •  Conclusions  
  22. Green   Marina  Zapater  |    Going  Green   22

      Our approach •  Global  strategy  to  allow  the   use  of  mul?ple  informa?on   sources  to  coordinate   decisions  in  order  to  reduce   the  total  energy  consump?on   •  Use  of  knowledge  about  the   energy  demand   characteris?cs  of  the   applicaNons,  and     characteris?cs  of  compuNng   and  cooling  resources  to   implement  proacNve   opNmizaNon  techniques  
  23. Green   Marina  Zapater  |    Going  Green   23

      Energy Optimization: Holistic Approach Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  24. Green   Marina  Zapater  |    Going  Green   24

      Resource Management at the Room level Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  25. Green   Marina  Zapater  |    Going  Green   25

      Resource Management at the Room level Leveraging heterogeneity – IT perspective •  Use  heterogeneity  to  minimize  energy  consump?on  from  a   sta?c/dynamic  point  of  view   –  StaNc:  Finding  the  best  data  center  set-­‐up,  given  a  number  of   heterogeneous  machines   –  Dynamic:  Op?miza?on  of  task  alloca?on  in  the  Resource  Manager   •  We  show  that  the  best  solu?on  implies  an  heterogeneous  data   center   –  Most  data  centers  are  heterogeneous  (several  genera?ons  of   computers)   –  5  to  22%  energy  savings  for  sta?c  solu?on   –  24%  to  47%  energy  savings  for  dynamic  solu?on   M.  Zapater,  J.M.  Moya,  J.L.  Ayala.  Leveraging  Heterogeneity  for   Energy  Minimiza?on  in  Data  Centers,  CCGrid  2012  
  26. Green   Marina  Zapater  |    Going  Green   26

      Resource Management at the Room level Leveraging heterogeneity – IT perspective •  Energy  profiling  of  tasks  of  the  SPEC  CPU  2006  benchmark   •  Usage  of  MILP  algorithms  to  schedule  tasks  in  servers  where   they  consume  less  energy   •  Implemented  in  a  real  resource  manager  (SLURM)  
  27. Green   Marina  Zapater  |    Going  Green   27

      Resource Management at the Room level IT + Cooling perspective •  Genera?ng  a  thermal  model  for   the  data  room:   –  Data  Center  environmental   monitoring  to  gather  temperature,   humidity,  differen?al  pressure   –  Predict  server  temperature  and   room  temperature   •  Op?mum  resource   management  ajending  to   cooling  and  IT  power   –  Real  environment  with   heterogeneous  servers   –  SLURM  resource  manager  
  28. Green   Marina  Zapater  |    Going  Green   28

      Resource Management at the Server level Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  29. Green   Marina  Zapater  |    Going  Green   29

      Resource Management at the Server level Leakage-temperature tradeoffs - Cooling •  Exploring  the  leakage-­‐temperature  tradeoffs  at  the  server  level   –  At  higher  temperatures,  CPU  increases  power  consump?on  due  to   leakage   –  To  decrease  CPU  temperature,  fan  speed  raises,  increasing  server   cooling  consump?on.   M.  Zapater,  J.L.  Ayala.,  J.M.  Moya,  K.  Vaidyanathan,  K.  Gross,  and  A.  K.  Coskun,  “Leakage  and   temperature  aware  server  control  for  improving  energy  efficiency  in  data  centers”,  DATE  2013  
  30. Green   Marina  Zapater  |    Going  Green   30

      Resource Management at the Server level Leakage-temperature tradeoffs - Cooling •  Implemented  fan  speed  controllers  that  reduce  server  power   consump?on  by  10%.   Fig. 4. Test 3 temperature sensor readings for the three different controllers 0.1 0.2 kWh) Energy difference between 1800RPM and 2400RPM for clustered allocation analytical model for leakage p fan speeds for varying utilization model, we implement a cooling Fig. 4. Test 3 temperature sensor readings for the three different controllers nd 2400RPM for clustered allocation analytical model for leakage power and find the optimum fan speeds for varying utilization values. Based our analytical model, we implement a cooling controller that adjusts the fan
  31. Green   Marina  Zapater  |    Going  Green   31

      Resource Management at the Chip level Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  32. Green   Marina  Zapater  |    Going  Green   32

      Scheduling and resource allocation policies in MPSoCs A.  Coskun  ,  T.  Rosing  ,  K.  Whisnant  and  K.  Gross    "Sta(c  and  dynamic  temperature-­‐ aware  scheduling  for  mul(processor  SoCs",    IEEE  Trans.  Very  Large  Scale  Integr.  Syst.,     vol.  16,    no.  9,    pp.1127  -­‐1140  2008     Fig. 3. Distribution of thermal hot spots, with DPM (ILP). A. Static Scheduling Techniques We next provide an extensive comparison of the ILP based techniques. We refer to our static approach as Min-Th&Sp. As discussed in Section III, we implemented the ILP for min- imizing thermal hot spots (Min-Th), energy balancing (Bal- En), and energy minimization (Min-En) to compare against Fig. 4. Distribution of spatial gradients, with DPM (ILP). hot spots. While Min-Th reduces the high spatial differentials above 15 C, we observe a substantial increase in the spatial gradients above 10 C. In contrast, our method achieves lower and more balanced temperature distribution in the die. In Fig. 5, we show how the magnitudes of thermal cycles vary with the scheduling method. We demonstrate the average per- Fig. 3. Distribution of thermal hot spots, with DPM (ILP). A. Static Scheduling Techniques We next provide an extensive comparison of the ILP based techniques. We refer to our static approach as Min-Th&Sp. As discussed in Section III, we implemented the ILP for min- imizing thermal hot spots (Min-Th), energy balancing (Bal- Fig. 4. Distribution of spatial gradients, with DPM (ILP). hot spots. While Min-Th reduces the high spatial differentials above 15 C, we observe a substantial increase in the spatial gradients above 10 C. In contrast, our method achieves lower and more balanced temperature distribution in the die. In Fig. 5, we show how the magnitudes of thermal cycles vary UCSD – System Energy Efficiency Lab
  33. Green   Marina  Zapater  |    Going  Green   33

      Scheduling and resource allocation policies in MPSoCs •  Energy  characteriza?on  of  applica?ons  allows  to   define  proac?ve  scheduling  and  resource  alloca?on   policies,  minimizing  hotspots   •  Hotspot  reduc?on  allows  to  raise  cooling   temperature   +1oC  means  around  7%  cooling  energy  savings  
  34. Green   Marina  Zapater  |    Going  Green   34

      Energy Optimization: Holistic Approach Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  35. Green   Marina  Zapater  |    Going  Green   35

      JIT Compilation in Virtual Machines •  Virtual  machines  compile     (JIT  compila?on)  the   applica?ons  into  na?ve  code   for  performance  reasons   •  The  op?mizer  is  general-­‐ purpose  and  focused  in   performance  opNmizaNon  
  36. Green   Marina  Zapater  |    Going  Green   36

      Back-­‐end   JIT compilation for energy minimization •  Applica?on-­‐aware  compiler   –  Energy  characteriza?on  of  applica?ons  and  transforma?ons   –  Applica?on-­‐dependent  op?mizer   –  Global  view  of  the  data  center  workload   •  Energy  op?mizer   –  Currently,  compilers  for  high-­‐end  processors  oriented  to  performance   op?miza?on   Front-­‐end   Op?mizer   Code  generator  
  37. Green   Marina  Zapater  |    Going  Green   37

      Energy saving potential for the compiler (MPSoCs) T.  Simunic,  G.  de  Micheli,  L.  Benini,  and  M.  Hans.  “Source  code  op?miza?on  and   profiling  of  energy  consump?on  in  embedded  systems,”  Interna?onal  Symposium  on   System  Synthesis,  pages  193  –  199,  Sept.  2000   – 77%  energy  reduc?on  in  MP3  decoder   Fei,  Y.,  Ravi,  S.,  Raghunathan,  A.,  and  Jha,  N.  K.  2004.  Energy-­‐op?mizing  source  code   transforma?ons  for  OS-­‐driven  embedded  sovware.  In  Proceedings  of  the  Interna?onal   Conference  VLSI  Design.  261–266.   – Up  to  37,9%  (mean  23,8%)  energy  savings  in   mul?process  applica?ons  running  on  Linux  
  38. Green   Marina  Zapater  |    Going  Green   38

      Global Management of Low Power Modes Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  39. Green   Marina  Zapater  |    Going  Green   39

      Global Management of Low-power modes (DVFS) •  DVFS  (Dynamic  Voltage  and  Frequency  Scaling)  is  based  upon:   –  As  suppy  voltage  decreases,  power  decreases  quadra?cally   –  But  delay  increases  (performance  decreases)  only  linearly   –  The  maximum  frequency  also  decreases  linearly   •  Currently,  low-­‐power  modes,  if  used,  are  ac?vated  by   inac?vity  of  the  server  opera?ng  system   •  To  minimize  energy  consump?on,  changes  between  modes   should  be  minimized   •  On  the  other  hand,  workload  knowledge  allows  to  globally   schedule  low-­‐power  modes  without  any  impact  in   performance  
  40. Green   Marina  Zapater  |    Going  Green   40

      Global Management of Low-power modes (DVFS) •  By  using  a  thermal  model,   we  can  predict  the   behaviour  of  a  workload   under  each  power  mode   •  We  can  use  resource   management  algorithms   to  change  DVFS  on   run?me,  adap?ng  to  our   workload.  
  41. Green   Marina  Zapater  |    Going  Green   41

      Temperature-aware floorplanning of MPSoCs Chip   Server   Rack   Room   MulN-­‐ room   Sched  &  alloc   2   2   1   ApplicaNon   OS/middleware   Compiler/VM   3   3   architecture   4   4   technology   5  
  42. Green   Marina  Zapater  |    Going  Green   42

      Temperature-aware floorplanning of MPSoCs
  43. Green   Marina  Zapater  |    Going  Green   43

      Potential energy savings with floorplanning –  Up  to  21oC  reduc?on  of  maximum  temperature   –  Mean:  -­‐12oC  in  maximum  temperature   –  Bejer  results  in  the  most  cri?cal  examples   Y.  Han,  I.  Koren,  and  C.  A.  Moritz.  Temperature  Aware  Floorplanning.  In  Proc.  of  the     Second  Workshop  on  Temperature-­‐Aware  Computer  Systems,  June  2005  
  44. Green   Marina  Zapater  |    Going  Green   44

      Temperature-aware floorplanning in 3D chips •  3D  chips  are  gewng  interest  due  to:   –  ↑  ↑    Scalability:  reduces  2D   equivalent  area   –  ↑  ↑    Performance:  shorter  wire   length   –  ↑  Reliability:  less  wiring     •  Drawback:   –  Huge  increment  of  hotspots       compared  with  2D  equivalent  designs  
  45. Green   Marina  Zapater  |    Going  Green   45

      Temperature-aware floorplanning in 3D chips •  Up  to  30oC  reduc?on  per  layer  in  a  3D  chip  with  4  layers  and   48  cores  
  46. Green   Marina  Zapater  |    Going  Green   46

      Outline •  Why  Data  Centers  (DC)  in  this   Workshop?   •  The  DC  in  next-­‐genera?on   applica?ons   •  Energy  consump?on  at  the   Data  Center   •  Insight  on  op?miza?on   strategies   •  Conclusions  
  47. Green   Marina  Zapater  |    Going  Green   47

      There is still much more to be done •  Smart  Grids   – Consume  energy  when  everybody  else  does  not   – Decrease  energy  consump?on  when  everybody   else  is  consuming   •  Reducing  the  electricity  bill   – Variable  electricity  rates   – Reac?ve  power  coefficient   – Peak  energy  demand  
  48. Green   Marina  Zapater  |    Going  Green   48

      Conclusions •  Reducing  PUE  is  not  the  same  than  reducing  energy   consump?on   –  IT  energy  consump?on  dominates  in  state-­‐of-­‐the-­‐art  data   centers   •  Applica?on  and  resources  knowledge  can  be  effec?vely   used  to  define  proacNve  policies  to  reduce  the  total  energy   consump?on   –  At  different  levels   –  In  different  scopes   –  Taking  into  account  cooling  and  computa?on  at  the  same  ?me   •  Proper  management  of  the  knowledge  of  the  data  center   thermal  behavior  can  reduce  reliability  issues   •  Reducing  energy  consump?on  is  not  the  same  than   reducing  the  electricity  bill  
  49. Green   Marina  Zapater  |    Going  Green   49

      Thank you for your attention Marina Zapater marina@die.upm.es   hjp://greenlsi.die.upm.es   (+34)  91  549  57  00    x-­‐4227     ETSI de Telecomunicación, B105 Avenida Complutense, 30 Madrid 28040, Spain Thanks  to  our  collaborators: