Green
Marina
Zapater
|
Going
Green
3
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐genera?on
applica?ons
• Energy
consump?on
at
the
Data
Center
• Insight
on
op?miza?on
strategies
• Conclusions
Green
Marina
Zapater
|
Going
Green
4
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐genera?on
applica?ons
• Energy
consump?on
at
the
Data
Center
• Insight
on
op?miza?on
strategies
• Conclusions
Green
Marina
Zapater
|
Going
Green
6
Motivation • Energy
consump?on
of
data
centers
– 1.3%
of
worldwide
energy
produc?on
in
2010
– USA:
80
mill
MWh/year
in
2011
=
1,5
x
NYC
– 1
data
center
=
25
000
houses
• More
than
43
Million
Tons
of
CO2
emissions
per
year
(2%
worldwide)
• More
water
consump?on
than
many
industries
(paper,
automo?ve,
petrol,
wood,
or
plas?c)
Jonathan
Koomey.
2011.
Growth
in
Data
center
electricity
use
2005
to
2010
Green
Marina
Zapater
|
Going
Green
7
Motivation José
M.Moya
|
Madrid
(Spain),
July
27,
2012
7
• It
is
expected
for
total
data
center
electricity
use
to
exceed
400
GWh/year
by
2015.
• The
required
energy
for
cooling
will
con?nue
to
be
at
least
as
important
as
the
energy
required
for
the
computa?on.
• Energy
op?miza?on
of
future
data
centers
will
require
a
global
and
mul?-‐disciplinary
approach.
0
5000
10000
15000
20000
25000
30000
35000
2000
2005
2010
World
server
installed
base
(thousands)
High-‐end
servers
Mid-‐range
servers
Volume
servers
0
50
100
150
200
250
300
2000
2005
2010
Electricity
use
(billion
kWh/year)
Infrastructure
Communica?ons
Storage
High-‐end
servers
Mid-‐range
servers
Volume
servers
5,75
Million
new
servers
per
year
10%
unused
servers
(CO2
emissions
similar
to
6,5
million
cars)
Green
Marina
Zapater
|
Going
Green
10
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐generaNon
applicaNons
• Energy
consump?on
at
the
Data
Center
• Insight
on
op?miza?on
strategies
• Our
vision
and
future
trends
Green
Marina
Zapater
|
Going
Green
11
The DC in next generation applications • Tradi?onal
uses
of
Data
Centers:
– Webmail,
Web
search,
Databases,
Social
networking
or
distributed
storage,
High-‐performance
compu?ng
(HPC),
Cloud
compu?ng
• Next-‐genera?on
applica?ons:
– Popula?on
monitoring
applica?ons:
e-‐Health,
Ambient
Assisted
Living
– Smart
ci?es
• Next-‐genera?on
applica?ons
generate
huge
amounts
of
data
• Need
to
store,
analize
and
generate
knowledge
Green
Marina
Zapater
|
Going
Green
12
Global energy optimization • Solu?on:
GoingGreen!
• How:
Global
energy
op?miza?on
strategies
– Proposal
of
a
holis?c
energy
op?miza?on
framework
– Minimizing
overall
power
consump?on
– Mul?-‐level
op?miza?on:
WBSN,
Personal
Servers
and
Data
Centers
Green
Marina
Zapater
|
Going
Green
13
Global energy optimization • Execu?ng
part
of
the
workload
in
the
Personal
Servers
– Classifying
tasks
depending
on
their
demand
– Resource
management
techniques
based
on
fast
run?me
alloca?on
algorithms
executed
on
the
Personal
Servers
– Execu?ng
some
tasks
in
Personal
Servers
instead
of
forwarding
load
to
DC.
– Up
to
10%
in
energy
savings
and
15%
execu?on
?me
savings
Green
Marina
Zapater
|
Going
Green
14
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐genera?on
applica?ons
• Energy
consumpNon
at
the
Data
Center
• Insight
on
op?miza?on
strategies
• Conclusions
Green
Marina
Zapater
|
Going
Green
15
Energy Consumption at the DC What is really a Data Center? hjp://cesvima.upm.es
WORKLOAD
Scheduler
Resource
Manager
ExecuNon
Green
Marina
Zapater
|
Going
Green
17
Energy Consumption at the DC Power consumption breakdown • The
major
contributors
to
electricity
costs
are:
– Cooling
(around
50%)
– Servers
(around
30-‐40%)
• The
most
common
metric
to
measure
efficiency
in
Data
Centers
is
PUE
(Power
Usage
Effec?veness)
Green
Marina
Zapater
|
Going
Green
18
Power Usage Effectiveness (PUE) • Average
PUE
≈
2
• State
of
the
Art:
PUE
≈
1,2
– The
important
part
is
IT
energy
consump?on
– Current
work
in
energy
efficient
data
centers
is
focused
in
decreasing
PUE
– Decreasing
PIT
does
not
decrease
PUE,
but
it
has
in
impact
on
the
electricity
bill
!"# = 1 !"#$ = !!"!#$ !!" =! ! !!!!!!!!!!= !!"#$% + !!""#$%& + !!"# !!"#$% ≈ !!""#$%& + !!" !!" !
Green
Marina
Zapater
|
Going
Green
20
Research trends Abstrac?on
level
• Higher
levels
of
abstrac?on
bring
more
benefits
• Some
areas
have
brought
more
benefits
than
others
Solu?ons
proposed
by
the
State
of
the
Art
Green
Marina
Zapater
|
Going
Green
21
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐genera?on
applica?ons
• Energy
consump?on
at
the
Data
Center
• Insight
on
opNmizaNon
strategies
• Conclusions
Green
Marina
Zapater
|
Going
Green
22
Our approach • Global
strategy
to
allow
the
use
of
mul?ple
informa?on
sources
to
coordinate
decisions
in
order
to
reduce
the
total
energy
consump?on
• Use
of
knowledge
about
the
energy
demand
characteris?cs
of
the
applicaNons,
and
characteris?cs
of
compuNng
and
cooling
resources
to
implement
proacNve
opNmizaNon
techniques
Green
Marina
Zapater
|
Going
Green
25
Resource Management at the Room level Leveraging heterogeneity – IT perspective • Use
heterogeneity
to
minimize
energy
consump?on
from
a
sta?c/dynamic
point
of
view
– StaNc:
Finding
the
best
data
center
set-‐up,
given
a
number
of
heterogeneous
machines
– Dynamic:
Op?miza?on
of
task
alloca?on
in
the
Resource
Manager
• We
show
that
the
best
solu?on
implies
an
heterogeneous
data
center
– Most
data
centers
are
heterogeneous
(several
genera?ons
of
computers)
– 5
to
22%
energy
savings
for
sta?c
solu?on
– 24%
to
47%
energy
savings
for
dynamic
solu?on
M.
Zapater,
J.M.
Moya,
J.L.
Ayala.
Leveraging
Heterogeneity
for
Energy
Minimiza?on
in
Data
Centers,
CCGrid
2012
Green
Marina
Zapater
|
Going
Green
26
Resource Management at the Room level Leveraging heterogeneity – IT perspective • Energy
profiling
of
tasks
of
the
SPEC
CPU
2006
benchmark
• Usage
of
MILP
algorithms
to
schedule
tasks
in
servers
where
they
consume
less
energy
• Implemented
in
a
real
resource
manager
(SLURM)
Green
Marina
Zapater
|
Going
Green
27
Resource Management at the Room level IT + Cooling perspective • Genera?ng
a
thermal
model
for
the
data
room:
– Data
Center
environmental
monitoring
to
gather
temperature,
humidity,
differen?al
pressure
– Predict
server
temperature
and
room
temperature
• Op?mum
resource
management
ajending
to
cooling
and
IT
power
– Real
environment
with
heterogeneous
servers
– SLURM
resource
manager
Green
Marina
Zapater
|
Going
Green
29
Resource Management at the Server level Leakage-temperature tradeoffs - Cooling • Exploring
the
leakage-‐temperature
tradeoffs
at
the
server
level
– At
higher
temperatures,
CPU
increases
power
consump?on
due
to
leakage
– To
decrease
CPU
temperature,
fan
speed
raises,
increasing
server
cooling
consump?on.
M.
Zapater,
J.L.
Ayala.,
J.M.
Moya,
K.
Vaidyanathan,
K.
Gross,
and
A.
K.
Coskun,
“Leakage
and
temperature
aware
server
control
for
improving
energy
efficiency
in
data
centers”,
DATE
2013
Green
Marina
Zapater
|
Going
Green
30
Resource Management at the Server level Leakage-temperature tradeoffs - Cooling • Implemented
fan
speed
controllers
that
reduce
server
power
consump?on
by
10%.
Fig. 4. Test 3 temperature sensor readings for the three different controllers 0.1 0.2 kWh) Energy difference between 1800RPM and 2400RPM for clustered allocation analytical model for leakage p fan speeds for varying utilization model, we implement a cooling Fig. 4. Test 3 temperature sensor readings for the three different controllers nd 2400RPM for clustered allocation analytical model for leakage power and find the optimum fan speeds for varying utilization values. Based our analytical model, we implement a cooling controller that adjusts the fan
Green
Marina
Zapater
|
Going
Green
32
Scheduling and resource allocation policies in MPSoCs A.
Coskun
,
T.
Rosing
,
K.
Whisnant
and
K.
Gross
"Sta(c
and
dynamic
temperature-‐ aware
scheduling
for
mul(processor
SoCs",
IEEE
Trans.
Very
Large
Scale
Integr.
Syst.,
vol.
16,
no.
9,
pp.1127
-‐1140
2008
Fig. 3. Distribution of thermal hot spots, with DPM (ILP). A. Static Scheduling Techniques We next provide an extensive comparison of the ILP based techniques. We refer to our static approach as Min-Th&Sp. As discussed in Section III, we implemented the ILP for min- imizing thermal hot spots (Min-Th), energy balancing (Bal- En), and energy minimization (Min-En) to compare against Fig. 4. Distribution of spatial gradients, with DPM (ILP). hot spots. While Min-Th reduces the high spatial differentials above 15 C, we observe a substantial increase in the spatial gradients above 10 C. In contrast, our method achieves lower and more balanced temperature distribution in the die. In Fig. 5, we show how the magnitudes of thermal cycles vary with the scheduling method. We demonstrate the average per- Fig. 3. Distribution of thermal hot spots, with DPM (ILP). A. Static Scheduling Techniques We next provide an extensive comparison of the ILP based techniques. We refer to our static approach as Min-Th&Sp. As discussed in Section III, we implemented the ILP for min- imizing thermal hot spots (Min-Th), energy balancing (Bal- Fig. 4. Distribution of spatial gradients, with DPM (ILP). hot spots. While Min-Th reduces the high spatial differentials above 15 C, we observe a substantial increase in the spatial gradients above 10 C. In contrast, our method achieves lower and more balanced temperature distribution in the die. In Fig. 5, we show how the magnitudes of thermal cycles vary UCSD – System Energy Efficiency Lab
Green
Marina
Zapater
|
Going
Green
33
Scheduling and resource allocation policies in MPSoCs • Energy
characteriza?on
of
applica?ons
allows
to
define
proac?ve
scheduling
and
resource
alloca?on
policies,
minimizing
hotspots
• Hotspot
reduc?on
allows
to
raise
cooling
temperature
+1oC
means
around
7%
cooling
energy
savings
Green
Marina
Zapater
|
Going
Green
35
JIT Compilation in Virtual Machines • Virtual
machines
compile
(JIT
compila?on)
the
applica?ons
into
na?ve
code
for
performance
reasons
• The
op?mizer
is
general-‐ purpose
and
focused
in
performance
opNmizaNon
Green
Marina
Zapater
|
Going
Green
36
Back-‐end
JIT compilation for energy minimization • Applica?on-‐aware
compiler
– Energy
characteriza?on
of
applica?ons
and
transforma?ons
– Applica?on-‐dependent
op?mizer
– Global
view
of
the
data
center
workload
• Energy
op?mizer
– Currently,
compilers
for
high-‐end
processors
oriented
to
performance
op?miza?on
Front-‐end
Op?mizer
Code
generator
Green
Marina
Zapater
|
Going
Green
37
Energy saving potential for the compiler (MPSoCs) T.
Simunic,
G.
de
Micheli,
L.
Benini,
and
M.
Hans.
“Source
code
op?miza?on
and
profiling
of
energy
consump?on
in
embedded
systems,”
Interna?onal
Symposium
on
System
Synthesis,
pages
193
–
199,
Sept.
2000
– 77%
energy
reduc?on
in
MP3
decoder
Fei,
Y.,
Ravi,
S.,
Raghunathan,
A.,
and
Jha,
N.
K.
2004.
Energy-‐op?mizing
source
code
transforma?ons
for
OS-‐driven
embedded
sovware.
In
Proceedings
of
the
Interna?onal
Conference
VLSI
Design.
261–266.
– Up
to
37,9%
(mean
23,8%)
energy
savings
in
mul?process
applica?ons
running
on
Linux
Green
Marina
Zapater
|
Going
Green
39
Global Management of Low-power modes (DVFS) • DVFS
(Dynamic
Voltage
and
Frequency
Scaling)
is
based
upon:
– As
suppy
voltage
decreases,
power
decreases
quadra?cally
– But
delay
increases
(performance
decreases)
only
linearly
– The
maximum
frequency
also
decreases
linearly
• Currently,
low-‐power
modes,
if
used,
are
ac?vated
by
inac?vity
of
the
server
opera?ng
system
• To
minimize
energy
consump?on,
changes
between
modes
should
be
minimized
• On
the
other
hand,
workload
knowledge
allows
to
globally
schedule
low-‐power
modes
without
any
impact
in
performance
Green
Marina
Zapater
|
Going
Green
40
Global Management of Low-power modes (DVFS) • By
using
a
thermal
model,
we
can
predict
the
behaviour
of
a
workload
under
each
power
mode
• We
can
use
resource
management
algorithms
to
change
DVFS
on
run?me,
adap?ng
to
our
workload.
Green
Marina
Zapater
|
Going
Green
43
Potential energy savings with floorplanning – Up
to
21oC
reduc?on
of
maximum
temperature
– Mean:
-‐12oC
in
maximum
temperature
– Bejer
results
in
the
most
cri?cal
examples
Y.
Han,
I.
Koren,
and
C.
A.
Moritz.
Temperature
Aware
Floorplanning.
In
Proc.
of
the
Second
Workshop
on
Temperature-‐Aware
Computer
Systems,
June
2005
Green
Marina
Zapater
|
Going
Green
44
Temperature-aware floorplanning in 3D chips • 3D
chips
are
gewng
interest
due
to:
– ↑
↑
Scalability:
reduces
2D
equivalent
area
– ↑
↑
Performance:
shorter
wire
length
– ↑
Reliability:
less
wiring
• Drawback:
– Huge
increment
of
hotspots
compared
with
2D
equivalent
designs
Green
Marina
Zapater
|
Going
Green
45
Temperature-aware floorplanning in 3D chips • Up
to
30oC
reduc?on
per
layer
in
a
3D
chip
with
4
layers
and
48
cores
Green
Marina
Zapater
|
Going
Green
46
Outline • Why
Data
Centers
(DC)
in
this
Workshop?
• The
DC
in
next-‐genera?on
applica?ons
• Energy
consump?on
at
the
Data
Center
• Insight
on
op?miza?on
strategies
• Conclusions
Green
Marina
Zapater
|
Going
Green
47
There is still much more to be done • Smart
Grids
– Consume
energy
when
everybody
else
does
not
– Decrease
energy
consump?on
when
everybody
else
is
consuming
• Reducing
the
electricity
bill
– Variable
electricity
rates
– Reac?ve
power
coefficient
– Peak
energy
demand
Green
Marina
Zapater
|
Going
Green
48
Conclusions • Reducing
PUE
is
not
the
same
than
reducing
energy
consump?on
– IT
energy
consump?on
dominates
in
state-‐of-‐the-‐art
data
centers
• Applica?on
and
resources
knowledge
can
be
effec?vely
used
to
define
proacNve
policies
to
reduce
the
total
energy
consump?on
– At
different
levels
– In
different
scopes
– Taking
into
account
cooling
and
computa?on
at
the
same
?me
• Proper
management
of
the
knowledge
of
the
data
center
thermal
behavior
can
reduce
reliability
issues
• Reducing
energy
consump?on
is
not
the
same
than
reducing
the
electricity
bill
Green
Marina
Zapater
|
Going
Green
49
Thank you for your attention Marina Zapater [email protected] hjp://greenlsi.die.upm.es
(+34)
91
549
57
00
x-‐4227
ETSI de Telecomunicación, B105 Avenida Complutense, 30 Madrid 28040, Spain Thanks
to
our
collaborators:
greenlsi
upura
itakigawa
hirokiwatanabe
kionawalker
yamdan
tatsui
o_hasegawa
shuntaros
kanojikajino
kitachan_black
tathi
bermonpainter
tammielis
michaelherold
smashingmag
afnizarnur
jnunemaker
phodgson
mclloyd
lara
kneath
moore
lemiorhan