Velocity 6-24-2010 Velocity 6-24-2010 1 1 Hidden Scalability Hidden Scalability Gotchas Gotchas in in Memcached Memcached and Friends and Friends Neil Neil Gunther Gunther, , Performance Dynamics Performance Dynamics Shanti Subramanyam Shanti Subramanyam, , Oracle Corp Oracle Corporation oration Stefan Stefan Parvu Parvu, , Oracle Finland Oracle Finland
Velocity 6-24-2010 Velocity 6-24-2010 4 4 Scalability Strategies Scalability Strategies • Qualitative scalability – Scale up, e.g., big SMP servers – Scale out,e.g, many cheap servers (Unis) • Quantitative scalability – What this talk is about – Need controlled measurements – Need numbers to see cost-benefit
Velocity 6-24-2010 Velocity 6-24-2010 6 6 Capacity Planning Capacity Planning • You know you need it – The planning bit, especially – Data ain’t information – Info is hidden in the data • Just like finance, you need a model Metrics + Models == Information
Velocity 6-24-2010 Velocity 6-24-2010 8 8 Why Controlled Measurements? Why Controlled Measurements? Trying to predict scalability by looking at time series data is like trying to predict the stock mkt by watching the DJX ticker
Velocity 6-24-2010 Velocity 6-24-2010 9 9 Bad Throughput Bad Throughput Measurements Measurements Need x-axis to be load (N) defined in terms of processes or users Need throughput measured in steady state (which this isn’t)
Velocity 6-24-2010 Velocity 6-24-2010 10 10 Average Throughput in Time Average Throughput in Time This is what steady state looks like as function of time. It corresponds to ONE throughput load point (N).
Velocity 6-24-2010 Velocity 6-24-2010 27 27 Parameter interpretation Parameter interpretation • Why α ~ 0 – Cache further partitioned – Single lock replaced by multiple locks • Why β > 0? – Is it in mcd code? – Could it be in O/S, H/W, …?
Velocity 6-24-2010 Velocity 6-24-2010 28 28 Scaling Among Friends Scaling Among Friends Scalability as a function of Scalability as a function of virtual virtual users users ( (“ “friends friends” ”) not threads ) not threads
Velocity 6-24-2010 Velocity 6-24-2010 33 33 Old scaling rules will be broken Old scaling rules will be broken • Current scale-out strategy relies on using older cheap hardware • Older hardware is often single CPU – Single-threadedness of mcd is ok • Newer hardware will be multicore – New hardware is faster with lots of cores – But mcd won’t be able to utilize all cores – Multiple mcd instances are mgmt headache
Velocity 6-24-2010 Velocity 6-24-2010 35 35 Summary Summary • Current mcd versions are thread limited – OK for older uniprocessor servers – Not OK for deployment on new multicores – Reason: unused processor capacity • Controlled measurements – Not time-series prod data – Steady state throughput • Quantify scalability – Data + Models == Information – Goal is reduce contention (α) and coherency (β) – Nmax: increased from 6 to 48 threads
drqz
scarletplover
oracle4engineer
dzeyelid
line_developers
nabedge
nearme_tech
hato3isfriend
bob3bob3
kojira
k1low
mixi_engineers
mot_techtalk
lara
revolveconf
sachag
philhawksworth
denniskardys
chriscoyier
rocio
bkeepers
schacon
iamctodd
zakiyama
geeforr