Ways To Measure Your ORM's Cost

A presentation by @stuherbert  for @GanbaroDigital Ways To Measure Your
ORM’s Cost

@GanbaroDigital Today’s Talk

@GanbaroDigital 1. Why Measure

@GanbaroDigital 2. What To Measure

@GanbaroDigital 3. How To Measure

@GanbaroDigital You’ll take away practical advice that you can apply
straight away.

@GanbaroDigital Accepted Wisdom? “PHP is rarely the bottleneck”  - Rasmus
Lerdorf, 2010 http://talks.php.net/show/ digg/7

@GanbaroDigital agree? Do you

@GanbaroDigital Today, using an ORM is common practice.

@GanbaroDigital When you use an ORM, you are hiding persistent
data storage and retrieval behind an abstraction.

@GanbaroDigital Abstractions trade dev time for runtime

@GanbaroDigital is slow PHP’s runtime

@GanbaroDigital The current fashion is to write Java-style code for
a language that doesn’t have JVM-style performance.

@GanbaroDigital a free lunch Your ORM of choice is not

@GanbaroDigital Used wisely, an ORM can be a good thing.

@GanbaroDigital Used blindly, an ORM can cripple your Production performance.

@GanbaroDigital what your ORM costs? Do you know

@GanbaroDigital to know? Do you need

@GanbaroDigital Need To Measure? Your App MySQL ORM Single Server

@GanbaroDigital Need To Measure? Your App MySQL ORM Separate Tiers

@GanbaroDigital Need To Measure? Your App MySQL ORM Your App
ORM Scale Out Architecture

@GanbaroDigital Need To Measure? Your App MySQL ORM Your App
ORM Cloud Architecture MySQL

@GanbaroDigital You need to measure the costs of how you’re
using the ORM.

@GanbaroDigital How you use the ORM determines what the costs
are.

@GanbaroDigital It isn’t the ORM that is slow. It only
does what you ask it to.

@GanbaroDigital Measure to improve your understanding

@GanbaroDigital Measure to prevent problems reaching Production

@GanbaroDigital What would YOU measure?

@GanbaroDigital What To Measure? Your App MySQL ORM Your App
ORM

@GanbaroDigital What To Measure? Your App ORM MySQL Storage Network
Network

@GanbaroDigital What To Measure • Storage IOPS • Demands on
the network • Efﬁciency of data requests • Where the time goes

@GanbaroDigital Storage IOPS Your App ORM MySQL Storage Network Network

@GanbaroDigital Network Demand Your App ORM MySQL Storage Network Network

@GanbaroDigital Data Efﬁciency Your App ORM MySQL Storage Network Network

@GanbaroDigital Where The Time Goes Your App ORM MySQL Storage
Network Network

@GanbaroDigital Storage IOPS? Why measure

@GanbaroDigital Dev Environments • Dev laptops use PCIe SSDs •
Typically perform around 100K IOPS • Cheap SATA SSDs can do 50K+ IOPS

@GanbaroDigital Servers • Servers typically use SAS RAID arrays •
Yields 0.4 - 2K IOPS • SATA-equipped servers are even slower!

@GanbaroDigital In The Cloud • Servers typically over-subscribed • Need
large, slow disks to support all the customers and their VMs • Maybe 1K IOPS shared between many VMs!

@GanbaroDigital Storage Comparison Dev  Laptop Dedicated  Server Cloud   Server
50-100K 0.4-2K 1K (shared)

@GanbaroDigital Storage Comparison Dev Laptop Dedicated Server Cloud Server 0
25 50 75 100

@GanbaroDigital For the ﬁrst time in the   history of
web development, your dev machine’s storage is faster than Production’s storage.

@GanbaroDigital Faster?!? It stomps all over Production!

@GanbaroDigital Storage IOPS How to measure

@GanbaroDigital iostat -d -x 5

@GanbaroDigital Two Key Columns • r/s • w/s

@GanbaroDigital Other Tips • Move your database onto its own
ﬁlesystem • Don’t forget to monitor how much IOPS your logging causes

@GanbaroDigital Storage IOPS Your App ORM MySQL Storage Network Network

@GanbaroDigital Network Demands? Why measure

Stopwatch - William Warby - https://ﬂic.kr/p/62n3eY

@GanbaroDigital Request Timeline Your App ORM MySQL Network

@GanbaroDigital Your app is paused until the ORM has ﬁnished
processing your request.

@GanbaroDigital In the meantime, your customer is kept waiting for
their webpage.

@GanbaroDigital How many SQL requests does your ORM actually make?

@GanbaroDigital Your app can make requests without you realising.

@GanbaroDigital The ORM can make multiple requests without you realising.

@GanbaroDigital Solve By Scaling Out? Your App MySQL ORM Your
App ORM Scale Out Architecture

@GanbaroDigital Scaling out allows you to serve more customers at
once. It does not help you to serve a single customer faster.

@GanbaroDigital Network Demand How to measure

@GanbaroDigital mysql-proxy Completely independent logging

@GanbaroDigital MySQL Proxy • Point your app at mysql-proxy •
Use the ‘debug-blind.lua’ proxy plugin

@GanbaroDigital What It Tells Us • Complete log of all
SQL requests • How long each request takes • Number of rows returned • Whether or not an index was used

@GanbaroDigital Network Demand Your App ORM MySQL Storage Network Network

@GanbaroDigital Data Efﬁciency? Why measure

Junk Mail - Judith E. Bell - https://ﬂic.kr/p/pgMxPA

@GanbaroDigital Your ORM can request lots of data that your
app’s code never sees.

@GanbaroDigital Why? Hydration of complex relationships.

@GanbaroDigital http://ocramius.github.io/blog/ doctrine-orm-optimization-hydration/

@GanbaroDigital Your app may use less than 10% of the
data retrieved from the database.

@GanbaroDigital Costs Twice • Database load • Application server load

@GanbaroDigital Puts Pressure On • CPU • RAM • Storage
IOPS

@GanbaroDigital Hurts • Response times • Simultaneous requests • Database
replication

@GanbaroDigital Data Efﬁciency How to measure

@GanbaroDigital manual hacks required

@GanbaroDigital The Approach 1. Modify your ORM’s database driver to
log fields from the database. 2. Modify your code to count the size of your entities. 3. Efficiency = entity size / fields from DB

@GanbaroDigital The lower the efﬁciency, the more time is being
wasted.

@GanbaroDigital Data Efﬁciency Your App ORM MySQL Storage Network Network

@GanbaroDigital where the time goes? Why measure

Wait - Hernán Piñera - https://ﬂic.kr/p/o1mSAi

@GanbaroDigital Your app can spend up to 80% of its
total time waiting for the ORM.

@GanbaroDigital where the time goes? How to measure

@GanbaroDigital Proﬁle your app

@GanbaroDigital • Xdebug • XHProf • Blackﬁre.IO • Zend Z-Ray
• AppDynamics • or build your own instrumentation using Graphite Proﬁlers For PHP

@GanbaroDigital Where The Time Goes Your App ORM MySQL Storage
Network Network

@GanbaroDigital Plenty of existing conference talks about using a proﬁler.
Go and Google for them.

@GanbaroDigital Proﬁle your app

@GanbaroDigital said than done Much easier

@GanbaroDigital Proﬁle your app under realistic load

@GanbaroDigital Proﬁle your app under realistic load in a representative
environment.

@GanbaroDigital Proﬁle your app with production data under realistic load
in a representative environment.

Stuart Herbert ~ @stuherbert Founder @GanbaroDigital

Ways To Measure Your ORM's Cost

Ways To Measure Your ORM's Cost

More Decks by Stuart Herbert

Other Decks in Programming

Featured

Transcript