Brooks, Cooks and Response Time Scalability

Slide 1

Slide 1 text

Brooks, Cooks, and Response Time Scalability Dr. Neil Gunther A.A. Michelson Award 2008 Performance Dynamics Hotsos Symposium March 8, 2011 SM c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 1 / 51

Slide 2

Slide 2 text

Motivation Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 2 / 51

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Motivation When we left our story last time... Learned how to quantify scalability: Hotsos 2007: “Guerrilla Scalability: How To Do Virtual Load Testing” Hotsos 2010: “How to Quantify Oracle Database Scalability: Fundamentals” Hotsos 2010: “How to Quantify Oracle Database Scalability: Examples” Equal bang for the buck: linear concurrency c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 3 / 51

Slide 5

Slide 5 text

Motivation When we left our story last time... Learned how to quantify scalability: Hotsos 2007: “Guerrilla Scalability: How To Do Virtual Load Testing” Hotsos 2010: “How to Quantify Oracle Database Scalability: Fundamentals” Hotsos 2010: “How to Quantify Oracle Database Scalability: Examples” Equal bang for the buck: linear concurrency Diminishing Returns: contention overhead c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 3 / 51

Slide 6

Slide 6 text

Motivation When we left our story last time... Learned how to quantify scalability: Hotsos 2007: “Guerrilla Scalability: How To Do Virtual Load Testing” Hotsos 2010: “How to Quantify Oracle Database Scalability: Fundamentals” Hotsos 2010: “How to Quantify Oracle Database Scalability: Examples” Equal bang for the buck: linear concurrency Diminishing Returns: contention overhead Negative return on investment: coherency overhead c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 3 / 51

Slide 7

Slide 7 text

Motivation When we left our story last time... Learned how to quantify scalability: Hotsos 2007: “Guerrilla Scalability: How To Do Virtual Load Testing” Hotsos 2010: “How to Quantify Oracle Database Scalability: Fundamentals” Hotsos 2010: “How to Quantify Oracle Database Scalability: Examples” Equal bang for the buck: linear concurrency Diminishing Returns: contention overhead Negative return on investment: coherency overhead Calculate scalability curve from performance measurements c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 3 / 51

Slide 8

Slide 8 text

Motivation Universal Scalability Law (USL) N users or Oracle processes c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 9

Slide 9 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 10

Slide 10 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N CN(α, β) = N 1 + α (N − 1) + β N(N − 1) (1) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 11

Slide 11 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N CN(α, β) = N 1 + α (N − 1) + β N(N − 1) (1) Three Cs: c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 12

Slide 12 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N CN(α, β) = N 1 + α (N − 1) + β N(N − 1) (1) Three Cs: 1 Concurrency c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 13

Slide 13 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N CN(α, β) = N 1 + α (N − 1) + β N(N − 1) (1) Three Cs: 1 Concurrency 2 Contention (amount α < 1) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 14

Slide 14 text

Motivation Universal Scalability Law (USL) N users or Oracle processes C relative capacity function of N CN(α, β) = N 1 + α (N − 1) + β N(N − 1) (1) Three Cs: 1 Concurrency 2 Contention (amount α < 1) 3 Coherency (amount β < 1) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 4 / 51

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Motivation Coherency overhead in Oracle apps Example Many ORACLE processes running on a multiprocessor OLTP application: shared writable data An ORA process requests to update data in the row of a table Must wait for RDBMS lock ... Finally, process gets the ORA lock: permission to write ORA process starts executing on CPU but cannot write Question: Why not? Hint: Multiple CPUs means multiple caches Answer: Your cache line is ﬂagged “dirty” (out of date) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 5 / 51

Slide 18

Slide 18 text

Motivation Coherency overhead in Oracle apps Example Many ORACLE processes running on a multiprocessor OLTP application: shared writable data An ORA process requests to update data in the row of a table Must wait for RDBMS lock ... Finally, process gets the ORA lock: permission to write ORA process starts executing on CPU but cannot write Question: Why not? Hint: Multiple CPUs means multiple caches Answer: Your cache line is ﬂagged “dirty” (out of date) Data in your copy of DB table in not the most recent c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 5 / 51

Slide 19

Slide 19 text

Motivation Coherency overhead in Oracle apps Example Many ORACLE processes running on a multiprocessor OLTP application: shared writable data An ORA process requests to update data in the row of a table Must wait for RDBMS lock ... Finally, process gets the ORA lock: permission to write ORA process starts executing on CPU but cannot write Question: Why not? Hint: Multiple CPUs means multiple caches Answer: Your cache line is ﬂagged “dirty” (out of date) Data in your copy of DB table in not the most recent Must request latest copy before proceeding c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 5 / 51

Slide 20

Slide 20 text

Motivation Coherency overhead in Oracle apps Example Many ORACLE processes running on a multiprocessor OLTP application: shared writable data An ORA process requests to update data in the row of a table Must wait for RDBMS lock ... Finally, process gets the ORA lock: permission to write ORA process starts executing on CPU but cannot write Question: Why not? Hint: Multiple CPUs means multiple caches Answer: Your cache line is ﬂagged “dirty” (out of date) Data in your copy of DB table in not the most recent Must request latest copy before proceeding Wait longer for your data to become consistent, i.e., β > 0 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 5 / 51

Slide 21

Slide 21 text

Motivation Problem for Oracle DBAs Natural to expresses scalability growth as throughput X ORA presents “waiting time” metrics, not throughputs Standard v$ tables AWR (Automatic Workload Repository) OWI (Oracle Wait Interface) Will see how to get there via round-trip times : RTT = N X (2) from queueing theory Eqn.(2) is equivalent to Little’s law: N = XR NOTE: R always inversely proportional to throughput X See my blog post Bandwidth vs Latency—The World is Curved c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 6 / 51

Slide 22

Slide 22 text

Motivation Shape of performance metrics Raw time-series data = information Throughput as a function of load X(N) is generally a concave function Response time as a function of load R(N) is generally a convex function c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 7 / 51

Slide 23

Slide 23 text

Brooks’ Law and Speedup Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 8 / 51

Slide 24

Slide 24 text

Brooks’ Law and Speedup The Mythic Fred Brooks Original edition 1965 Original edition 1975 Theorem (Brooks) Adding more manpower to a late project just makes it later. c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 9 / 51

Slide 25

Slide 25 text

Brooks’ Law and Speedup Brooks’ law does not mean this c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 10 / 51

Slide 26

Slide 26 text

Brooks’ Law and Speedup That’s actually a case of Amdahl’s law Theorem (Amdahl) Parallel speedup is limited by the serial portion of the program. Example Pregnancy is a serial process. Only runs on a single (married?) processor. Minimum gestation is T1 9 mths No matter how many women (processors) are involved. More about Amdahl’s law shortly c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 11 / 51

Slide 27

Slide 27 text

Brooks’ Law and Speedup Hell’s Kitchen Brooks’ law == Too many cooks c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 12 / 51

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Brooks’ Law and Speedup Sources of delay Modeling the Mythical Man Month Turns out Brook’s law contains components too. My November 4, 2007 blog post: Modeling the Mythical Man Month 3 components: 1 Raw manpower to shorten the schedule c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 13 / 51

Slide 30

Slide 30 text

Brooks’ Law and Speedup Sources of delay Modeling the Mythical Man Month Turns out Brook’s law contains components too. My November 4, 2007 blog post: Modeling the Mythical Man Month 3 components: 1 Raw manpower to shorten the schedule 2 Round-table conferencing to share status c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 13 / 51

Slide 31

Slide 31 text

Brooks’ Law and Speedup Sources of delay Modeling the Mythical Man Month Turns out Brook’s law contains components too. My November 4, 2007 blog post: Modeling the Mythical Man Month 3 components: 1 Raw manpower to shorten the schedule 2 Round-table conferencing to share status 3 One-on-one conversations b/w experts and novices c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 13 / 51

Slide 32

Slide 32 text

Brooks’ Law and Speedup Sources of delay 1. Raw manpower c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 14 / 51

Slide 33

Slide 33 text

Brooks’ Law and Speedup Sources of delay 1. Raw manpower Divide and conquer c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 14 / 51

Slide 34

Slide 34 text

Brooks’ Law and Speedup Sources of delay Program takes time T1 to execute on a uniprocessor Work T1 −→ → → → → → → Divide across N = 6 processors (people) as equal sub-tasks Conquer it by running all 6 sub-tasks simultaneously Parallel performance: T6 = T1 6 = 0.17T1 (i.e., 16.67% of N = 1 time) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 15 / 51

Slide 35

Slide 35 text

Brooks’ Law and Speedup Sources of delay In general TN = T1 N 0 2 4 6 8 10 People 2 4 6 8 10 12 Months N = 1 person takes 10 months N = 10 people take 1 month No people take ∞t time c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 16 / 51

Slide 36

Slide 36 text

Brooks’ Law and Speedup Sources of delay 2. Round-table conferencing c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 17 / 51

Slide 37

Slide 37 text

Brooks’ Law and Speedup Sources of delay 2. Round-table conferencing Bored meeting c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 17 / 51

Slide 38

Slide 38 text

Brooks’ Law and Speedup Sources of delay Everyone stops what they’re doing to listen to 1 person’s instructions −→ → → → → → → → Suppose this happens just 25% of the time T1 (dark blue) Only remaining 75% of T1 can be executed in parallel by 6 people T6 = 3 4 T1 6 + 1 4 T1 = 3 8 T1 = 0.375T1 (i.e., only 38% of N1 ) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 18 / 51

Slide 39

Slide 39 text

Brooks’ Law and Speedup Sources of delay In general: TN = (1 − α) T1 N parallel portion + αT1 sequential portion Fixed delay due to meetings 0 2 4 6 8 10 People 2 4 6 8 10 Months For α = 0.25 and T1 = 10 months, best time will be αT1 = 2.5 months even with ∞t number of people! c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 19 / 51

Slide 40

Slide 40 text

Brooks’ Law and Speedup Sources of delay 3. Tˆ ete-` a-tˆ ete conversations c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 20 / 51

Slide 41

Slide 41 text

Brooks’ Law and Speedup Sources of delay 3. Tˆ ete-` a-tˆ ete conversations 1-on-1 == pairwise interaction c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 20 / 51

Slide 42

Slide 42 text

Brooks’ Law and Speedup Sources of delay Number of possible pairwise combinations: N 2 ≡ N! (N − 2)! 2! = N(N − 1) 2 For N = 6 people 1 2 6(6 − 1) = 15 conversations Growing delay due to 1 on 1 mtgs 0 2 4 6 8 10 People 5 10 15 Months This is really Brooks’ law (grows linearly with N) TN = T1 N + βN(N − 1) T1 N c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 21 / 51

Slide 43

Slide 43 text

Brooks’ Law and Speedup Sources of delay Combining all delay types TN = T1 N + α(N − 1) T1 N + βN(N − 1) T1 N 0 2 4 6 8 10 People 5 10 15 20 Months Shifts the scheduling delay curve even higher! c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 22 / 51

Slide 44

Slide 44 text

Brooks’ Law and Speedup Productivity Speedup == Productivity Speedup deﬁnition: SN = T1 TN (3) Overhead for just round-table meetings: SN = T1 T1 N + α(N − 1) T1 N = 1 1 N + α N − 1 N which we can simplify further to c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 23 / 51

Slide 45

Slide 45 text

Brooks’ Law and Speedup Productivity Amdahl productivity SN = N 1 + α(N − 1) (4) 0 2 4 6 8 10 People 1 2 3 4 5 Output Example Fraction of time in roundtable meetings α = 25% = 1/4 Ceiling on productivity 1/α = 4 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 24 / 51

Slide 46

Slide 46 text

Brooks’ Law and Speedup Productivity Amdahl speedup Amdahl speedup has no 1-on-1 delays (i.e., β = 0) Rewrite (4) as T1 TN = 1 1 N + α N − 1 N Example In previous example α = 1/4 and N = 6 T6 = 1 6 T1 + 1 4 5 6 T1 = 3 8 T1 same as before Speedup is only S6 = 8 3 = 2.67 (cf. S6 = 6 linear) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 25 / 51

Slide 47

Slide 47 text

Brooks’ Law and Speedup Productivity Combined speedup Combining all 3 types of delay effects: SN = T1 T1 N + α(N − 1) T1 N + βN(N − 1) T1 N = 1 1 N + α N − 1 N + βN N − 1 N which we can simplify further as: SN = N 1 + α(N − 1) + βN(N − 1) (5) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 26 / 51

Slide 48

Slide 48 text

Brooks’ Law and Speedup Productivity Combined productivity But eqn.(5) is precisely the USL scalability law in eqn.(1) CN(α, β) = N 1 + α(N − 1) + βN(N − 1) 0 2 4 6 8 10 People 0.5 1.0 1.5 2.0 Output Handles worse scalability than Amdahl’s law, even if β < α ! c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 27 / 51

Slide 49

Slide 49 text

Repairman Queueing Model Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 28 / 51

Slide 50

Slide 50 text

Repairman Queueing Model Meet the Repairman Repairs are done at the Genius Bar Finite number of people in the store at any time Average rate at which products need ﬁxing Z mths, yrs Average repair time at Genius Bar S mins, hrs Genius Bar is its own queue c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 29 / 51

Slide 51

Slide 51 text

Repairman Queueing Model Repairman queueing model Asynchronous throughput: Synchronous throughput: Normalized synchronous throughput: XN = N RN + Z XN = N NS + Z XN X1 = N NS + Z 1 S + Z −1 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 30 / 51

Slide 52

Slide 52 text

Repairman Queueing Model Mr. Amdahl synchronizes his visits to the Repairman From roundtable meetings (Amdahl’s law): TN = αT1 + (1 − α) T1 N But T1 = S + Z = serial + parallel for N = 1 TN = α(S + Z) + (1 − α) S + Z N Magic identity: α = S S + Z produces this simpliﬁcation TN = S + Z N Speedup SN = T1 TN = S + Z S + Z N = N(S + Z) NS + Z SN = N NS + Z (S + Z) Same as normalized synchronous throughput in Repairman c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 31 / 51

Slide 53

Slide 53 text

Repairman Queueing Model The Repairman and the USL Theorem (Gunther 2002) Amdahl’s law is equivalent to the synchronous throughput bound in the machine repairman queueing model of a multiprocessor or multicore. Theorem (Gunther 2008) The USL is equivalent to the synchronous throughput bound in the machine repairman queueing model with linearly load-dependent service time. Corollary The USL subsumes Amdahl’s law when β = 0. Proof. See “A General Theory of Computational Scalability Based on Rational Functions” http://arxiv.org/abs/0808.1431 It’s all connected by queueing theory c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 32 / 51

Slide 54

Slide 54 text

USL Response Time Scalability Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 33 / 51

Slide 55

Slide 55 text

USL Response Time Scalability From performance measurements (X1 , X2 , . . . , XN ) on test rig: CN = XN X1 From USL regression in Excel we get: CN (α, β) = N 1 + α(N − 1) + βN(N − 1) Use this to calculate XN for any N value using: XN = CN (α, β) X1 Round-trip time eqn.(2) in repairman queue is given by: RTT = RN + Z = N XN Use this to determine RN and compare with response-time measurements c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 34 / 51

Slide 56

Slide 56 text

Worked Examples Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 35 / 51

Slide 57

Slide 57 text

Worked Examples Oracle Order-Entry Workload Example 1: Oracle Order-Entry app Controlled measurements using Benchmark Factory1 Use USLcalc.xls spreadsheet available from my web site (http://www.perfdynamics.com/) 1Peter Stalder, “How to Quantify Oracle Database Scalability: Part II”, Hotsos 2010 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 36 / 51

Slide 58

Slide 58 text

Worked Examples Oracle Order-Entry Workload Example 1: Oracle Order-Entry app Regression analysis in Excel c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 37 / 51

Slide 59

Slide 59 text

Worked Examples Oracle Order-Entry Workload Example 1: Oracle Order-Entry app USL parameter mappings α = b − a β = a Nmax = 1 − α β Nopt = 1 α c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 38 / 51

Slide 60

Slide 60 text

Worked Examples Oracle Order-Entry Workload Example 1: Oracle Order-Entry app USL throughput data and USL model (note concave shape) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 39 / 51

Slide 61

Slide 61 text

Worked Examples Oracle Order-Entry Workload Example 1: Oracle Order-Entry app USL response-time data and USL model (note convex shape) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 40 / 51

Slide 62

Slide 62 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app November 15, 09 © Performance Dynamics 5 Load Test Measurements 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0 5 10 15 20 25 30 VUsers (N) Searches per Second Measured TPS VUs (N) X(N) 1 0.3311 5 0.8986 10 1.0899 15 1.1485 20 1.1199 25 1.1280 Throughput Measurements Controlled measurements using LoadRunner 7.5 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 41 / 51

Slide 63

Slide 63 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app November 15, 09 © Performance Dynamics 6 Quadratic Transform y = 0.0025x2 + 0.2051x R2 = 0.9995 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 0 5 10 15 20 25 30 Effective Users (N- 1) Deviation from Linear Regression Analysis Regression analysis in Excel c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 42 / 51

Slide 64

Slide 64 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app November 15, 09 © Performance Dynamics 7 Trendline Parameters Super Serial Quadratic Coefficients Parameter Values a 2.50E-03 ! 0.2026 b 0.2051 " 0.0123 c 0.0000 Nmax 19 Nopt 5 Parameters Mappings USL parameter mappings α = b − a β = a Nmax = 1 − α β Nopt = 1 α c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 43 / 51

Slide 65

Slide 65 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app November 15, 09 © Performance Dynamics 9 Predicted and Modeled X(N) Predicted Scalability 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 20 25 30 LR 7,5 VUsers Searches per Second Modeled Measured USL throughput data and USL model (note concave shape) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 44 / 51

Slide 66

Slide 66 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app Response Time Measurements Predicted Delay Percent VU Modelled Measured Error 1 3.02 2.96 2.03 5 5.62 5.44 3.29 10 9.21 8.94 3.00 15 13.17 12.69 3.80 20 17.52 17.23 1.66 25 22.24 21.44 3.71 Z X(N) N R(N) ! = Calculated response times c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 45 / 51

Slide 67

Slide 67 text

Worked Examples Oracle CRM Application Example 2: Oracle CRM app November 15, 09 © Performance Dynamics 11 Predicted Application Delay Predicted Delay (Z=0) 0.00 5.00 10.00 15.00 20.00 25.00 0 5 10 15 20 25 30 LR 7.5 VUsers Seconds Modelled Measured USL response-time data and USL model (note convex shape) c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 46 / 51

Slide 68

Slide 68 text

Wrap Up Outline 1 Motivation 2 Brooks’ Law and Speedup Sources of delay Productivity 3 Repairman Queueing Model 4 USL Response Time Scalability 5 Worked Examples Oracle Order-Entry Workload Oracle CRM Application 6 Wrap Up c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 47 / 51

Slide 69

Slide 69 text

Wrap Up Summary Model throughput XN with USL 2 Calculate RN from Little’s law: RTT = N/XN 3 Apply USL in Excel, R, Mathematica, etc. Cf. v$, AWR, OWI metrics at selected user load N USL contains Brooks’ law and Amdahl’s law (a good thing) USL is equivalent to synchronous repairman queueing model USL models worst case synchronous queueing Your app should scale better than USL If it doesn’t, change your data 2Gunther, Hotsos 2007 3Gunther and Stalder, Hotsos 2010 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 48 / 51

Slide 70

Slide 70 text

Wrap Up This just in ... 1 B. Schwartz: “Is VoltDB really as scalable as they claim?”, Feb 28, 2011 Applies USL model Quantifies scalabiity of no-SQL VoltDB Not response time analysis Not Oracle 2 D. Abercrombie: “Response time analysis based on AWR data”, Mar 5, 2011 Definitely Oracle Definitely response-time analysis Not USL ... (yet) 3 G. Hendriksen: “GAPP Improvements: a Method to Diagnose and Predict Performance in Complex Architectures”, Mar 7, 2011 Uses Oracle Data Mining (ODM) Definitely response-time analysis Not USL ... (yet) See my blog post: “Mine the GAPP” Details at 12 ... Hotsos 2012 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 49 / 51

Slide 71

Slide 71 text

Wrap Up Wanna know even more? Chapters 6 and 14 Chapters 4–6 Also check out: Special USL web page Guerrilla performance classes c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 50 / 51

Slide 72

Slide 72 text

Wrap Up Thank you for attending! Castro Valley, California www.perfdynamics.com perfdynamics.blogspot.com twitter.com/DrQz [email protected] +1-510-537-5758 c 2017 Performance Dynamics Brooks, Cooks, and Response Time Scalability September 6, 2017 51 / 51