Better Visualization Tools for Better Performance Management

Updated version of the presentation given at the annual Hotsos Symposium, March 5, 2008

Transcript

1. Better Visualization Tools for Better Performance Management Dr. Neil J. Gunther Performance Dynamics Hotsos Symposium. March 5, 2008 (updated) www.perfdynamics.com c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 1 / 34
2. The Problem Outline 1 The Problem Performance Visualization Defined What Makes PerfViz Hard? 2 Role Models 1960 to 1980 1980 to 2000 Century 21 3 Barycentric View Revisiting the Goal Some Facts About Triangles 4 Applications Multiprocessor Visualization Network Visualization Response Time Visualization ORACLE Visualization 5 Postscript c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 2 / 34
3. The Problem The Business Problem “Improving data visualization paradigms for performance management is an orphaned area of performance tool development [4]...” (because): 1 Tool vendors avoid investing in development if they don’t see any demand 2 Performance analysts and capacity planners don’t demand what they have not conceived c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 3 / 34
4. The Problem The Technical Problem 17 Cognitive Impedance Matching Z L Z S Z L Z S Digital computer Cognitive computer Find the best cognitive impedence match: ZSrc ZLoad c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 4 / 34
5. The Problem Performance Visualization Defined Performance Visualization The Question: Is it possible to present performance data collected from modern complex computing environments in a way similar to scientific visualization as applied to complex physical data? We’ll call this goal “PerfViz” for short Get beyond data reporting [1, 2] to data exploration [3] Some PerfViz tools already exists, especially for HPC Can we use them? Can we do better? c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 5 / 34
6. The Problem What Makes PerfViz Hard? Best Cognitive Impedence Match Sounds simple. What is cognition? How does the brain work? (See [10]) What is vision? If we understood our neural circuitry, we could choose the best visualization paradigms. c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 6 / 34
7. The Problem What Makes PerfViz Hard? Neural Circuitry All mammals have a neocotex. Mapped complete circuitry for the Makak monkey. Hierarchy of Cortical Regions • Hierarchy e mammal ne • High variab between sp (relatively fixed • Relative siz can vary by between ind a species From: Felleman and Van Essen c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 7 / 34
8. The Problem What Makes PerfViz Hard? Human Neural Circuits We know more about a monkey brain than a human brain. We also know it can be easily fooled. “I am NOT an animal!” (He’s actually a performance analyst.) Good Z match ⇒ “So simple, a caveman could do it!” c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 8 / 34
9. The Problem What Makes PerfViz Hard? Scientific Visualization Physical data is generally (3 + 1)-dimensional 3-space and 1-time dimension Weather forecasting Tornado simulations (Navier-Stokes equation) Oil exploration CAT scans (volume tomography) c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 9 / 34
10. The Problem What Makes PerfViz Hard? Demo: Tornadoes and Cyclones Tornado simulation Navier-Stokes equation Text structure as a cyclone [8] c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 10 / 34
11. The Problem What Makes PerfViz Hard? Performance Visualization Performance data is N-dimensional (and N is large) Solaris captures about N = 300 UNIX system metrics Oracle 10g v$-tables contain about N = 200 metrics Pick N = 2: your screen is only 2-dimensional c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 11 / 34
12. Role Models Outline 1 The Problem Performance Visualization Defined What Makes PerfViz Hard? 2 Role Models 1960 to 1980 1980 to 2000 Century 21 3 Barycentric View Revisiting the Goal Some Facts About Triangles 4 Applications Multiprocessor Visualization Network Visualization Response Time Visualization ORACLE Visualization 5 Postscript c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 12 / 34
13. Role Models 1960 to 1980 The Swinging 60’s and 70’s c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 13 / 34
14. Role Models 1980 to 2000 The Roaring 80’s and 90’s c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 14 / 34
15. Role Models Century 21 21st Century InfoViz c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 15 / 34
16. Barycentric View Outline 1 The Problem Performance Visualization Defined What Makes PerfViz Hard? 2 Role Models 1960 to 1980 1980 to 2000 Century 21 3 Barycentric View Revisiting the Goal Some Facts About Triangles 4 Applications Multiprocessor Visualization Network Visualization Response Time Visualization ORACLE Visualization 5 Postscript c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 16 / 34
17. Barycentric View Revisiting the Goal The Problem (Restated) The Question: How can we squash (encode) N-dimensional performance data onto a 2-dimensional display? In the following slides, I shall demonstrate how to pack at least N = 3 performance metrics into d = 2 dimensions whilst keeping the screen real-estate invariant (and relatively small). We can generalize from here to encode more dimensions, including time (through animation) [4]. c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 17 / 34
18. Barycentric View Revisiting the Goal My First Attempt (Circa 1992) NJG Develops Barry in 1992 My X11 desktop c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 18 / 34
19. Barycentric View Some Facts About Triangles The Centroid d “center of gravity” is 1/3rd height of the ! (h) entroid is at 1/3rd length of each bisector (b and c) a = h and also know b = c = a + c = h (sum rule) B C A h a b c P Centroid (P) is center of gravity at 1/3rd height (h) By symmetry, centroid is at 1/3rd length of each bisector (b and c) Therefore: since b = c = a @ P, then a + a + a = h Suggests: a + b + c = h (sum rule) even if a = b = c c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 19 / 34
20. Barycentric View Some Facts About Triangles The Barycenter is moved away from centroid b + c = h still holds point inside the ! as a convenient normalization that sum to 1 can mapped to this coordinate system c Point B C A h=1 a b c P Move P away from centroid Sum rule: a + b + c = h still holds Imagine non-uniform plate with more mass (heavier) on AB side P is still the center of mass or balance point Choose h = 1 as a convenient normalization Any 3 metrics that sum to 1, will map to this coordinate system Hence, “Barry-3” c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 20 / 34
21. Applications Outline 1 The Problem Performance Visualization Defined What Makes PerfViz Hard? 2 Role Models 1960 to 1980 1980 to 2000 Century 21 3 Barycentric View Revisiting the Goal Some Facts About Triangles 4 Applications Multiprocessor Visualization Network Visualization Response Time Visualization ORACLE Visualization 5 Postscript c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 21 / 34
22. Applications Multiprocessor Visualization Barry3 for Multiprocessors peripheral vision • Updated periodically for dynamic clustering cues Display many CPUs at once Visual area independent of CPU count Easy on the eyes, trigger off peripheral vision Updated periodically (animation ⇒ t-development) Dynamic cues e.g., clustering of points Demo: ORA %cpu vs time Demo: Barry3 coords c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 22 / 34
23. Applications Multiprocessor Visualization Adding More Metrics with Simplexes s Ø False, Boxed Ø FalseD Simplex ne = 880, 0, 0<, 80, 1, 0<<; ine = Line@v3LineD; Graphics3D@8Opacity@.5D, EdgeForm@BlackD, Thick, segLine, Sphere@v3Line@@1DD, rD, Sphere@v3Line@@2DD, rD< ê. r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD 3 - Simplex v3TriEq = 881, 0, 0<, 80, Sqrt@3D, 0<, 8-1, 0, 0<<; triEquilateral = 8Thick, Polygon@v3TriEqD<; Ballz.nb [348]:= Graphics3D@8Opacity@.5D, EdgeForm@BlackD, triEquilateral, Sphere@v3TriEq@@1DD, rD, Sphere@v3TriEq@@2DD, rD, Sphere@v3TriEq@@3DD, rD< ê. r Ø 1, Axes Ø False, Boxed Ø FalseD ut[348]= 4 - Simplex BarryBallz.nb 3 354]:= Graphics3D@8Opacity@.5D, EdgeForm@BlackD, PolyhedronData@"Tetrahedron", "Faces"D, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@1DD, rD, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@2DD, rD, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@3DD, rD, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@4DD, rD< ê. r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD ut[354]= BarryBallz.nb Point: d = 0 → 0-simplex Line: d = 1 → 1-simplex Triangle (equilateral): d = 2 → 2-simplex (Used to define Barry3) Tetrahedron: d = 3 → 3-simplex c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 23 / 34
24. Applications Network Visualization Barry4 for Network Performance Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@2DD, rD, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@3DD, rD, Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@4DD, rD< ê. r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD 354]= Networks use 4 metrics: unicast, multicast, broadcast, idle [4] Metrics obey the sum rule: u + m + b + i = 1 (100%) Must be a tetrahedron for Barry4 Example shows 1000 network segments (points) Cluster into “clouds” of points Allows the analyst to swivel their viewpoint (Tukey [3]) Animated demo: http://www.perfdynamics. com/Tools/tools.html c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 24 / 34
25. Applications Network Visualization How About More Metrics? Displaying 5 metrics would require a 4-simplex. Theorem There is no 4-simplex in 3-dimensions. Proof. owB raphics3DB:Opacity@.5D, EdgeForm@BlackD, PolyhedronData@"Tetrahedron", "Faces"D, Sphere@ thVertex@@1DD, rD, Sphere@thVertex@@2DD, rD, Sphere@thVertex@@3DD, rD, PointSize@0.05D, PointB::0, 0, - 2 3 - 1 2 6 >, :0, 0, 2 3 - 1 2 6 >>F, Sphere@thVertex@@4DD, rD> ê. r Ø 1 ê 2F, raphics3D@8Dashed, Thickness@0.005D, Line@8invTetra@@1DD, invTetra@@2DD<D, Line@8invTetra@@1DD, invTetra@@3DD<D, Line@8invTetra@@1DD, invTetra@@4DD<D<D, xes Ø False, Boxed Ø False, AxesLabel Ø 8"x", "y", "z"< BarryzBallzMario.nb 3 Add another sphere (below the others) Creates a new vertex Like 2 tetrahedra “glued” together But now 6 independent faces (not 5) Limbs to arbitrary point violate sum rule c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 25 / 34
26. Applications Response Time Visualization Apdex Metric Application responsiveness measured from user perspective and expressed for management as a single metric (AT ). Specification at http://www.apdex.org/ ©2005, Apdex Alliance, all rights reserved. Slide 14 Tolerating Apdex T = Total samples Satisfied 2 + Frustrated Satisfied Tolerating Good Fair Poor Unacceptable 0.00 T 0.50 T 1.00 T 0.85 T 0.94 T 0.70 T Excellent Report Group: Application User Group Time Period Existing Task Response Time Measurement Samples T 1 2 3 4 5 6 F ! " Define T for the application T = the application target time (threshold between satisfied and tolerating users). F = threshold between tolerating and frustrated users is calculated (F = 4T). #" Define a Report Group (details available are tool dependent). $" Extract data set from existing measurements for Report Group. % " Count the number of samples in three performance zones. &" Calculate the Apdex formula. '" Display Apdex result (T is always shown as part of the result). AT = S + 1 2 T S + T + F = S C + 1 2 T C = s + t/2 Measurement → Metrification → Ranking c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 26 / 34
27. Applications Response Time Visualization Representing AT in Barry3 metrics that sum to 1 can mapped to Barry-3 system ex categories: s + t + f = 1 (height) ows {s,t,f} range from each side (min=0) to opp. interior angle (max=1) ion ’t know the numerical value of A T g A T in Barry-3 t f s s t f A T Minimal Frustration (f = 0) Maximal Satisfaction Maximal Frustration (f > 0) Minimal Satisfaction Apdex categories: s + t + f = 1 (height) Arrows s, t, f range from each side (min=0) to opp. interior angle (max=1) But we can’t see the value of AT (yet) ... c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 27 / 34
28. Applications Response Time Visualization Including the Apdex Zones ning A T with Zones Minimal Satisfaction A=0.50 A=0.75 A=0.50 A=0.75 Maximal Satisfaction A=1.00 A=1.00 A=0.00 t f s Minimal Frustration (f = 0) Maximal Frustration (f > 0) s t f A T sually estimate the value A T from the Zone boundaries AT zones are diagonal bands in Barry3 Zone boundaries are lines of constant AT (isoclines) Zones are actually independent of Barry-3 coordinates Animated demo: http://www. perfdynamics.com/ Tools/tools.html c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 28 / 34
29. Applications ORACLE Visualization What About Oracle? Remark Whichever ORA metrics you choose, they must add up to 100% for a Barry3 or Barry4 representation. c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 29 / 34
30. Applications ORACLE Visualization Conclusion Of course, there are many other kinds of visualization paradigms besides the barycentric system that I’ve shown you today [4, 5, 6]. Remember ... Vendors avoid investing in development if they don’t see a demand, whilst performance analysts and capacity planners don’t demand what they have not conceived. Hopefully, this talk has helped you to conceive of what Oracle Corporation could be providing for performance visualization. If you have some ideas, consider presenting them at CMG 2008 in the PerfViz track [6] (I’m the Session Chair ). c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 30 / 34
31. Postscript Outline 1 The Problem Performance Visualization Defined What Makes PerfViz Hard? 2 Role Models 1960 to 1980 1980 to 2000 Century 21 3 Barycentric View Revisiting the Goal Some Facts About Triangles 4 Applications Multiprocessor Visualization Network Visualization Response Time Visualization ORACLE Visualization 5 Postscript c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 31 / 34
32. Postscript Postscript Cary required that we all try to “inspire someone” at Hotsos08. In fact, several people came up to me after I gave this presentation with suggestions for applying PerfViz concepts to Oracle data. Two that seemed very noteworthy were: Tanel P˜ oder (Independent) has ideas about applying a Barry3 or Barry4 representation to data from the Oracle Wait interface. Marco Gralike (Amis, NL) is interested in finding performance visualizations for document transformations between SQL and XML. If we’re lucky, you might see some of these developments at Hotsos 2009. c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 32 / 34
33. Postscript References I R. Harris, Information Graphics: A Comprehensive Illustrated Reference, Oxford. Univ. Press, 1999 E. R. Tufte, The Visual Display of Quantitative Information, Graphics Press, 1983 J. W. Tukey, Graphics 1965–1985, In W. S. Cleveland, editor, The Collected Works of John W. Tukey, Vol. V, Wadsworth & Brooks/Cole, Pacific Grove, California, 1988 N. J. Gunther and M. F. Jauvin, “Seeing It All at Once with Barry,” CMG Intl. Conf., San Diego, California, Dec. 2007 (http://www.perfdynamics.com/Papers/barry007.pdf) P. McMahon and J. A. Martin. “Death To Dashboards And Other Thoughts On Data Visualization,” CMG Intl. Conf., San Diego, California, Dec. 2007 c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 33 / 34
34. Postscript References II Performance Agora, “CMG 2008: Call for PerfViz Papers,” (http://perfdynamics.blogspot.com/2007/12/ cmg-2008-call-for-perfviz-papers.html) F. Werblin and B. Roska, “The Movies in Our Eyes,” Scientific American, April 2007 (http://www.sciamdigital.com/index.cfm?fa=Products. ViewIssuePreview&ARTICLEID CHAR= 335A7BD5-2B35-221B-68F3D42B56283016) J. K. Keller, “Volumetric Redundancies,” (http://www.c71123.com/visualizations/more-visualizations/) Digg Labs, Swarm (for Digg blogs), (http://labs.digg.com/swarm/) J. Hawkins and S. Blakeslee, On Intelligence, Owl Books, 2004 c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 34 / 34