Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Better Visualization Tools for Better Performance Management

Better Visualization Tools for Better Performance Management

Updated version of the presentation given at the annual Hotsos Symposium, March 5, 2008

Dr. Neil Gunther

March 05, 2008
Tweet

More Decks by Dr. Neil Gunther

Other Decks in Research

Transcript

  1. Better Visualization Tools
    for Better Performance Management
    Dr. Neil J. Gunther
    Performance Dynamics
    Hotsos Symposium. March 5, 2008 (updated)
    www.perfdynamics.com
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 1 / 34

    View Slide

  2. The Problem
    Outline
    1 The Problem
    Performance Visualization Defined
    What Makes PerfViz Hard?
    2 Role Models
    1960 to 1980
    1980 to 2000
    Century 21
    3 Barycentric View
    Revisiting the Goal
    Some Facts About Triangles
    4 Applications
    Multiprocessor Visualization
    Network Visualization
    Response Time Visualization
    ORACLE Visualization
    5 Postscript
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 2 / 34

    View Slide

  3. The Problem
    The Business Problem
    “Improving data visualization paradigms for performance management
    is an orphaned area of performance tool development [4]...” (because):
    1 Tool vendors avoid investing in development if they don’t see any
    demand
    2 Performance analysts and capacity planners don’t demand what
    they have not conceived
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 3 / 34

    View Slide

  4. The Problem
    The Technical Problem
    17
    Cognitive Impedance Matching
    Z
    L
    Z
    S Z
    L
    Z
    S
    Digital computer Cognitive computer
    Find the best cognitive impedence match: ZSrc
    ZLoad
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 4 / 34

    View Slide

  5. The Problem Performance Visualization Defined
    Performance Visualization
    The Question:
    Is it possible to present performance data collected from modern
    complex computing environments in a way similar to scientific
    visualization as applied to complex physical data?
    We’ll call this goal “PerfViz” for short
    Get beyond data reporting [1, 2] to data exploration [3]
    Some PerfViz tools already exists, especially for HPC
    Can we use them?
    Can we do better?
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 5 / 34

    View Slide

  6. The Problem What Makes PerfViz Hard?
    Best Cognitive Impedence Match
    Sounds simple.
    What is cognition?
    How does the brain
    work? (See [10])
    What is vision?
    If we understood our neural circuitry, we could choose the best
    visualization paradigms.
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 6 / 34

    View Slide

  7. The Problem What Makes PerfViz Hard?
    Neural Circuitry
    All mammals have a neocotex.
    Mapped complete circuitry for the Makak monkey.
    Hierarchy of Cortical Regions
    • Hierarchy e
    mammal ne
    • High variab
    between sp
    (relatively fixed
    • Relative siz
    can vary by
    between ind
    a species
    From: Felleman and Van Essen
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 7 / 34

    View Slide

  8. The Problem What Makes PerfViz Hard?
    Human Neural Circuits
    We know more about a monkey brain than a human brain.
    We also know it can be easily fooled.
    “I am NOT an animal!”
    (He’s actually a performance analyst.)
    Good Z match ⇒ “So simple, a caveman could do it!”
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 8 / 34

    View Slide

  9. The Problem What Makes PerfViz Hard?
    Scientific Visualization
    Physical data is generally (3 + 1)-dimensional
    3-space and 1-time dimension
    Weather forecasting
    Tornado simulations (Navier-Stokes equation)
    Oil exploration
    CAT scans (volume tomography)
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 9 / 34

    View Slide

  10. The Problem What Makes PerfViz Hard?
    Demo: Tornadoes and Cyclones
    Tornado simulation
    Navier-Stokes equation
    Text structure as a cyclone [8]
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 10 / 34

    View Slide

  11. The Problem What Makes PerfViz Hard?
    Performance Visualization
    Performance data is N-dimensional (and N is large)
    Solaris captures about N = 300 UNIX system metrics
    Oracle 10g v$-tables contain about N = 200 metrics
    Pick N = 2: your screen is only 2-dimensional
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 11 / 34

    View Slide

  12. Role Models
    Outline
    1 The Problem
    Performance Visualization Defined
    What Makes PerfViz Hard?
    2 Role Models
    1960 to 1980
    1980 to 2000
    Century 21
    3 Barycentric View
    Revisiting the Goal
    Some Facts About Triangles
    4 Applications
    Multiprocessor Visualization
    Network Visualization
    Response Time Visualization
    ORACLE Visualization
    5 Postscript
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 12 / 34

    View Slide

  13. Role Models 1960 to 1980
    The Swinging 60’s and 70’s
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 13 / 34

    View Slide

  14. Role Models 1980 to 2000
    The Roaring 80’s and 90’s
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 14 / 34

    View Slide

  15. Role Models Century 21
    21st Century InfoViz
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 15 / 34

    View Slide

  16. Barycentric View
    Outline
    1 The Problem
    Performance Visualization Defined
    What Makes PerfViz Hard?
    2 Role Models
    1960 to 1980
    1980 to 2000
    Century 21
    3 Barycentric View
    Revisiting the Goal
    Some Facts About Triangles
    4 Applications
    Multiprocessor Visualization
    Network Visualization
    Response Time Visualization
    ORACLE Visualization
    5 Postscript
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 16 / 34

    View Slide

  17. Barycentric View Revisiting the Goal
    The Problem (Restated)
    The Question:
    How can we squash (encode) N-dimensional performance data onto a
    2-dimensional display?
    In the following slides, I shall demonstrate how to pack at least N = 3
    performance metrics into d = 2 dimensions whilst keeping the screen
    real-estate invariant (and relatively small).
    We can generalize from here to encode more dimensions, including
    time (through animation) [4].
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 17 / 34

    View Slide

  18. Barycentric View Revisiting the Goal
    My First Attempt (Circa 1992)
    NJG Develops Barry in 1992
    My X11 desktop
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 18 / 34

    View Slide

  19. Barycentric View Some Facts About Triangles
    The Centroid
    d
    “center of gravity” is 1/3rd height of the ! (h)
    entroid is at 1/3rd length of each bisector (b and c)
    a = h and also know b = c = a
    + c = h (sum rule)
    B C
    A
    h
    a
    b
    c
    P
    Centroid (P) is center of gravity at
    1/3rd height (h)
    By symmetry, centroid is at 1/3rd
    length of each bisector (b and c)
    Therefore: since b = c = a @ P,
    then a + a + a = h
    Suggests: a + b + c = h (sum rule)
    even if a = b = c
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 19 / 34

    View Slide

  20. Barycentric View Some Facts About Triangles
    The Barycenter
    is moved away from centroid
    b + c = h still holds
    point inside the !
    as a convenient normalization
    that sum to 1 can mapped to this coordinate system
    c Point
    B C
    A
    h=1
    a
    b
    c
    P
    Move P away from centroid
    Sum rule: a + b + c = h still holds
    Imagine non-uniform plate with
    more mass (heavier) on AB side
    P is still the center of mass or
    balance point
    Choose h = 1 as a convenient
    normalization
    Any 3 metrics that sum to 1, will
    map to this coordinate system
    Hence, “Barry-3”
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 20 / 34

    View Slide

  21. Applications
    Outline
    1 The Problem
    Performance Visualization Defined
    What Makes PerfViz Hard?
    2 Role Models
    1960 to 1980
    1980 to 2000
    Century 21
    3 Barycentric View
    Revisiting the Goal
    Some Facts About Triangles
    4 Applications
    Multiprocessor Visualization
    Network Visualization
    Response Time Visualization
    ORACLE Visualization
    5 Postscript
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 21 / 34

    View Slide

  22. Applications Multiprocessor Visualization
    Barry3 for Multiprocessors
    peripheral vision
    • Updated periodically for
    dynamic clustering cues
    Display many CPUs at once
    Visual area independent of CPU
    count
    Easy on the eyes, trigger off
    peripheral vision
    Updated periodically (animation
    ⇒ t-development)
    Dynamic cues e.g., clustering of
    points
    Demo: ORA %cpu vs time
    Demo: Barry3 coords
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 22 / 34

    View Slide

  23. Applications Multiprocessor Visualization
    Adding More Metrics with Simplexes
    s Ø False, Boxed Ø FalseD
    Simplex
    ne = 880, 0, 0<, 80, 1, 0<<;
    ine = Line@v3LineD;
    Graphics3D@[email protected], EdgeForm@BlackD, Thick, segLine,
    Sphere@v3Line@@1DD, rD, Sphere@v3Line@@2DD, rD< ê. r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD
    3 - Simplex
    v3TriEq = 881, 0, 0<, 80, Sqrt@3D, 0<, 8-1, 0, 0<<;
    triEquilateral = 8Thick, Polygon@v3TriEqD<;
    Ballz.nb
    [348]:= Graphics3D@8Opacit[email protected], EdgeForm@BlackD, triEquilateral, Sphere@v3TriEq@@1DD, rD,
    Sphere@v3TriEq@@2DD, rD, Sphere@v3TriEq@@3DD, rD< ê. r Ø 1, Axes Ø False, Boxed Ø FalseD
    ut[348]=
    4 - Simplex
    BarryBallz.nb 3
    354]:= Graphics3D@8Opaci[email protected], EdgeForm@BlackD, PolyhedronData@"Tetrahedron", "Faces"D,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@1DD, rD,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@2DD, rD,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@3DD, rD,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@4DD, rD< ê.
    r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD
    ut[354]=
    BarryBallz.nb
    Point: d = 0 → 0-simplex
    Line: d = 1 → 1-simplex
    Triangle (equilateral): d = 2 → 2-simplex
    (Used to define Barry3)
    Tetrahedron: d = 3 → 3-simplex
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 23 / 34

    View Slide

  24. Applications Network Visualization
    Barry4 for Network Performance
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@2DD, rD,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@3DD, rD,
    Sphere@PolyhedronData@"Tetrahedron", "VertexCoordinates"D@@4DD, rD< ê.
    r Ø 1 ê 2, Axes Ø False, Boxed Ø FalseD
    354]=
    Networks use 4 metrics: unicast,
    multicast, broadcast, idle [4]
    Metrics obey the sum rule:
    u + m + b + i = 1 (100%)
    Must be a tetrahedron for Barry4
    Example shows 1000 network
    segments (points)
    Cluster into “clouds” of points
    Allows the analyst to swivel their
    viewpoint (Tukey [3])
    Animated demo:
    http://www.perfdynamics.
    com/Tools/tools.html
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 24 / 34

    View Slide

  25. Applications Network Visualization
    How About More Metrics?
    Displaying 5 metrics would require a 4-simplex.
    Theorem
    There is no 4-simplex in 3-dimensions.
    Proof.
    owB
    raphics3DB:[email protected], EdgeForm@BlackD, PolyhedronData@"Tetrahedron", "Faces"D, Sphere@
    thVertex@@1DD, rD, Sphere@thVertex@@2DD, rD, Sphere@thVertex@@3DD, rD, [email protected],
    PointB::0, 0, -
    2
    3
    -
    1
    2 6
    >, :0, 0,
    2
    3
    -
    1
    2 6
    >>F, Sphere@thVertex@@4DD, rD> ê. r Ø 1 ê 2F,
    raphics3D@8Dashed, [email protected], Line@8invTetra@@1DD, invTetra@@2DDLine@8invTetra@@1DD, invTetra@@3DDxes Ø False, Boxed Ø False, AxesLabel Ø 8"x", "y", "z"<
    BarryzBallzMario.nb 3
    Add another sphere (below the others)
    Creates a new vertex
    Like 2 tetrahedra “glued” together
    But now 6 independent faces (not 5)
    Limbs to arbitrary point violate sum rule
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 25 / 34

    View Slide

  26. Applications Response Time Visualization
    Apdex Metric
    Application responsiveness measured from user perspective and
    expressed for management as a single metric (AT
    ). Specification at
    http://www.apdex.org/
    ©2005, Apdex Alliance, all rights reserved. Slide 14
    Tolerating
    Apdex
    T
    =
    Total samples
    Satisfied
    2
    +
    Frustrated
    Satisfied
    Tolerating
    Good
    Fair
    Poor
    Unacceptable
    0.00
    T
    0.50
    T
    1.00
    T
    0.85
    T
    0.94
    T
    0.70
    T
    Excellent
    Report Group:
    Application
    User Group
    Time Period
    Existing Task
    Response Time
    Measurement
    Samples
    T
    1
    2
    3
    4
    5
    6
    F
    !
    "
    Define T for the application
    T = the application target time (threshold between satisfied and tolerating users).
    F = threshold between tolerating and frustrated users is calculated (F = 4T).
    #"
    Define a Report Group (details available are tool dependent).
    $"
    Extract data set from existing measurements for Report Group.
    %
    "
    Count the number of samples in three performance zones.
    &"
    Calculate the Apdex formula.
    '"
    Display Apdex result (T is always shown as part of the result).
    AT =
    S + 1
    2
    T
    S + T + F
    =
    S
    C
    +
    1
    2
    T
    C
    = s + t/2
    Measurement → Metrification → Ranking
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 26 / 34

    View Slide

  27. Applications Response Time Visualization
    Representing AT
    in Barry3
    metrics that sum to 1 can mapped to Barry-3 system
    ex categories: s + t + f = 1 (height)
    ows {s,t,f} range from each side (min=0) to opp. interior angle (max=1)
    ion
    ’t know the numerical value of A
    T
    g A
    T
    in Barry-3
    t f
    s
    s
    t
    f
    A
    T
    Minimal Frustration (f = 0)
    Maximal Satisfaction
    Maximal Frustration (f > 0)
    Minimal Satisfaction
    Apdex categories:
    s + t + f = 1 (height)
    Arrows s, t, f range
    from each side (min=0)
    to opp. interior angle
    (max=1)
    But we can’t see the
    value of AT
    (yet) ...
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 27 / 34

    View Slide

  28. Applications Response Time Visualization
    Including the Apdex Zones
    ning A
    T
    with Zones
    Minimal Satisfaction
    A=0.50
    A=0.75
    A=0.50
    A=0.75
    Maximal Satisfaction
    A=1.00 A=1.00
    A=0.00
    t f
    s
    Minimal Frustration (f = 0)
    Maximal Frustration (f > 0)
    s
    t
    f
    A
    T
    sually estimate the value A
    T
    from the Zone boundaries
    AT
    zones are diagonal
    bands in Barry3
    Zone boundaries are
    lines of constant AT
    (isoclines)
    Zones are actually
    independent of Barry-3
    coordinates
    Animated demo:
    http://www.
    perfdynamics.com/
    Tools/tools.html
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 28 / 34

    View Slide

  29. Applications ORACLE Visualization
    What About Oracle?
    Remark
    Whichever ORA metrics you choose, they must add up to 100% for a Barry3
    or Barry4 representation.
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 29 / 34

    View Slide

  30. Applications ORACLE Visualization
    Conclusion
    Of course, there are many other kinds of visualization paradigms
    besides the barycentric system that I’ve shown you today [4, 5, 6].
    Remember ...
    Vendors avoid investing in development if they don’t see a demand,
    whilst performance analysts and capacity planners don’t demand what
    they have not conceived.
    Hopefully, this talk has helped you to conceive of what Oracle
    Corporation could be providing for performance visualization.
    If you have some ideas, consider presenting them at CMG 2008 in the
    PerfViz track [6] (I’m the Session Chair ).
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 30 / 34

    View Slide

  31. Postscript
    Outline
    1 The Problem
    Performance Visualization Defined
    What Makes PerfViz Hard?
    2 Role Models
    1960 to 1980
    1980 to 2000
    Century 21
    3 Barycentric View
    Revisiting the Goal
    Some Facts About Triangles
    4 Applications
    Multiprocessor Visualization
    Network Visualization
    Response Time Visualization
    ORACLE Visualization
    5 Postscript
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 31 / 34

    View Slide

  32. Postscript
    Postscript
    Cary required that we all try to “inspire someone” at Hotsos08. In fact,
    several people came up to me after I gave this presentation with
    suggestions for applying PerfViz concepts to Oracle data.
    Two that seemed very noteworthy were:
    Tanel P˜
    oder (Independent) has ideas about applying a Barry3 or
    Barry4 representation to data from the Oracle Wait interface.
    Marco Gralike (Amis, NL) is interested in finding performance
    visualizations for document transformations between SQL and
    XML.
    If we’re lucky, you might see some of these developments at Hotsos
    2009.
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 32 / 34

    View Slide

  33. Postscript
    References I
    R. Harris,
    Information Graphics: A Comprehensive Illustrated Reference,
    Oxford. Univ. Press, 1999
    E. R. Tufte,
    The Visual Display of Quantitative Information, Graphics Press, 1983
    J. W. Tukey,
    Graphics 1965–1985,
    In W. S. Cleveland, editor, The Collected Works of John W. Tukey, Vol. V, Wadsworth &
    Brooks/Cole, Pacific Grove, California, 1988
    N. J. Gunther and M. F. Jauvin,
    “Seeing It All at Once with Barry,”
    CMG Intl. Conf., San Diego, California, Dec. 2007
    (http://www.perfdynamics.com/Papers/barry007.pdf)
    P. McMahon and J. A. Martin.
    “Death To Dashboards And Other Thoughts On Data Visualization,”
    CMG Intl. Conf., San Diego, California, Dec. 2007
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 33 / 34

    View Slide

  34. Postscript
    References II
    Performance Agora,
    “CMG 2008: Call for PerfViz Papers,”
    (http://perfdynamics.blogspot.com/2007/12/
    cmg-2008-call-for-perfviz-papers.html)
    F. Werblin and B. Roska,
    “The Movies in Our Eyes,”
    Scientific American, April 2007
    (http://www.sciamdigital.com/index.cfm?fa=Products.
    ViewIssuePreview&ARTICLEID CHAR=
    335A7BD5-2B35-221B-68F3D42B56283016)
    J. K. Keller,
    “Volumetric Redundancies,”
    (http://www.c71123.com/visualizations/more-visualizations/)
    Digg Labs,
    Swarm (for Digg blogs),
    (http://labs.digg.com/swarm/)
    J. Hawkins and S. Blakeslee,
    On Intelligence,
    Owl Books, 2004
    c 2008 Performance Dynamics Better Visualization Tools April 1, 2008 34 / 34

    View Slide