Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Walls, Pillars and Beams: A 3D Decomposition of Quality Anomalies (vissoft2016)

Yuriy Tymchuk
October 04, 2016

Walls, Pillars and Beams: A 3D Decomposition of Quality Anomalies (vissoft2016)

Quality rules are used to capture important implementation and design decisions embedded in a software system’s architecture. They can automatically analyze software and assign quality grades to its components. To provide a meaningful evaluation of quality, rules have to stay up-to-date with the continuously evolving system that they describe. However one would encounter unexpected anomalies during a historical overview because the notion of quality is always changing, while the qualitative evolution analysis requires it to remain constant.
To understand the anomalies in a quality history of a real-world software system we use an immersive visualization that lays out the quality fluctuations in three dimensions based on two co-evolving properties: quality rules and source code. This helps us to identify and separate the impact caused by the changes of each property, and allows us to detect significant mistakes that happened during the development process.

Yuriy Tymchuk

October 04, 2016
Tweet

More Decks by Yuriy Tymchuk

Other Decks in Research

Transcript

  1. Walls, Pillars and Beams:
    A 3D Decomposition of Quality Anomalies
    , Leonel Merino, Mohammad Ghafari and Oscar Nierstrasz
    Software Composition Group @ University of Bern
    Yuriy Tymchuk

    View full-size slide

  2. Walls, Pillars and Beams:
    A 3D Decomposition of Quality Anomalies
    , Leonel Merino, Mohammad Ghafari and Oscar Nierstrasz
    Software Composition Group @ University of Bern
    @yuriy_tymchuk

    View full-size slide

  3. An intensive conference…

    View full-size slide

  4. Time to Relax!

    View full-size slide

  5. Height
    Time
    Analyzing the Process of Palmgrowing

    View full-size slide

  6. Height
    Time
    Analyzing the Process of Palmgrowing

    View full-size slide

  7. Height
    Analyzing the Process of Palmgrowing
    Time

    View full-size slide

  8. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  9. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  10. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  11. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  12. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  13. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  14. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  15. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  16. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  17. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  18. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  19. Analyzing the Process of Palmgrowing
    Height
    Time

    View full-size slide

  20. 62.5
    125
    187.5
    Analyzing the Process of
    Height
    Time

    View full-size slide

  21. Pharo 5
    Height
    Time
    62.5
    125
    187.5

    View full-size slide

  22. Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    Height
    Time

    View full-size slide

  23. Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    Height

    View full-size slide

  24. Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates*
    CI & Live
    Height

    View full-size slide

  25. Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates*
    Tricorder: Building a Program Analysis Ecosystem. C. Sadowski, J. Gogh, C. Jaspan, E. Soederberg, C. Winter. ICSE’15, pp. 598–608, IEEE, 2015
    When QualityAssistant Meets Pharo. Y. Tymchuk , M. Ghafari, O. Nierstrasz. IWST’16, to-be-published, ACM, 2016
    *
    CI & Live
    Height

    View full-size slide

  26. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    isNil ifTrue: ifNil:
    car isNil ifTrue: [
    ^ car := Car new ]
    CI & Live
    *

    View full-size slide

  27. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    isNil ifTrue: ifNil:
    car isNil ifTrue: [
    ^ car := Car new ]
    CI & Live
    *

    View full-size slide

  28. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    car isNil ifTrue: [
    ^ car := Car new ]
    CI & Live
    isNil ifTrue: ifNil:

    View full-size slide

  29. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    car isNil ifTrue: [
    ^ car := Car new ]
    Critique
    CI & Live
    isNil ifTrue: ifNil:

    View full-size slide

  30. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    CI & Live
    isNil ifTrue: ifNil:
    Critiques
    car isNil ifTrue: [
    ^ car := Car new ]

    View full-size slide

  31. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    Time
    CI & Live
    isNil ifTrue: ifNil:
    car isNil ifTrue: [
    ^ car := Car new ]
    Critiques

    View full-size slide

  32. Height
    Time
    Pharo 5
    240 packages
    5.5K classes
    92K methods
    680 updates
    SmallLint
    8 groups
    124 rules
    250 updates
    Time
    CI & Live
    isNil ifTrue: ifNil:
    car isNil ifTrue: [
    ^ car := Car new ]
    Critiques

    View full-size slide

  33. Pharo 5 SmallLint Time
    Critiques
    Height
    Time
    Palm - Pharo map

    View full-size slide

  34. Analyzing the Process of Pharoevolution

    View full-size slide

  35. 680
    Versions
    Critiques
    Analyzing the Process of Pharoevolution

    View full-size slide

  36. 40K
    30K
    20K
    10K
    680
    Versions
    Critiques
    0
    Analyzing the Process of Pharoevolution

    View full-size slide

  37. 40K
    30K
    20K
    10K
    680
    Versions
    Critiques
    0
    Analyzing the Process of Pharoevolution
    Number of Critiques Σ=19.5M

    View full-size slide

  38. 68
    Versions
    Analyzing the Process of Pharoevolution
    Number of Critiques Σ=19.5M Δ=64.5K

    View full-size slide

  39. Bad Pharoevolution

    View full-size slide

  40. Good Pharoevolution

    View full-size slide

  41. 5K Rule #xed

    View full-size slide

  42. 5K Rule #xed
    2.5K Rule removed

    View full-size slide

  43. 5K Rule #xed
    3.5K Rule added
    2.5K Rule removed

    View full-size slide

  44. 0.8K Code added
    5K Rule #xed
    3.5K Rule added
    2.5K Rule removed

    View full-size slide

  45. 5K Rule #xed
    3.5K Rule added
    1K Violation solved
    2.5K Rule removed
    0.8K Code added

    View full-size slide

  46. Why did this irregularity occur?
    What are the irregularities in the system’s evolution?

    View full-size slide

  47. A picture is worth
    a thousand words


    English idiom

    View full-size slide

  48. Every block of stone
    has a statue inside it
    and it is the task of the
    sculptor to discover it


    Michelangelo

    View full-size slide

  49. Every piece of data has a visualization
    that tells a story about it and it is the task
    of the visualization builder to discover it


    View full-size slide

  50. Time (Versions)
    Metrics (Rules)

    View full-size slide

  51. Time (Versions)
    Metrics (Rules)

    View full-size slide

  52. Time (Versions)
    Metrics (Rules)

    View full-size slide

  53. Time (Versions)
    Metrics (Rules)

    View full-size slide

  54. Time (Versions)
    Metrics (Rules)

    View full-size slide

  55. Time (Versions)
    Metrics (Rules)

    View full-size slide

  56. Time (Versions)
    Metrics (Rules)

    View full-size slide

  57. Time (Versions)
    Metrics (Rules)

    View full-size slide

  58. Time (Versions)
    Metrics (Rules)
    Code (Packages)

    View full-size slide

  59. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  60. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  61. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  62. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  63. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  64. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  65. e (Packages)
    Time (Versions)

    View full-size slide

  66. e (Packages)
    Time (Versions)

    View full-size slide

  67. e (Packages)
    Time (Versions)

    View full-size slide

  68. Code (Packages)
    Time (Versions)
    Metrics (Rules)

    View full-size slide

  69. After about 1 working day…

    View full-size slide

  70. Packages
    Versions
    Rules

    View full-size slide

  71. What are the irregularities in the system’s evolution?

    View full-size slide

  72. What are the irregularities in the system’s evolution?
    “Anti-Matrix”

    View full-size slide

  73. Packages
    Versions
    Rules

    View full-size slide

  74. Packages
    Versions
    Rules

    View full-size slide

  75. Packages
    Versions
    Rules

    View full-size slide

  76. Packages
    Versions
    Rules

    View full-size slide

  77. Packages
    Versions
    Rules

    View full-size slide

  78. Packages
    Versions
    Rules

    View full-size slide

  79. Color Code
    Critiques added
    Critiques removed

    View full-size slide

  80. Color Code
    No change
    Critiques added
    Critiques removed
    High change
    High change

    View full-size slide

  81. Why did this irregularity occur?

    View full-size slide

  82. Yes, that’s exactly what we did!

    View full-size slide

  83. Meet the Anomalies

    View full-size slide

  84. Wall
    Something Went Completely Wrong

    View full-size slide

  85. Beam
    Added/Removed Rule
    Fixed/Broken Rule
    Violated Single Rule
    Resolved Critiques

    View full-size slide

  86. Beams on a Rule-Pillar

    View full-size slide

  87. Beams on a Rule-Pillar
    Refactoring?

    View full-size slide

  88. Beams on a Rule-Pillar
    Refactoring?
    1 Non-Related Beam

    View full-size slide

  89. Beams on a Rule-Pillar
    Refactoring?
    1 Non-Related Beam
    2 Related Beams

    View full-size slide

  90. Beams on a Rule-Pillar
    Refactoring?
    1 Non-Related Beam
    1 Rule broken & Fixed
    2 Related Beams

    View full-size slide

  91. Beams on a Rule-Pillar
    Refactoring?
    1 Non-Related Beam
    1 Rule broken & Fixed
    2 Related Beams

    View full-size slide

  92. Pillar
    Package Addition/Removal

    View full-size slide

  93. Pillar
    Package Addition/Removal

    View full-size slide

  94. Rules on Package-Beam

    View full-size slide

  95. Rules on Package-Beam

    View full-size slide

  96. Waldo
    ±2000 critiques

    View full-size slide

  97. Waldo
    Wall Pillar Beam

    View full-size slide

  98. Initial Visualization

    View full-size slide

  99. Final Visualization

    View full-size slide

  100. Final Visualization

    View full-size slide

  101. Final Visualization
    Walls 3×2

    View full-size slide

  102. Final Visualization
    Walls
    added/removed
    !xed/broken
    other (non-rule)
    Beams/Rules
    3×2
    8
    7
    17

    View full-size slide

  103. Final Visualization
    Walls
    added/removed
    !xed/broken
    other (non-rule)
    added/removed
    modi!ed
    Beams/Rules
    Pillars/Packages
    3×2
    8
    17
    7
    42
    3

    View full-size slide

  104. The Original Bar Chart

    View full-size slide

  105. The Original Bar Chart
    Cleansed

    View full-size slide