Upgrade to Pro — share decks privately, control downloads, hide ads and more …

On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support

On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support

Pooyan Jamshidi

May 26, 2022
Tweet

More Decks by Pooyan Jamshidi

Other Decks in Research

Transcript

  1. On Debugging the Performance of
    Configurable Software Systems
    Miguel Velez
    (CMU -> Google)
    Norbert Siegmund
    (Leipzig)
    Pooyan Jamshidi
    (UofSC & Google)
    Sven Apel
    (Saarland)
    Christian Kästner
    (CMU)
    Developer Needs and Tailored Tool Support

    View full-size slide

  2. Most software is
    con
    fi
    gurable

    View full-size slide

  3. Indexing Encryption Compression Logging
    Core functionality
    Configurable System
    Need: Need: Need:

    View full-size slide

  4. Indexing Encryption Compression Logging
    Core functionality
    Configurable System

    View full-size slide

  5. 6
    Performance Execution time
    Energy consumption Operational costs

    View full-size slide

  6. Developer
    • Design, implement, and maintain efficient software
    • Debug surprising performance behaviors

    View full-size slide

  7. 8
    59% - Con
    fi
    guration-related performance issues
    61% - Average
    fi
    x time of 5 weeks
    50% - Con
    fi
    guration-related patches

    View full-size slide

  8. Traditional Off-the-Shelf Profilers

    View full-size slide

  9. Searching Inefficient Coding Patterns

    View full-size slide

  10. Program Debugging Techniques
    12

    View full-size slide

  11. 14
    Limited empirical evidence of the
    usefulness to help debug the
    performance of con
    fi
    gurable systems

    View full-size slide

  12. Developer
    • Understand how options affect performance
    • Design, implement, and maintain efficient software
    Help developers debug the performance
    of configurable software systems

    View full-size slide

  13. Help developers debug the performance
    of configurable software systems

    View full-size slide

  14. Identify Information Needs
    Influencing Options Option Hotspots Cause-Effect Chain
    Help developers debug the performance
    of configurable software systems

    View full-size slide

  15. Identify Information Needs
    Influencing Options Option Hotspots Cause-Effect Chain
    Help developers debug the performance
    of configurable software systems
    Tailor Ingredients
    CPU Profiling Program Slicing
    Performance Modeling
    Global Local

    View full-size slide

  16. 22
    User Study to Identify Information Needs

    View full-size slide

  17. 23
    User Study to Identify Information Needs
    Overview of Ingredients

    View full-size slide

  18. 24
    User Study to Identify Information Needs
    Overview of Ingredients
    Tailor and Evaluate Ingredients

    View full-size slide

  19. 25
    Overview of Ingredients
    Tailor and Evaluate Ingredients
    User Study to Identify Information Needs

    View full-size slide

  20. Information Needs
    RQ1: What information do developers look for when
    debugging the performance of con
    fi
    gurable systems?
    RQ2: What is the process that developers follow and the
    activities that they perform to obtain this information?

    View full-size slide

  21. Information Needs
    RQ1: What information do developers look for when
    debugging the performance of con
    fi
    gurable systems?
    RQ2: What is the process that developers follow and the
    activities that they perform to obtain this information?
    Exploratory User Study

    View full-size slide

  22. Exploratory User Study

    View full-size slide

  23. Exploratory User Study
    Influencing Options
    The option or interaction causing the unexpected
    performance behavior

    View full-size slide

  24. Exploratory User Study
    Influencing Options Option Hotspots
    The methods where options affect the performance of
    the system

    View full-size slide

  25. Exploratory User Study
    Influencing Options Option Hotspots Cause-Effect Chain
    How in
    fl
    uencing options are used in the implementation
    to directly and indirectly affect the performance of
    option hotspots

    View full-size slide

  26. Exploratory User Study
    Influencing Options Option Hotspots Cause-Effect Chain
    Compared problematic con
    fi
    guration to a
    non-problematic con
    fi
    guration

    View full-size slide

  27. User Study to Identify Information Needs
    Influencing Options Option Hotspots Cause-Effect Chain
    Help developers debug the performance
    of configurable software systems

    View full-size slide

  28. 35
    Tailor and Evaluate Ingredients
    User Study to Identify Information Needs
    Overview of Ingredients

    View full-size slide

  29. The option or interaction causing the unexpected
    performance behavior
    Influencing Options

    View full-size slide

  30. The option or interaction causing the unexpected
    performance behavior
    Influencing Options
    Global Performance-In
    fl
    uence Models

    View full-size slide


  31. 23 seconds
    28 seconds
    20 seconds
    Con
    fi
    gurations Performance
    T = 25 + 3· - 5· + 9· ·
    Performance-Influence Models

    View full-size slide


  32. 23 seconds
    28 seconds
    20 seconds
    Con
    fi
    gurations Performance
    T = 25 + 3· - 5· + 9· ·
    Performance-Influence Models

    View full-size slide


  33. 23 seconds
    28 seconds
    20 seconds
    Con
    fi
    gurations Performance
    T = 25 + 3· - 5· + 9· ·
    Performance-Influence Models

    View full-size slide

  34. Modeling
    Global
    T = 25 + 3· - 5· + 9· ·
    Influencing Options

    View full-size slide

  35. The methods where options affect the performance of
    the system
    Option Hotspots

    View full-size slide

  36. The methods where options affect the performance of
    the system
    Option Hotspots
    Local Performance-In
    fl
    uence Models

    View full-size slide

  37. Modeling
    Local
    Tfoo = 5 + 3· + 9· ·
    Option Hotspots

    View full-size slide

  38. How in
    fl
    uencing options are used in the implementation
    to directly and indirectly affect the performance of
    option hotspots
    Cause-Effect Chain

    View full-size slide

  39. How in
    fl
    uencing options are used in the implementation
    to directly and indirectly affect the performance of
    option hotspots
    Cause-Effect Chain
    CPU Pro
    fi
    ling & Program Slicing

    View full-size slide

  40. CPU Profiling
    Cause-Effect Chain

    View full-size slide

  41. Program Slicing
    Cause-Effect Chain

    View full-size slide

  42. Overview of Ingredients
    Influencing Options Option Hotspots Cause-Effect Chain
    Help developers debug the performance
    of configurable software systems
    CPU Profiling Program Slicing
    Modeling
    Global Local

    View full-size slide

  43. 50
    User Study to Identify Information Needs
    Overview of Ingredients
    Tailor and Evaluate Ingredients

    View full-size slide

  44. Influencing Options Option Hotspots Cause-Effect Chain
    CPU Profiling Program Slicing
    Performance Modeling
    Global Local

    View full-size slide

  45. Influencing Options Option Hotspots Cause-Effect Chain
    CPU Profiling Program Slicing
    Performance Modeling
    Global Local
    Information Providers

    View full-size slide

  46. Influencing Options
    Performance Modeling
    Global

    View full-size slide

  47. Influencing Options
    Performance Modeling
    Global

    View full-size slide

  48. Influencing Options
    Performance Modeling
    Global

    View full-size slide

  49. Influencing Options
    Performance Modeling
    Global

    View full-size slide

  50. Influencing Options
    Performance Modeling
    Global

    View full-size slide

  51. Option Hotspots
    Performance Modeling
    Local

    View full-size slide

  52. Option Hotspots
    Performance Modeling
    Local

    View full-size slide

  53. Option Hotspots
    Performance Modeling
    Local

    View full-size slide

  54. Option Hotspots
    Performance Modeling
    Local

    View full-size slide

  55. Cause-Effect Chain
    CPU Profiling Program Slicing

    View full-size slide

  56. Cause-Effect Chain
    CPU Profiling

    View full-size slide

  57. Cause-Effect Chain
    CPU Profiling

    View full-size slide

  58. Cause-Effect Chain
    CPU Profiling

    View full-size slide

  59. Cause-Effect Chain
    CPU Profiling

    View full-size slide

  60. Cause-Effect Chain
    CPU Profiling

    View full-size slide

  61. Cause-Effect Chain
    Program Slicing

    View full-size slide

  62. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  63. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  64. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  65. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  66. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  67. Main.main(…)
    Cursor.put(…)
    Main.init(…)
    Database.init(…)
    def init(Database db)
    Database.checkForNullParam(db.name, "dbName");
    Log.msg(Level.INFO, "db " + db.name + " open");
    ...
    if(db.replicated)
    configReplicated(...);
    ...
    this.cacheMode = db.cacheMode;
    this.commit = db.trans ? true : false;
    this.sync = db.dups ? true : false;
    if(db.evict)
    Evictor.init(db.evictorThreads);
    ...
    Cause-Effect Chain
    Program Slicing

    View full-size slide

  68. Influencing Options Option Hotspots Cause-Effect Chain
    Performance Modeling
    Global

    View full-size slide

  69. Influencing Options Option Hotspots Cause-Effect Chain
    Performance Modeling
    Global Local
    Performance Modeling

    View full-size slide

  70. Influencing Options Option Hotspots Cause-Effect Chain
    CPU Profiling
    Program Slicing
    Performance Modeling
    Global Local
    Performance Modeling

    View full-size slide

  71. Influencing Options Option Hotspots Cause-Effect Chain
    CPU Profiling
    Program Slicing
    Performance Modeling
    Global Local
    Performance Modeling
    GLIMPS

    View full-size slide

  72. Evaluation
    RQ1: To what extent do the designed information
    providers help developers debug the performance of
    con
    fi
    gurable software systems?

    View full-size slide

  73. Evaluation
    RQ1: To what extent do the designed information
    providers help developers debug the performance of
    con
    fi
    gurable software systems?
    Validation User Study
    Confirmatory User Study

    View full-size slide

  74. “Within-subject” study
    Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    System: Density Converter

    View full-size slide

  75. Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    “Within-subject” study
    System: Density Converter

    View full-size slide

  76. Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    “Within-subject” study
    System: Density Converter

    View full-size slide

  77. Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    “Within-subject” study
    System: Density Converter

    View full-size slide

  78. Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    SCALE +86.3
    AConverter.compress(…) +67.4
    “Within-subject” study
    System: Density Converter

    View full-size slide

  79. Validation User Study
    30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Control - Did not use GLIMPS
    Treatment - Used GLIMPS
    SCALE +86.3
    AConverter.compress(…) +67.4
    “Within-subject” study
    System: Density Converter
    Information providers support information
    needs

    View full-size slide

  80. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study

    View full-size slide

  81. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study

    View full-size slide

  82. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study

    View full-size slide

  83. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study

    View full-size slide

  84. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study

    View full-size slide

  85. 30’ 35’ 45’
    05’ 10’ 20’ 25’
    15’ 40’ 50’
    Influencing Options Option Hotspots Cause-Effect Chain
    Confirmatory User Study
    55’ 60’
    Treatment - Used GLIMPS
    Control - Did not use GLIMPS
    System: Berkeley DB
    Between-subject study
    Information providers help developers
    debug the performance of complex
    con
    fi
    gurable systems

    View full-size slide

  86. Influencing Options Option Hotspots Cause-Effect Chain
    Help developers debug the performance
    of configurable software systems
    CPU Profiling Program Slicing
    Performance Modeling
    Global Local
    Tailor and Evaluate Ingredients

    View full-size slide

  87. On Debugging the Performance of Configurable Software
    Systems: Developer Needs and Tailored Tool Support

    View full-size slide

  88. 96
    Overview of Ingredients
    Tailor and Evaluate Ingredients
    CONTRIBUTION
    User Study to Identify Information Needs
    CONTRIBUTION
    White-box Performance-In
    fl
    uence Modeling
    CONTRIBUTION

    View full-size slide

  89. Compositionality Compression
    Insights!

    View full-size slide

  90. Compositionality
    Performance-in
    fl
    uence models can be built by composing models
    built independently for smaller regions of code
    System Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region
    Region
    Region
    Measure System
    Decompose
    Measure Regions
    Compose
    98

    View full-size slide

  91. Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region
    Region
    Region
    Measure System
    Decompose
    Measure Regions
    Compose
    System
    99
    Compositionality
    Performance-in
    fl
    uence models can be built by composing models
    built independently for smaller regions of code

    View full-size slide

  92. Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region
    Region
    Region
    Measure System
    Decompose
    Measure Regions
    Compose
    System
    100
    Compositionality
    Performance-in
    fl
    uence models can be built by composing models
    built independently for smaller regions of code

    View full-size slide

  93. Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region
    Region
    Region
    Measure System
    Decompose
    Measure Regions
    Compose
    System
    101
    Compositionality
    Performance-in
    fl
    uence models can be built by composing models
    built independently for smaller regions of code

    View full-size slide

  94. Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Local Performance-In
    fl
    uence Model
    Region
    Region
    Region
    Measure System
    Decompose
    Measure Regions
    Compose
    System
    102
    Compositionality
    Performance-in
    fl
    uence models can be built by composing models
    built independently for smaller regions of code

    View full-size slide

  95. def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    103

    View full-size slide

  96. Time
    TRUE
    FALSE
    Time
    TRUE TRUE
    TRUE FALSE
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    104

    View full-size slide

  97. Time
    TRUE 5
    FALSE 0
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    105
    Time
    TRUE TRUE
    TRUE FALSE

    View full-size slide

  98. Time
    TRUE 5
    FALSE 0
    Tprocess = 5·
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    106
    Time
    TRUE TRUE
    TRUE FALSE

    View full-size slide

  99. Time
    TRUE 5
    FALSE 0
    Tprocess = 5·
    Time
    TRUE TRUE 3
    TRUE FALSE 2
    Tconvert = 2· + 1·. ·
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    107

    View full-size slide

  100. Tprocess = 5·
    Tconvert = 2· + 1·. ·
    Compositionality
    +
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    108

    View full-size slide

  101. Tprocess = 5·
    Tconvert = 2· + 1·. ·
    Compositionality
    +
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    T = 7· + 1· ·
    109

    View full-size slide

  102. Compositionality
    System
    System
    27 = 128 con
    fi
    gurations
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations 110

    View full-size slide

  103. Compositionality
    System
    System
    27 = 128 con
    fi
    gurations
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations 111

    View full-size slide

  104. Compositionality
    System
    System
    27 = 128 con
    fi
    gurations
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations 112

    View full-size slide

  105. Compositionality
    Insights!
    Compression

    View full-size slide

  106. Compression
    Compression allow us to simultaneously explore paths in multiple
    independent regions with a few con
    fi
    gurations
    114

    View full-size slide

  107. Compression
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    115

    View full-size slide

  108. Compression
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    116

    View full-size slide

  109. FALSE FALSE FALSE
    TRUE FALSE FALSE
    TRUE TRUE FALSE
    Compression
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    117

    View full-size slide

  110. FALSE FALSE FALSE
    TRUE FALSE FALSE
    TRUE TRUE FALSE
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    Time
    TRUE
    FALSE
    118
    Compression

    View full-size slide

  111. FALSE FALSE FALSE
    TRUE FALSE FALSE
    TRUE TRUE FALSE
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    Time
    TRUE
    FALSE 0
    Time
    TRUE
    FALSE 0
    Time
    TRUE
    FALSE 0
    119
    Compression

    View full-size slide

  112. FALSE FALSE FALSE
    TRUE FALSE FALSE
    TRUE TRUE FALSE
    if(a)
    ... // execution time: 1 second
    if(b)
    ... // execution time: 2 seconds
    if(c)
    ... // execution time: 3 seconds
    Time
    TRUE 1
    FALSE 0
    Time
    TRUE 3
    FALSE 0
    Time
    TRUE 2
    FALSE 0
    120
    Compression

    View full-size slide

  113. System
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations
    System
    4 con
    fi
    gurations 121
    Compression

    View full-size slide

  114. System
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations
    System
    4 con
    fi
    gurations 122
    Compression

    View full-size slide

  115. System
    22 + 24 + 22 + 24 + 22 = 38 con
    fi
    gurations
    System
    4 con
    fi
    gurations 123
    Compression

    View full-size slide

  116. System
    4 con
    fi
    gurations
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    TRUE FALSE TRUE TRUE FALSE TRUE TRUE
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    FALSE FALSE TRUE FALSE FALSE TRUE FALSE
    124
    Compression

    View full-size slide

  117. System
    4 con
    fi
    gurations
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    TRUE FALSE TRUE TRUE FALSE TRUE TRUE
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    FALSE FALSE TRUE FALSE FALSE TRUE FALSE
    125
    Compression

    View full-size slide

  118. System
    4 con
    fi
    gurations
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    TRUE FALSE TRUE TRUE FALSE TRUE TRUE
    FALSE FALSE FALSE FALSE FALSE FALSE FALSE
    FALSE FALSE TRUE FALSE FALSE TRUE FALSE
    126
    Compression

    View full-size slide

  119. Compression
    Measure Performance Build Model
    Compositionality
    System
    127

    View full-size slide

  120. Taint Analysis

    View full-size slide

  121. def main() {
    boolean a =
    ...
    process(a, b);
    }
    def process(boolean x, boolean y) {
    if(x)
    convert(y);
    ... // execution time: 5 seconds
    }
    def convert(boolean x) {
    if(x)
    ... // execution time: 3 seconds
    else
    ... // execution time: 2 seconds
    }
    In
    fl
    uencing options
    129

    View full-size slide

  122. Analyze Regions
    Measure Performance
    Build Model
    Taint Analysis
    Compression
    Compositionality
    System
    130

    View full-size slide

  123. ConfigCrusher Comprex
    Static Taint Analysis
    Regions: Control-flow
    statements
    Performance measurement:
    Instrumentation
    Dynamic Taint Analysis
    Regions: Methods
    Performance measurement:
    Off-the-shelf profiler
    131

    View full-size slide

  124. ConfigCrusher Comprex
    Static Taint Analysis
    Regions: Control-flow
    statements
    Dynamic Taint Analysis
    Regions: Methods
    132
    Both prototypes ef
    fi
    ciently build accurate
    and interpretable models
    Dynamic taint analysis scales to larger
    systems
    Method-level granularity does not
    sacri
    fi
    ces compression potential

    View full-size slide

  125. 134
    ConfigCrusher Comprex
    Evaluate cost, accuracy, interpretability
    13 open-source Java systems
    Evaluation
    50 combinations of
    Sampling
    and Machine
    Learning

    View full-size slide

  126. Results: Density Converter
    0
    Cost (minutes)
    Error (MAPE)
    0
    40
    20
    Comprex
    x
    200 random con
    fi
    gurations
    & Random Forest
    5
    10
    200 random con
    fi
    gurations
    & Stepwise linear regression
    200
    400
    100
    300
    x
    x
    60
    Con
    fi
    gCrusher
    x
    135

    View full-size slide

  127. x +
    +
    +
    Results: Density Converter
    0
    Cost (minutes)
    Error (MAPE)
    0
    40
    20
    Comprex 200 random con
    fi
    gurations
    & Random Forest
    5
    10
    200 random con
    fi
    gurations
    & Stepwise linear regression
    200
    400
    100
    300
    60
    Con
    fi
    gCrusher
    + Interpretable
    x Not interpretable
    136

    View full-size slide

  128. x +
    +
    +
    Results: Density Converter
    0
    Cost (minutes)
    Error (MAPE)
    0
    40
    20
    Comprex 200 random con
    fi
    gurations
    & Random Forest
    5
    10
    200 random con
    fi
    gurations
    & Stepwise linear regression
    200
    400
    100
    300
    60
    Con
    fi
    gCrusher
    + Interpretable
    x Not interpretable
    137
    Ef
    fi
    ciently build accurate performance-
    in
    fl
    uence models
    Models are interpretable for debugging
    purposes

    View full-size slide

  129. Influencing Options Option Hotspots
    White-box Modeling
    Global Local
    138
    Help developers debug the performance
    of configurable software systems
    White-box Performance-In
    fl
    uence Modeling
    CONTRIBUTION

    View full-size slide

  130. Thesis Statement
    Tailoring speci
    fi
    c white-box analyses to track how con
    fi
    guration
    options in
    fl
    uence the performance of code-level structures in
    con
    fi
    gurable software systems helps developers to
    (1) ef
    fi
    ciently build accurate and interpretable global and
    local performance-in
    fl
    uence models
    (2) more easily inspect, trace, understand, and debug
    con
    fi
    guration-related performance issues.
    139

    View full-size slide