Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Context-Based Analytics - Establishing Explicit 
Links between Runtime Traces and Source Code

Context-Based Analytics - Establishing Explicit 
Links between Runtime Traces and Source Code

Joint work with IBM Research on improving cloud analytics presented at the International Conference on Software Engineering (ICSE'17) in Buenos Aires, Argentina

Jürgen Cito

May 25, 2017
Tweet

More Decks by Jürgen Cito

Other Decks in Research

Transcript

  1. Context-Based Analytics - Establishing Explicit 

    Links between Runtime Traces and Source Code
    ICSE’17, Software Engineering in Practice, Buenos Aires, Argentina
    Source: Jesus Alexander Reyes Sánchez https://flic.kr/p/6T8rNY

    View full-size slide

  2. 2
    Distributed Architectures
    Multiple Technologies
    Infrastructure at Scale
    Scattered Information
    Figure taken from Chen et al. A provider-side viewof web search response time. 2013.
    Modern Software Systems
    2

    View full-size slide

  3. State-of-Practice
    Log Management, Dashboard
    3

    View full-size slide

  4. Solving Problems in Modern Software Systems
    Metrics Dashboard
    .......
    [26/06/2015:21205.0], responseTime, "CustomerS", 204
    [26/06/2015:21215.0], responseTime, "CustomerS", 169
    [26/06/2015:21216.0], connection, "VM2", 0.73
    [26/06/2015:21216.0], connection, "VM1", 0.69
    [26/06/2015:21216.1], vmBilled, “CSVM2”, 0.35
    [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216]
    ........
    Hint#1
    Distributed Logs &
    Configuration
    .......
    [26/06/2015:21205.0], environment, “DB”, test
    [26/06/2015:21215.0], environment, “USR”, admin
    [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216]
    ........
    Hint#2
    Version Control History
    Hint#3
    Search Code Files
    Hint#4
    Hint#5
    4

    View full-size slide

  5. Developers build a mental context-model


    They establish implicit links between fragments
    5

    View full-size slide

  6. Idea: Establish Explicit Links between 

    Runtime Traces and Source Code
    [26/06/2015:21216.0], connection, "VM2", 0.73
    [26/06/2015:21216.0], connection, "VM1", 0.69
    Connection
    [26/06/2015:21205.0], environment, “DB”, test
    [26/06/2015:21215.0], environment, “USR”,admin
    Environment
    Relevant Code Fragment
    6

    View full-size slide

  7. Conceptual Framework
    7
    Meta-Model
    Application
    Model
    Initial Graph
    Construction
    Context
    Expansion
    Offline Online

    View full-size slide

  8. Conceptual Framework

    Meta-Model
    Time Series Entity
    Application State
    Set Entity
    Unit Entity
    Program Fragment
    Program File
    Configuration File Script File
    n
    8

    View full-size slide

  9. Conceptual Framework

    Application-Model
    Network
    VM
    Instance
    Memory
    CPU
    Environment
    Variables
    Heat Template
    Bash Script
    Java File
    Java
    Method
    Access Rate
    Swap
    VCPU
    Disk IO
    Request
    Packetloss
    Adaptation Bitrate
    Jitter
    Frames
    Exception
    .ini-config
    Network
    Network
    Telecommunications
    Application
    Application
    Model
    Process
    Memory
    CPU
    Environment
    Variables
    Dockerfile
    Bash Script
    Python
    File
    Python
    Method
    API
    Endpoint
    Response Time
    Status Code
    Online Processing
    EV
    ‘DB’
    Process
    ‘python’
    Graph
    Construction
    Method
    ‘get_papers’
    Bash
    start_db.sh
    Context
    Expansion
    Feature
    Memory
    {74,78, 75…}
    Time Series Entity
    Runtime Entity
    Set Entity
    Unit Entity
    Code Fragment
    Program File
    n
    Cloud Middleware
    Service
    9

    View full-size slide

  10. Conceptual Framework
    Online Processing
    Variables Bash Script
    File
    Python
    Method
    API
    Endpoint
    Response Time
    Status Code
    Online Processing
    EV
    ‘DB’
    Process
    ‘python’
    Graph
    Construction
    Method
    ‘get_papers’
    Bash
    start_db.sh
    Context
    Expansion
    Feature
    Visualization
    Memory
    {74,78, 75…}
    Context Graph
    Starting Nodes
    Context Inference on Node
    Similarity Measure
    10

    View full-size slide

  11. Context-Based Analytics Prototype
    11

    View full-size slide

  12. Industrial Case Study
    Application & Tasks
    CASE STUDY APPLICATION
    active-deploy in IBM Bluemix
    8 typical problem diagnosis tasks from issue tracker
    12

    View full-size slide

  13. Industrial Case Study
    Effort Metrics
    # Steps: Number of steps taken
    - Data Query
    - Creating Visualizations/Plots
    - Software History Analysis
    # Traces: Number of traces inspected
    - Log statements
    - Graphical Representations
    - Datastore Entries
    13

    View full-size slide

  14. Industrial Case Study
    Baseline
    vs
    Company Baseline
    (Kibana)
    CBA
    Prototype
    14

    View full-size slide

  15. Industrial Case Study
    Preliminary Results
    Effort reduction of CBA compared to a company baseline, Kibana
    48% reduction in steps taken to solve the tasks
    40% reduction in traces inspected to solve the tasks
    15

    View full-size slide

  16. Discussion & Future Work
    Dynamic Context Models
    [Adapting application models on-the-fly through queries]
    Learning context inference models
    [Instrumenting queries and interaction with log data to learn models]
    Integrating Business Metrics
    [Establishing explicit links between business metrics and source code]
    16

    View full-size slide

  17. Jürgen Cito
    Context-Based Analytics establishes explicit links between
    runtime traces and source code to form a context-graph that
    developers can navigate to diagnose problems originating at
    runtime.
    Distributed Runtime Information
    Context-Based Analytics:
    Conceptual Framework & Prototype
    ON
    ix
    Case Study: active-deploy
    48% reduction in steps taken
    40% reduction in traces inspected
    Reduced effort in case study
    Meta-Model
    Application
    Model
    Process
    Memory
    CPU
    Environment
    Variables
    Dockerfile
    Bash Script
    Python
    File
    Python
    Method
    API
    Endpoint
    Response Time
    Status Code
    Online Processing
    EV
    ‘DB’
    Process
    ‘python’
    Graph
    Construction
    Method
    ‘get_papers’
    Bash
    start_db.sh
    Context
    Expansion
    Feature
    Visualization
    Memory
    {74,78, 75…}
    Time Series Entity
    Runtime Entity
    Set Entity
    Unit Entity
    Code Fragment
    Program File
    Configuration File Script File
    n
    ICSE’17, Software Engineering in Practice, Buenos Aires, Argentina
    @citostyle

    View full-size slide