Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud

Jürgen Cito
September 03, 2015

The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud

Presentation of our research paper "The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud" at the ESEC/FSE'15 conference in Bergamo, Italy.

Preprint is available here: http://arxiv.org/abs/1409.6502

Jürgen Cito

September 03, 2015
Tweet

More Decks by Jürgen Cito

Other Decks in Research

Transcript

  1. The Making of Cloud Applications:
    An Empirical Study on Software Development for the Cloud
    ESEC/FSE’15 - Bergamo, Italy
    Jürgen Cito
    Philipp Leitner
    Thomas Fritz
    Harald C. Gall

    View full-size slide

  2. What is different in the cloud for developers?
    How does the development of applications change in a
    cloud environment?
    What kind of metrics & tools do developers utilize for
    building software for the cloud?

    View full-size slide

  3. Cloud Computing
    Hardware
    Virtualization
    Database
    App
    Server
    Application Runtime
    Data
    Application
    IaaS PaaS SaaS
    Managed by Client Managed by Cloud Vendor
    Higher Abstraction
    Less
    Restrictions
    Hardware
    Virtualization
    Database
    App
    Server
    Application Runtime
    Data
    Application
    Hardware
    Virtualization
    Database
    App
    Server
    Application Runtime
    Data
    Application

    View full-size slide

  4. Related Work
    Cloud provider issues

    Server Management/Resource Allocation [Beloglazov et al. 2012, Marshall et al. 2011]
    Cloud Performance

    Scientific applications [Iosup et al. 2011], SaaS applications [Bezemer et al. 2014]
    Industry Surveys 

    DevOps Survey [PuppetLabs 2014], Enterprise Cloud Development [Gigaom 2014]
    None of the work discussed so far has empirically evaluated
    how cloud software is actually developed in practice

    View full-size slide

  5. Study Method
    Related Work Study to retrieve initial questions for interviews

    Interviews with 25 professional cloud developers

    8 companies of varying sizes, 9 years experience (± 6.5), deploy on IaaS or PaaS
    Quantitative Survey with 294 cloud developers
    Contacted Github users that favourited popular Cloud repositories (AWS, Azure,…)

    Analysis of transcribed interviews and quantitative survey data
    Extraction of codes, categories, comments

    View full-size slide

  6. Percentages
    0 5 10 15 20 25 30
    Focus on Product,
    Productivity
    Faster Delivery,
    Time-to-market
    Lower Costs/TCO
    (Total Cost of Ownership)
    Automation, Deployment Pipelines
    Easier Infrastructure Setup and Maintenance
    Scalability / Elasiticity 31%
    30%
    22%
    11%
    10%
    9%
    Perceived Advantages of 

    Software Development for the Cloud
    “What are the main differences for you when developing
    an application for the cloud as opposed to non-cloud environments?”

    View full-size slide

  7. Key Drivers for Change in Cloud Software Development
    API-driven
    Infrastructure-at-scale
    Cloud
    Instance Volatility

    View full-size slide

  8. Key Drivers cause Ripple Effect
    Deployment & Automation Communication

    Metrics & Tool Usage Design & Implementation

    Troubleshooting & Maintenance 


    View full-size slide

  9. Key Drivers cause Ripple Effect
    Deployment & Automation Communication

    Metrics & Tool Usage Design & Implementation

    Troubleshooting & Maintenance 


    View full-size slide

  10. Infrastructure Transparency
    Deployment & Automation / Communication
    Infrastructure provisioning and application deployment 

    in the cloud largely automated
    Servers are not seen as durable entities
    Any changes in infrastructure are defined in code
    ”What happens in our infrastructure is a lot more obvious.
    Everything we do on that level [infrastructure] is over code (...)
    So, I don’t need ask my colleague what he did to get that process
    running - I just look at the code and maybe the commit history”

    View full-size slide

  11. Key Drivers cause Ripple Effect
    Deployment & Automation Communication

    Metrics & Tool Usage Design & Implementation

    Troubleshooting & Maintenance 


    View full-size slide

  12. Metric Awareness & Availability
    Monitoring & Production Data

    View full-size slide

  13. Metric Awareness & Availability
    Monitoring & Production Data
    62% say more metrics are
    available in the cloud
    neutral
    strongly
    disagree
    strongly
    agree

    View full-size slide

  14. Metric Awareness & Availability
    Monitoring & Production Data
    84% say they look at
    metrics on a regular basis
    neutral
    strongly
    disagree
    strongly
    agree

    View full-size slide

  15. Tooling for the Cloud
    Changes in Tools and Metric Usage
    “Which tools do you use specifically for development for the
    cloud that you did not use before?”
    Percentages
    0 10 20 30 40 50 60
    Other Analytics
    (Usage/Cost Tracking, ..)
    Cloud Provider SDK
    (EC2 API, Heroku Toolbelt,...)
    Performance Management
    (APM Tools, New Relic, Pingdom,...)
    Log Management & Analytics
    (Splunk, Loggly, Librato, ...)
    Configuration Management
    (Chef, Vagrant, Docker,...)
    13%
    18%
    27%
    39%
    57%

    View full-size slide

  16. Key Drivers cause Ripple Effect
    Deployment & Automation Communication

    Metrics & Tool Usage Design & Implementation

    Troubleshooting & Maintenance 


    View full-size slide

  17. Troubleshooting & Maintenance
    Problematic cloud instances often not accessible anymore
    Hot fixes are discouraged and searching for logs in production 

    is often not possible anymore
    Reproducing issues has become both tougher and easier:

    > Tougher because cloud applications are inherently distributed

    > Easier through deployment automation to spin up test environment

    View full-size slide

  18. Performance Troubleshooting
    Monitoring & Production Data
    Developers have a reactive approach to dealing with metrics


    Alerts (beware of alert fatigue!)


    Performance Issue reported in issue trackers


    ”I try to reproduce and solve the issue locally. Looking for the
    particular issue in the dashboard and jumping back and forth
    between the code is rather tedious”
    Developers rather go by their gut feeling than looking 

    at runtime data

    View full-size slide

  19. What is different in the cloud for developers?


    [Mixed-method study interviewing 25 developers and surveying 294 developers]
    Cloud is full of uncertainty. Servers are not durable entities
    anymore, they are terminated and re-created regularly.


    Developers must rely heavily on infrastructure automation, log
    and metric management — which are also best practice, but in the
    cloud they are a must for successful development and operation.


    View full-size slide

  20. Implications
    Research on Infrastructure Evolution

    Infrastructure-as-Code and container technology make infrastructure part of software evolution


    Support Developers in Managing Logs and Metrics

    Log-writing recommender systems in accordance with log management solutions


    Tools for Developer Targeted Analytics

    Refine analytics data to be actionable for developers
    J. Cito et al., “Runtime Metric Meets Developer – Building Better
    Cloud Applications Using Feedback,” in SPLASH Onward’15

    View full-size slide

  21. @citostyle
    Jürgen Cito
    Slides: speakerdeck.com/citostyle

    View full-size slide