The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud

Presentation of our research paper "The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud" at the ESEC/FSE'15 conference in Bergamo, Italy.

Preprint is available here: http://arxiv.org/abs/1409.6502

Transcript

1. The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud ESEC/FSE’15 - Bergamo, Italy Jürgen Cito Philipp Leitner Thomas Fritz Harald C. Gall
2. What is different in the cloud for developers? How does the development of applications change in a cloud environment? What kind of metrics & tools do developers utilize for building software for the cloud?
3. Cloud Computing Hardware Virtualization Database App Server Application Runtime Data Application IaaS PaaS SaaS Managed by Client Managed by Cloud Vendor Higher Abstraction Less Restrictions Hardware Virtualization Database App Server Application Runtime Data Application Hardware Virtualization Database App Server Application Runtime Data Application
4. Related Work Cloud provider issues
 Server Management/Resource Allocation [Beloglazov et al. 2012, Marshall et al. 2011] Cloud Performance
 Scientific applications [Iosup et al. 2011], SaaS applications [Bezemer et al. 2014] Industry Surveys 
 DevOps Survey [PuppetLabs 2014], Enterprise Cloud Development [Gigaom 2014] None of the work discussed so far has empirically evaluated how cloud software is actually developed in practice
5. Study Method Related Work Study to retrieve initial questions for interviews 
 Interviews with 25 professional cloud developers
 8 companies of varying sizes, 9 years experience (± 6.5), deploy on IaaS or PaaS Quantitative Survey with 294 cloud developers Contacted Github users that favourited popular Cloud repositories (AWS, Azure,…)
 Analysis of transcribed interviews and quantitative survey data Extraction of codes, categories, comments
6. Percentages 0 5 10 15 20 25 30 Focus on Product, Productivity Faster Delivery, Time-to-market Lower Costs/TCO (Total Cost of Ownership) Automation, Deployment Pipelines Easier Infrastructure Setup and Maintenance Scalability / Elasiticity 31% 30% 22% 11% 10% 9% Perceived Advantages of 
 Software Development for the Cloud “What are the main differences for you when developing an application for the cloud as opposed to non-cloud environments?”
7. Key Drivers for Change in Cloud Software Development API-driven Infrastructure-at-scale Cloud Instance Volatility
8. Key Drivers cause Ripple Effect Deployment & Automation Communication
 Metrics & Tool Usage Design & Implementation
 Troubleshooting & Maintenance 

9. Key Drivers cause Ripple Effect Deployment & Automation Communication
 Metrics & Tool Usage Design & Implementation
 Troubleshooting & Maintenance 

10. Infrastructure Transparency Deployment & Automation / Communication Infrastructure provisioning and application deployment 
 in the cloud largely automated Servers are not seen as durable entities Any changes in infrastructure are defined in code ”What happens in our infrastructure is a lot more obvious. Everything we do on that level [infrastructure] is over code (...) So, I don’t need ask my colleague what he did to get that process running - I just look at the code and maybe the commit history”
11. Key Drivers cause Ripple Effect Deployment & Automation Communication
 Metrics & Tool Usage Design & Implementation
 Troubleshooting & Maintenance 

12. Metric Awareness & Availability Monitoring & Production Data
13. Metric Awareness & Availability Monitoring & Production Data 62% say more metrics are available in the cloud neutral strongly disagree strongly agree
14. Metric Awareness & Availability Monitoring & Production Data 84% say they look at metrics on a regular basis neutral strongly disagree strongly agree
15. Tooling for the Cloud Changes in Tools and Metric Usage “Which tools do you use specifically for development for the cloud that you did not use before?” Percentages 0 10 20 30 40 50 60 Other Analytics (Usage/Cost Tracking, ..) Cloud Provider SDK (EC2 API, Heroku Toolbelt,...) Performance Management (APM Tools, New Relic, Pingdom,...) Log Management & Analytics (Splunk, Loggly, Librato, ...) Configuration Management (Chef, Vagrant, Docker,...) 13% 18% 27% 39% 57%
16. Key Drivers cause Ripple Effect Deployment & Automation Communication
 Metrics & Tool Usage Design & Implementation
 Troubleshooting & Maintenance 

17. Troubleshooting & Maintenance Problematic cloud instances often not accessible anymore Hot fixes are discouraged and searching for logs in production 
 is often not possible anymore Reproducing issues has become both tougher and easier:
 > Tougher because cloud applications are inherently distributed
 > Easier through deployment automation to spin up test environment
18. Performance Troubleshooting Monitoring & Production Data Developers have a reactive approach to dealing with metrics
 
 Alerts (beware of alert fatigue!)
 
 Performance Issue reported in issue trackers
 
 ”I try to reproduce and solve the issue locally. Looking for the particular issue in the dashboard and jumping back and forth between the code is rather tedious” Developers rather go by their gut feeling than looking 
 at runtime data
19. What is different in the cloud for developers?
 
 [Mixed-method study interviewing 25 developers and surveying 294 developers] Cloud is full of uncertainty. Servers are not durable entities anymore, they are terminated and re-created regularly.
 
 Developers must rely heavily on infrastructure automation, log and metric management — which are also best practice, but in the cloud they are a must for successful development and operation.

20. Implications Research on Infrastructure Evolution
 Infrastructure-as-Code and container technology make infrastructure part of software evolution 
 
 Support Developers in Managing Logs and Metrics
 Log-writing recommender systems in accordance with log management solutions
 
 Tools for Developer Targeted Analytics
 Refine analytics data to be actionable for developers J. Cito et al., “Runtime Metric Meets Developer – Building Better Cloud Applications Using Feedback,” in SPLASH Onward’15
21. @citostyle Jürgen Cito <cito@ifi.uzh.ch> Slides: speakerdeck.com/citostyle