Share
Embed
Runtime Metric Meets Developer - Building Better Cloud Applications using Feedback
Conference talk at SPLASH'15 Onward! on our paper "Runtime Metric Meets Developer"
Video of the conference talk: https://www.youtube.com/watch?v=mzHoNcP6fGA
Preprint of the paper: https://peerj.com/preprints/985/
Jürgen Cito
October 28, 2015
Transcript
80 slides
-
Runtime Metric Meets Developer Building Better Cloud Applications Using Feedback
Jürgen Cito, Philipp Leitner, Harald C. Gall (University of Zurich) Aryan Dadashi, Anne Keller, Andreas Roth (SAP SE) SPLASH Pittsburgh 2015 - Onward -
Continuous Delivery High velocity of change, push of smaller changes
more frequently ~50-60 releases per day -
Continuous Delivery High velocity of change, push of smaller changes
more frequently ~50-60 releases per day ~100 releases per day -
Study on Software Development for the Cloud “The Making of
Cloud Applications” - FSE’15 Adapted from https://xkcd.com/1423/ Interview with 25 software developers that deploy in the cloud Developer Me -
Study on Software Development for the Cloud “The Making of
Cloud Applications” - FSE’15 Adapted from https://xkcd.com/1423/ Topic: Solving problems that have been detected in production Nah, I rather go by intuition? Do you look at any metrics? -
Feedback Driven Development Feedback Visualization Feedback Prediction integrating metrics into
daily developer workflows -
Feedback Driven Development Feedback Visualization Feedback Prediction integrating metrics into
daily developer workflows bring metrics into context -
Feedback Driven Development Feedback Visualization Feedback Prediction integrating metrics into
daily developer workflows prevent bad things from happening bring metrics into context -
Code Artifacts Deployment Conceptual Overview Cloud Infrastructure VM1 VM2 Supplier
Service User Interface Purch S -
Code Artifacts Deployment observe Conceptual Overview Cloud Infrastructure VM1 VM2
Supplier Service User Interface Purch S -
Code Artifacts Deployment observe Conceptual Overview Cloud Infrastructure VM1 VM2
Supplier Service User Interface Purch S -
Code Artifacts Deployment ....... [26/06/2015:21205.0], responseTime, "CustomerService", 204 [26/06/2015:21215.0], responseTime,
"CustomerService", 169 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Operations Data observe Conceptual Overview Cloud Infrastructure VM1 VM2 Supplier Service User Interface Purch S -
Code Artifacts Deployment ....... [26/06/2015:21205.0], responseTime, "CustomerService", 204 [26/06/2015:21215.0], responseTime,
"CustomerService", 169 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Operations Data observe Conceptual Overview Cloud Infrastructure VM1 VM2 Supplier Service User Interface Purch S -
Code Artifacts Deployment ....... [26/06/2015:21205.0], responseTime, "CustomerService", 204 [26/06/2015:21215.0], responseTime,
"CustomerService", 169 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Operations Data observe Conceptual Overview Cloud Infrastructure VM1 VM2 Supplier Service User Interface Purch S -
Code Artifacts Deployment ....... [26/06/2015:21205.0], responseTime, "CustomerService", 204 [26/06/2015:21215.0], responseTime,
"CustomerService", 169 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, "CustomerServiceVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Operations Data observe readConnecti on getConnectio ns ids connectionPo ol readConnecti on getConnectio ns ids connectionPo ol readConnection getConnections ids connectionPool Feedback Annotated Dependency Graph Conceptual Overview Cloud Infrastructure VM1 VM2 Supplier Service User Interface Purch S -
(a) Code in development environment (c) Annotated dependency graph combining
code artefacts and operations data and creating feedback readConnections getConnections showConnections setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections, 204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ (b) Operations data gathered through monitoring getImage (d) Feedback Visualization in the IDE Feedback Mapping -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
readConnections connections getConnections getImage setConnectionImage setConnectionStatus ....... [26/06/2015:21205.0], responseTime, “showConnections,
204 [26/06/2015:21215.0], responseTime, “setConnectionImage, 169 [26/06/2015:21216.0], responseTime, “PaymentService”, 79 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, "CustomerServiceVM1", 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ ....... [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM2", 0.73 [26/06/2015:21216.0], cpuUtilization, “ConnectionsVM1", 0.69 [26/06/2015:21216.1], vmBilled, “PaymentServiceVM, 0.35 [26/06/2015:21219.4], ids, “connectionIDs, [1,16,32,189,216] ........ Feedback Specification -
Code Change overallRating readConnection New Code (a) Code changed by
developer (b) Dependency graph before and after the code change (existing feedback is already attached) overallRating readConnection size: suppliers getSuppliers Loop:suppliers getPurchaseRating overallRating readConnection size: suppliers getSuppliers Loop:suppliers getPurchaseRating Inferred Critical Entity [Feedback Propagation] [Statistical Inference] (c) Prediction through Statistical Inference of new nodes without feedback and Feedback Propagation of the changes (d) Feedback Prediction in the IDE ? Live Performance Prediction -
New Code overallRating readConnection Code Change overallRating readConnection size: suppliers
getSuppliers Loop:suppliers getPurchaseRating -
New Code overallRating readConnection Code Change overallRating readConnection size: suppliers
getSuppliers Loop:suppliers getPurchaseRating ? -
overallRating readConnection size: suppliers getSuppliers Loop:suppliers getPurchaseRating [Statistical Inference] New
Code [Feedback Propagation] Predicted Entity -
Conservative implementation of Live Programming Impact of change in cloud
runtimes (Scaling, Cost, etc.) Prediction in combination with Refactoring and Diagnosis Overlap with data driven IDEs (Onward! Keynote) Outlook @citostyle Jürgen Cito <cito@ifi.uzh.ch> Slides: speakerdeck.com/citostyle “Modern software is all about data. Development environments should be, too.”