Slide 1

Slide 1 text

Designing and Visualizing Cloud Architectures for Humans • Julie Ng Cloud Solution Architect Microsoft • Frontconf Munich 27 April 2019 Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 1

Slide 2

Slide 2 text

What is Good Architecture? In the opinion of a former Enterprise Architect 1

Slide 3

Slide 3 text

Hello World! ONCE UPON A TIME…

Slide 4

Slide 4 text

Decouple Frontends! DIFFERENT UI REQUIREMENTS

Slide 5

Slide 5 text

Backends for Frontends (BFF) DIFFERENT UX REQUIREMENTS

Slide 6

Slide 6 text

• Decoupling has continued: • microservices, containers, serverless… what’s next? • on-prem à cloud à where’s my data? • Failures more complex • Debugging harder • Does it have to be this way? • WHY is it this way? Did I order this Architecture Spaghe>? IT-TRANSFORMATION AT SCALE Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 6

Slide 7

Slide 7 text

• One that works • Has no downtime • Enterprise Architecture = bridging technology and business • Allianz Germany 2018 • 36.4 € Billion Revenue • 2.4 € Billion Operating Profit • 129 years old • So how much does 1 hour of downtime cost? What is a good architecture? LARGE SCALE - ENTERPRISE ARCHITECTURE COMPANY STATISTICS FROM h/ps://www.allianzdeutschland.de/die-allianz-deutschland-im-portraet/ Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 7

Slide 8

Slide 8 text

Cloud Context The Human Element What causes DownAme? PER GOOGLE NEXT (YELLOW), INDEPENDENT ORACLE USER GROUP (BLUE) 3rd Party or Cloud Server Failures Storage Failures Application Errors Usage Spikes/Surges Power Outages Network Outage Human Error Bad Deployments Monitoring Gaps Scaling Boundaries Uneven Sharding ' Humans cause downtime The cloud just works… when configured properly (by humans) Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 8

Slide 9

Slide 9 text

• Measure quality of architecture by downtime • Assume the cloud just works • Assume we humans caused the failure • Goal is to understand human factor in architecture • How do we behave? • Is there a pattern? • Why do we behave the way we do? The Human Factor on Architecture WHY I BUILT NEWTON.JS We’re pushing the buttons Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 9

Slide 10

Slide 10 text

How do you Document Architecture? What did we intend to build? 2

Slide 11

Slide 11 text

Architecture Diagrams CLASSIC DOCUMENTATION Diagram Source – Scalable Web Application Reference Architecture https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/app-service-web-app/scalable-web-app • Classic diagrams • Show intent • But very static • Not real-time. Quickly outdated. • What do arrows indicate? • Dependencies? • Data Flow? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 11

Slide 12

Slide 12 text

Typical Real Time Monitoring Dashboards INTERACTIVE DOCUMENTATION? • Latency • CPU Usage • Uptime • Storage Consumption • Memory Consumption • Nodes Count • Temperature Does this make sense for a cloud application? No. The cloud just works, remember? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 12

Slide 13

Slide 13 text

• For 10.000 feet overview - too much information • Missing relationships – remember those arrows? • To understand how everything is connected, we have to go return to the humans… Monitor uptime for hundreds of components? JUST UPTIME Graphic Source https://community.grafana.com/t/status-panel-for-more-than-200-servers/2710 Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 13

Slide 14

Slide 14 text

OrganizaAons in the Cloud The people behind the code 3

Slide 15

Slide 15 text

What we imagine – 1 team ORGANIZATIONS – UNIFORM AND STANDARDIZED IN LARGE OPEN SPACE OFFICES Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 15

Slide 16

Slide 16 text

Reality – many chaoAc teams, but diverse ORGANIZATIONS So do you really expect 1 architecture? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 16

Slide 17

Slide 17 text

Conway’s Law BEHAVIORAL PSYCHOLOGY “ Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations. - Melvin Conway. "How Do Committees Invent?” Datamation (April 1968). So despite independent teams, is there only a single possible overall result? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 17

Slide 18

Slide 18 text

• Installed on base images so every team uses them – easy win? • Large organizations and systems à too much noise • Arrows indicate request flow, but not intent - the relationships we wanted to build • How can we indicate intent? Automated graphs with trace agents AUTOMATE RELATIONSHIPS Example automated graph Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 18

Slide 19

Slide 19 text

Junior designers… make it look good JULIE ZHOU, VP PRODUCT FACEBOOK ON GOALS https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 19

Slide 20

Slide 20 text

Senior designers… make new value JULIE ZHOU, VP PRODUCT FACEBOOK ON GOALS https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e Before we automate, let’s create. How can we reintroduce intent to real time monitoring? Remember, that’s how we choose to measure good architecture. Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 20

Slide 21

Slide 21 text

Visualizations with d3.js Data Driven Transformations on the DOM 4

Slide 22

Slide 22 text

• There are no loops over the DOM • Based on data() • selectAll() does it all: • Adds Circles • Updates Circles • Removes Circles Data Driven – describe what, not how D3.JS - NO FOR LOOPS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 22

Slide 23

Slide 23 text

• Nodes array • IDs must be unique • Links Array • source & target reference IDs • References are key based, not Array index based #PROTIP Source and targets are not array index based – although d3.js documents and tutorials say they are. Example Data D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 23

Slide 24

Slide 24 text

General Update Pattern D3.JS DATA DOM ELEMENTS Enter Exit Update Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 24

Slide 25

Slide 25 text

• General Update Pattern • nodes.data(data) // join • nodes.exit() • nodes.enter() • Nodes.merge() // update Nodes D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 25

Slide 26

Slide 26 text

• selection.data(data, (d) => d.id) // does not work for links * Links, Key Functions D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 26

Slide 27

Slide 27 text

• Default Force, Particles Engine • Too uniform? D3.js Force Engine D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 27

Slide 28

Slide 28 text

Webcola Force Layout D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 28

Slide 29

Slide 29 text

• D3 Layout iterations: tick • Custom events via EventEmitter • this.emit('update') • Standard events • mouseover, mouseout • dragstart • etc. Event Driven D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 29

Slide 30

Slide 30 text

• Runtime vs. State • Transactions Visualize Dependencies – intent! D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 30

Slide 31

Slide 31 text

Live Demo! Newton.js Cloud Architecture Graphs for Humans https://github.com/julie-ng/newtonjs-graph Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 31

Slide 32

Slide 32 text

• Follow Me! • julie.io • github.com/julie-ng • twitter.com/jng5 • Credits - all stock photos are from amazing photographers on Unsplash https://unsplash.com/collections/4638940/frontconf-talk-cloud-architectures- visualizations/98850e9cfebb9cc0d350ce787e6986e4 • Note - all opinions are my own and do not reflect those of my employers past and present. Thank you! BYE BYE Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 32