Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Designing and Visualizing Cloud Architectures f...

jng
April 27, 2019

Designing and Visualizing Cloud Architectures for Humans

Today's software architectures are organic, scaling with cloud infrastructure and changing with agile development cycles. As software matures and teams grow, so does complexity. Combining d3.js visualizations and monitoring, we'll explore different dimensions of cloud architectures, both in design and operation to focus less on technical details and more on what matters: people.

jng

April 27, 2019
Tweet

More Decks by jng

Other Decks in Technology

Transcript

  1. Designing and Visualizing Cloud Architectures for Humans • Julie Ng

    Cloud Solution Architect Microsoft • Frontconf Munich 27 April 2019 Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 1
  2. • Decoupling has continued: • microservices, containers, serverless… what’s next?

    • on-prem à cloud à where’s my data? • Failures more complex • Debugging harder • Does it have to be this way? • WHY is it this way? Did I order this Architecture Spaghe>? IT-TRANSFORMATION AT SCALE Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 6
  3. • One that works • Has no downtime • Enterprise

    Architecture = bridging technology and business • Allianz Germany 2018 • 36.4 € Billion Revenue • 2.4 € Billion Operating Profit • 129 years old • So how much does 1 hour of downtime cost? What is a good architecture? LARGE SCALE - ENTERPRISE ARCHITECTURE COMPANY STATISTICS FROM h/ps://www.allianzdeutschland.de/die-allianz-deutschland-im-portraet/ Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 7
  4. Cloud Context The Human Element What causes DownAme? PER GOOGLE

    NEXT (YELLOW), INDEPENDENT ORACLE USER GROUP (BLUE) 3rd Party or Cloud Server Failures Storage Failures Application Errors Usage Spikes/Surges Power Outages Network Outage Human Error Bad Deployments Monitoring Gaps Scaling Boundaries Uneven Sharding ' Humans cause downtime The cloud just works… when configured properly (by humans) Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 8
  5. • Measure quality of architecture by downtime • Assume the

    cloud just works • Assume we humans caused the failure • Goal is to understand human factor in architecture • How do we behave? • Is there a pattern? • Why do we behave the way we do? The Human Factor on Architecture WHY I BUILT NEWTON.JS We’re pushing the buttons Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 9
  6. Architecture Diagrams CLASSIC DOCUMENTATION Diagram Source – Scalable Web Application

    Reference Architecture https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/app-service-web-app/scalable-web-app • Classic diagrams • Show intent • But very static • Not real-time. Quickly outdated. • What do arrows indicate? • Dependencies? • Data Flow? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 11
  7. Typical Real Time Monitoring Dashboards INTERACTIVE DOCUMENTATION? • Latency •

    CPU Usage • Uptime • Storage Consumption • Memory Consumption • Nodes Count • Temperature Does this make sense for a cloud application? No. The cloud just works, remember? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 12
  8. • For 10.000 feet overview - too much information •

    Missing relationships – remember those arrows? • To understand how everything is connected, we have to go return to the humans… Monitor uptime for hundreds of components? JUST UPTIME Graphic Source https://community.grafana.com/t/status-panel-for-more-than-200-servers/2710 Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 13
  9. What we imagine – 1 team ORGANIZATIONS – UNIFORM AND

    STANDARDIZED IN LARGE OPEN SPACE OFFICES Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 15
  10. Reality – many chaoAc teams, but diverse ORGANIZATIONS So do

    you really expect 1 architecture? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 16
  11. Conway’s Law BEHAVIORAL PSYCHOLOGY “ Organizations which design systems... are

    constrained to produce designs which are copies of the communication structures of these organizations. - Melvin Conway. "How Do Committees Invent?” Datamation (April 1968). So despite independent teams, is there only a single possible overall result? Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 17
  12. • Installed on base images so every team uses them

    – easy win? • Large organizations and systems à too much noise • Arrows indicate request flow, but not intent - the relationships we wanted to build • How can we indicate intent? Automated graphs with trace agents AUTOMATE RELATIONSHIPS Example automated graph Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 18
  13. Junior designers… make it look good JULIE ZHOU, VP PRODUCT

    FACEBOOK ON GOALS https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 19
  14. Senior designers… make new value JULIE ZHOU, VP PRODUCT FACEBOOK

    ON GOALS https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e Before we automate, let’s create. How can we reintroduce intent to real time monitoring? Remember, that’s how we choose to measure good architecture. Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 20
  15. • There are no loops over the DOM • Based

    on data() • selectAll() does it all: • Adds Circles • Updates Circles • Removes Circles Data Driven – describe what, not how D3.JS - NO FOR LOOPS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 22
  16. • Nodes array • IDs must be unique • Links

    Array • source & target reference IDs • References are key based, not Array index based #PROTIP Source and targets are not array index based – although d3.js documents and tutorials say they are. Example Data D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 23
  17. General Update Pattern D3.JS DATA DOM ELEMENTS Enter Exit Update

    Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 24
  18. • General Update Pattern • nodes.data(data) // join • nodes.exit()

    • nodes.enter() • Nodes.merge() // update Nodes D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 25
  19. • selection.data(data, (d) => d.id) // does not work for

    links * Links, Key Functions D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 26
  20. • Default Force, Particles Engine • Too uniform? D3.js Force

    Engine D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 27
  21. Webcola Force Layout D3.JS Frontconf - 27 April 2019 Designing

    and Visualizing Cloud Architectures for Humans 28
  22. • D3 Layout iterations: tick • Custom events via EventEmitter

    • this.emit('update') • Standard events • mouseover, mouseout • dragstart • etc. Event Driven D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 29
  23. • Runtime vs. State • Transactions Visualize Dependencies – intent!

    D3.JS Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 30
  24. Live Demo! Newton.js Cloud Architecture Graphs for Humans https://github.com/julie-ng/newtonjs-graph Frontconf

    - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 31
  25. • Follow Me! • julie.io • github.com/julie-ng • twitter.com/jng5 •

    Credits - all stock photos are from amazing photographers on Unsplash https://unsplash.com/collections/4638940/frontconf-talk-cloud-architectures- visualizations/98850e9cfebb9cc0d350ce787e6986e4 • Note - all opinions are my own and do not reflect those of my employers past and present. Thank you! BYE BYE Frontconf - 27 April 2019 Designing and Visualizing Cloud Architectures for Humans 32