Let a Sustainable Performance and Efficiency Culture Evolve!

“I’m totally fine, that you build and run slow, unscalable, unreliable systems. Here is my credit card you can spend how much ever you need to make that happen!”
– Nobody. Never. Ever.

OK, so let a sustainable performance and efficiency culture evolve!

Transcript

1. Lars Wolff Let a Sustainable Performance and Efficiency Culture Evolve!
2. Nobody. Never. Ever. “I’m totally fine, that you build and run slow, unscalable, unreliable systems. Here is my credit card – you can spend how much ever you need to make that happen!”
3. • Software Development • Agile Coaching • Founded load & performance testing SaaS StormForger with Sebastian Cohnen in 2014 • StormForger merged with Boston based machine learning company Carbon Relay in 2020, rebranded to StormForge • On the mission to help cloud-native organizations to challenge most efficient, high-quality delivery at speed • AWS UserGroup Lead Cologne/Germany • @larsvegas / lars@stormforge.io • #Post-it #Agile #Scrum #Kanban #LeanProduct #DevOps Lars Wolff
4. Challenges to Successfully Run Cloud-Native Workloads Performance matters • Speed and responsiveness? • Conversion? • Customer satisfaction? Scalability is key for the cloud • Capacity? • Thresholds for scaling? • Service utilization and impact? Reliability is not negotiable • System behavior? • Fault tolerance? • Fail-over and self- healing? (Cost) Efficiency is business crucial • Service configuration efficiency? • Service utilization? • Efficient budget spendings?
6. Common Understanding & Alignment To create and align on an overarching performance and efficiency promise (= quality promise) it's absolutely necessary to bring together all the stakeholders and create a common understanding. 👩💻 development 🤓 operations 🐨 QA 👷 product 💰 management 😱 marketing & sales
7. Coordination Requirements engineering Testing and Optimization —A lot of effort!
8. PerfTest & Optimize early and often! Let a sustainable culture evolve!
9. The Fourth Quadrant from “Agile Testing” by Lisa Crispin & Janet Gregory
10. Performance testing needs to evolve 10 Performance democratization Owned by dev, part of DevOps process Automated Performance Testing as Code Continuous Regular part of the release process Performance Culture Part of everyone’s job TO THIS: FROM THIS: Siloed QA department Bottleneck in the release process Manual Resource-intensive manual process Infrequent Done for major releases at best Low priority “Not my job” mentality Proactive Performance is a daily habit Reactive Performance is an afterthought
11. Test and optimize early and often! 1. Commitment to a quality and efficiency promise • External to the customer • Internal to the product company 2. Define (technical) indicators & metrics 3. (Initial scope and heavy lifting) 4. Iterate & automate 5. Over-communication and refinement of all non-functional requirements
12. Test and optimize early and often! Commitment to a quality and efficiency promise External to the customer “As a customer I receive any transaction from the product in < 1 second, so that I feel a responsive and useable product and stay happy.” Internal to the product company “As a product company • we commit to our customer quality promise and meet its criteria • we make sure that our system resources are in effective use of min 70% to keep our customers satisfied and the waste of compute resources minimal.” (100% available resources - 30% head room = 70% effective resource usage)
13. Test and optimize early and often! Define (technical) indicators & metrics • (SLA) • SLIs • SLOs • Non-functional requirements in general • e.g. “Request response time is < 1000ms for p99”
14. Test and optimize early and often! (Initial scope and heavy lifting) 1. Create a very first performance test case 2. Guess very first numbers 3. Prepare a very first version of test data 4. Run a small, low traffic test session with the whole team 5. Ask for help and improve continuously
15. Discover Define Build Execute Adapt • Run Tests • Measure/Inspect/ Observe • Identify and investigate on Issues • Optimize • Test Data • Test Case • Automation • NFR checks • Testing Strategy • Traffic Model • User Journeys / API Flows • System under Test (SUT) • Requirements / SLOs • Prioritization • Knowledge • Business needs • Ecosystem / Service landscape • Stakeholders / Collaborators • Important aspects / SLIs • SLAs Test and optimize early and often! Iteration
16. • Automate all the things! • Move in a development motion like: • Introduce new feature/endpoint ✅ • Create functional tests ✅ • Update/maintain performance test case 🤷👋👋👋 • Commit, integrate and run tests 👍 • Use our CLI to integrate and validate non-functional requirements continuously Automation – CI/CD Pipeline Integration Test and optimize early and often!
17. Test and optimize early and often! Over-communication and refinement of all non-functional requirements 1. Use your CoP to meet regularly 2. Always repeat your commitment to a quality and efficiency promise (and make it a vision to strive for) 3. Present results from every team, communicate achievements and violations of requirements 4. Discuss improvements and remove obstacles 5. Repeat :)
18. Lars Wolff A Sustainable Performance and Efficiency Culture Evolves!
19. Have fun! Thank you! – Q&A
20. Free DevOps from technical and organizational limits Deploy high-quality, scalable and reliable systems in very short cycles StormForge is an integral part of our production readiness strategy. Empowering our teams to assess the performance of each release within their continuous delivery pipeline, the team not only understands the risks related to their changes but it also enables us to run capacity planning and sizing for our Kubernetes cluster on AWS all the time. First, we used StormForge to prevent and detect performance issues. Over time, it evolved to the right tool for us to validate modification in architecture or infrastructure before every major change. StormForge is a great help to guarantee high-quality delivery to the users. Cynthia Dematteis-Krug Senior Quality Assurance Engineer, Shop Apotheke Service GmbH Alexander Heusingfeld Head of Digital Architecture & Infrastructure, Vorwerk