Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[DE] Performance Testing für DevOps in der Clou...

[DE] Performance Testing für DevOps in der Cloud @ Tech 'n' drinks by myposter.de

See: https://stormforger.com/presentations/2019/performance-testing-devops-cloud-techndrinks/

Demo: http://stormforger.com/demo

Host: https://team.myposter.de/tech-events/

Performancetests sind ein wichtiges Instrument, um Laufzeitumgebung und Verhalten eines Systems in Produktion besser zu verstehen. Es ist essentiell, nicht-funktionale Anforderungen wie Kapazität, Stabilität sowie Skalierbarkeit zu überprüfen, was zum Beispiel im E-Commerce-Bereich erfolgsentscheidend sein kann.

Ob Sales/Marketing-Aktion oder normaler Betrieb: der Nutzer und potentielle Kunde profitiert von hoher Verfügbarkeit und durchgängig niedrigen Antwortzeiten. Performance ist einer der wichtigsten Faktoren, die über kurz oder lang den Ausschlag für Erfolg oder Misserfolg eines Produktes geben.

Aber mein Cloud-Computing-Anbieter AWS skaliert doch für mich, so lange ich es mir leisten kann? Ja, aber… Die Amazon Web Services skalieren und arbeiten nur mit Ressourcen. Sie machen Ihr System nicht automatisch schnell, stabil und – noch wichtiger – skalierbar.

Performance Testing ist entscheidend, um das eigene System, Architektur-Design und Ihre Cloud-Hosting-Umgebung zu verstehen, so wie es auch im Performance-Efficiency-Pillar des AWS Well-Architected Framework empfohlen wird.

Der Talk zeigt auf, wie man auf einfache Weise mit Performance Testing in DevOps-Teams startet, welche Herausforderungen bestehen und wie die Qualität eines Systems mit Performancetests sichergestellt werden kann. Es wird veranschaulicht, wie z.B. HTTP-basierte Systeme ganz prinzipiell von regelmäßiger Testpraxis profitieren. So wird durch automatisierten und minimalen Aufwand Performance Testing zu einem kontinuierlichen Informationsgeber über Systemqualität.

Lars Wolff

July 03, 2019
Tweet

More Decks by Lars Wolff

Other Decks in Technology

Transcript

  1. Introduction @larsvegas
 [email protected] My Journey so far • Software engineering

    • Consulting and Agile coaching • Ship it! • AWS User Group Cologne & AWS Community Days Germany • Performance Testing
  2. Performance The ability of a system to execute one task


    within a defined dimension = efficiency of a system
 ↪ 1 server manages 250 rps with p99 500ms.
  3. Scalability capacity 0 1000 2000 3000 4000 resources 5 10

    15 20 25 30 Effectivity by which the capacity can be increased through adding resources
  4. What makes me recognize that I have a performance problem?

    http://www.slideshare.net/jboner/scalability-availability-stability-patterns/15-How_do_I_know_if
  5. What makes me recognize that I have a scaling problem?

    http://www.slideshare.net/jboner/scalability-availability-stability-patterns/15-How_do_I_know_if
  6. DevOps – CALMS Culture Automation Lean Metrics Sharing Hearts &

    Minds Embrace Change CI/CD “Infrastructure as Code” Focus on producing value for the user Small, shippable, increments Measure everything Show improvement Open information sharing Collaboration
  7. “In software engineering,
 performance testing is in general,
 a testing

    practice performed to
 determine how a system performs
 in terms of responsiveness and stability
 under a particular workload.”
 https://en.wikipedia.org/wiki/Software_performance_testing – Wikipedia
  8. " $

  9. Performance Tests • Collection of 
 non-functional test methods •

    Not to be defined with 
 absolute selectivity • Different goals and perspectives Load Testing Stress Testing Spike Testing Soak Testing Endurance Testing Resilience Testing Configuration Testing Scalability Testing
  10. Performance Tests? • Not only for testing releases • Evaluation

    of… • configuration • technology & proof of concepts, • troubleshooting & debugging
  11. Performance Testing Maturity • no performance tests, at all •

    tests if problems occur during production • tests (shortly) before large-scale (sales & marketing) campaigns ) • regular testing ) ) • tests as a regular, integrated DevOps instrument ) ) ) ) )
  12. Challenges • organizational obstacles, e.g. buy-in
 (management & team) •

    testing goal & scope • visibility in your own systems • The right performance test tool for the job
 – (usually) no tooling problem • start!
  13. Steps to Perf Tests™ • bring together the stakeholders +

    development operations QA product management marketing
  14. Steps to Perf Tests™ Kick off for 15 Min
 –

    All Stakeholders Heads Up STATE
 We’ll NOW create a performance testing culture to make sure that we can hold our promise to the business/customer to be fast and available under different amount of usage/users/transactions.
 We so match performance and availability requirements to keep the business save. ASK
 Do you all agree? STATE
 OK, we start with testing a very easy use case to raise a baseline of numbers about performance and availability. This friday, it’ll take a 2 hrs timebox. + development operations QA product management marketing
  15. Steps to Perf Tests™ Test Case • Create a first,

    simple Test Case • Q: Test Data? • Q: How much traffic?
  16. Steps to Perf Tests™ Test Case • Ask Analytics
 for

    the hour of the highest number of users in a week. • Analytics: 
 We have 10.000 visits in this hour. The Users visits flower-list, one flower detail.
 30% check out and payment.
  17. Steps to Perf Tests™ Test Case • Test Environment &


    Load Generator Setup • Since Infrastructure is code:
 Just start your test environment!
  18. Steps to Perf Tests™ Run Test • 20 Minutes •

    Invite ALL stakeholders! • It’s about collaboration
  19. Steps to Perf Tests™ Inspect Report • Response Time? •

    If errors occurred:
 Do you see these in your monitoring, too?
  20. Steps to Perf Tests™ Meet and communicate to stakeholders: •

    ONE Statement like
 We aim for baseline numbers.
 We tested flower-list and flower-detail.
 We created 4x traffic of peak hour.
 We saw 20% of errors.
 But always responded in under 1000ms (p99). • Ask
 Are you happy? • Ask
 Whats’ next? Checkout? + development operations QA product management marketing
  21. Steps to Perf Tests™ Non-functional requirements • No errors! •

    Each request must be faster than 250ms p99th
  22. Steps to Perf Tests™ Automate • Run test runs from

    any pipeline • Run automated checks of
 non-functional requirements
  23. It’s a feedback loop Stakeholder Requirements Definition
 (test scenario) Execution


    (test case) Result Interpretation & Reporting Business Needs
  24. Start creating a Community of practice Special Interest Group since

    performance testing is an orthogonal topic