Share
Embed
Load and Performance Testing Microservices (Digital Transformation Meetup)
At the first Digital Transformation Meetup in Düsseldorf, Germany (http://www.meetup.com/Digital-Transformation-Meetup/events/225841693/) I gave a talk on the challenges of testing for non-functional requirements.
Sebastian Cohnen
October 18, 2015
Transcript
36 slides
-
Sebastian Cohnen, M. Sc. CEO & CTO • Focus on
web performance and architectures • 5+ years consulting and development • Performance engineering & regular execution of dynamic performance analysis for customers Lars Wolff CEO & CPO • Focus on large, technical ambitious web projects and products • 10+ years consulting and development • Agile Coach / Certified ScrumMaster®; strategy, conception, product/project management plus development Hello! -
With load tests the system you want to test will
be loaded with a defined amount of requests. The behavior of the system will thereby be monitored and rated. -
Performance, Load, Configuration, […]-Testing • There are many variations on
what and how you can assess performance related aspects. • What you need to do depends on what you want to learn about your system. -
Fast Load Times = Better Conversion Rate = $$$ (especially
true for eCommerce) -
Why (Performance) Testing? • I'd argue, that every system has
issues. • Almost every project we worked on so far, has had (serious) issues. • YOU want to find issues before your customers do! -
What you might want to know / learn about your
system? • Speed, Latency, Performance • Stability • Resource Requirements, Capacity Planning • Resiliency, Failure Scenarios, Edge Cases -
Top Performance Issues 1. Configuration 2. Configuration 3. Configuration 4.
Bugs 5. New Features 6. Resource Contention (hello RDMS!) 7. Architectural Design -
Performance vs Scalability • Performance: "1x instance can handle
250 API requests per second with p99 at 50ms…" • Scalability: "10x instances can handle 2500 API requests per second with p99 at 50ms…" -
You want to know both! • Performance is important, because
it defines a baseline for latency and capacity. • Scalability is important, because it will tell you if and how far you can grow. -
(Micro-)Services • promising approach to build better isolated systems •
supposedly helps to scale to many independent teams • supposedly helps to scale your operation • CAUTION! Complexity is not abundant, it is moved "between" the services -
What about Performance? • A (micro-)service architecture can have a
beneficial impact on performance… • but it can also be very dangerous! -
Huh? • Your SLA: "We need 500ms 95% of the
time" • Your endpoint requires a service, which requires a service, which in turn re……… • What happens to your SLAs? For three services in a chain? You are left with 500ms for 86% of the time! -
Challenges • Distributed teams build a distributed system • Communication
is the primary challenge • For non-functional requirements, you have to end up coordinating with other teams -
Lots of Open Questions • Test Environments? Test Data? •
Test Coordination? Supervision? Monitoring? • What about CI and CD? -
Empower Individuals as much as possible! • Everyone should be
able to run a test – anytime • You don't want have independent teams and then block them by orthogonal tasks like perf testing -
Work "Bottom-Up" • Start with very simple and basic tests
as building blocks • Continue with coarser grained tests, combining basic blocks • Only start with complex business flows in the end -
Why "Bottom-Up"? • Systems and stacks are complex • To
understand a system under load, you need to be able to identify what happens and you have to be able to reproduce a test. • Low-Level Tests can be performed by each team in isolation. -
Have Dedicated Test Environments • Ideally you have a 1:1
Environment for Performance Testing ($$$) • But you should have at least a scaled version of it and be able to size it up • Very, very important: Have a representative data set available! -
StormForger • SaaS Product and Consulting on Performance and Load
Testing • We take care of the heavy lifting and pave the way for regular performance assessments -
https://stormforger.com Thank you! How can we help you? Sebastian Cohnen
[email protected] +49 176 / 22 82 30 58 Lars Wolff [email protected] +49 177 / 6 00 97 80 StormForger GmbH, Kyffhäuserstr. 17, 50674 Köln/Germany. Registergericht: Amtsgericht Köln, HRB 82582. Geschäftsführer: Sebastian Cohnen, Lars Wolff -
— MANUEL KIESSLING, @MANUELKIESSLING HEAD OF TECHNICAL PLATFORM ONLINE GALERIA
KAUFHOF GMBH — RALF ROTTMANN, @RALF, MANAGING PARTNER GRANDCENTRIX GMBH „For the German interactive TV Show Quizduell im Ersten, we reached out to Sebastian Cohnen after the show started with some issues. Working closely together, we conducted large scale performance tests, to assess the ability of our system to withstand the anticipated load of 1 million users.“ — JÉRÔME PATT, PROJECT MANAGER RTL INTERACTIVE GMBH „We were actually quite relaxed during the show premiere. Everything worked flawlessly and our complex workflows performed pretty well. Sebastian provided a great deal of help with his professional guidance and the extensive load tests!“ — JOHANNES HÖRLE, LEAD SOFTWARE ENGINEER YOOCHOOSE GMBH „We at YOOCHOOSE use load tests to assess our recommendation engine and tracking APIs. Being able to send well defined load patterns against our systems and getting detailed analytics on service latency, throughput and error rates is a perfect fit for our performance analysis needs.“ „We used different open source and commercial tools to conduct performance and load tests. After creating reference tests we decided StormForger would be the right tool for the job. Performance analysis *is* a tool decision.” — MICHAEL WIRTH, CTO DEALINI AG „As a full-service provider for nation-wide special promotions for large retailers we are relying on a solid IT architecture to handle the load of our web and mobile solutions. I asked my development team to checkout our backend performance using StormForger. The short version: Thumbs up! We’ll use StormForger for every launch now. Thanks!“