Performance Testing sucks! How to get started testing RESTful APIs and GraphQL

GraphQL is a newer query language released by Facebook in 2015 while REST is around since 2000 and a paradigm for stateless distributed systems. REST is considered as a defacto standard for building HTTP-APIs and today some see GraphQL as a replacement for this.
But is GraphQL really a replacement for REST? It can be taken for granted, that both GraphQL and RESTful APIs have its very own justification. As so often it’s all about context: when to decide for REST or when for GraphQL or even for both?
Aside from deciding for a concept when building apps and fit functional requirements we are forced to consider non-functional requirements for web scale systems, too. What are the challenges to face for API design and in terms of performance in distributed scalable systems?
The talk points out what to consider when using REST or GraphQL and what to take care of. It shows best practices for an Agile and DevOps environment and introduces to the common pitfalls when it comes to performance of systems build on RESTful API’s and GraphQL.

Please sent feedback to support@stormforger.com or https://twitter.com/larsvegas

Transcript

1. stormforger.com - stormforger.com - stormforger.com stormforger.com - stormforger.com - stormforger.com
2. EXPLICIT EVOLVING PARENTAL ADVISORY
3. Performance Testing sucks! 
 How to get started testing
 RESTful APIs and GraphQL
4. EHLO[1] Lars Wolff
 
 
 @larsvegas • support@stormforger.com • Co-founder & CEO of
 stormforger.com • Software Development of web based systems • Agile Hippie (Coach)
 Certified ScrumMaster®, Kanban Management Professional • Amazon Web Services User Group Cologne
 #AWSUGCGN • Web Performance User Group Cologne [1] http://www.ietf.org/rfc/rfc2821.txt
5. APIs
 are eating the world
6. REST
7. REST • Representational state transfer • Defined by Roy Fielding REST in his 2000 PhD dissertation • Comes often with Hypermedia and HATEOAS in XML or JSON • Out-of-the Box implementations for pretty much every development framework and language
8. REST Representation of a resource
 with its properties and
 probably links to its entities and related resources.
9. REST vs. RESTful
10. RESTful curl -X GET https://host/api/people/14/
11. RESTful { "name": "Han Solo", "height": "180", "mass": "80", "homeworld": "https://host/api/planets/22/", "films": [ "https://host/api/films/2/", "https://host/api/films/3/", "https://host/api/films/1/", "https://host/api/films/7/" ], "created": "2014-12-10T16:49:14.582000Z", "edited": "2014-12-20T21:17:50.334000Z", "url": “https://host/api/people/14/" }
12. GraphQL
15. GraphQL • Started internally at Facebook called “SuperGraph” in 2012,
 released as GraphQL in 2015 • Plenty of adoption: Facebook of course, GitHub, Shopify, Pinterest, … • Client and server libraries for all the major development frameworks and languages
16. GraphQL A query language
 to your API
17. { person(id: “14") { name height mass homeworld { name id } filmConnection { edges { node { id } } } } } GraphQL
18. GraphQL curl -X GET ‘https://host/graphql/? query=%20%7B%20human(id%3A%20%2214%22)%20%7B%20id%20species%20name%20friends%20%7B %20id%20name%20%7D%20appearsIn%20%7B%20id%20title%20%7D%20homePlanet%20%7D%20%7D%2 0'
19. { "data": { "human": { "id": “14", "name": "Han Solo", "species": "HUMAN", "appearsIn": [ { "id": "Awakens", "title": "The Force Awakens" }, { "id": "Empire", "title": "The Empire Strikes Back" }, // […] ] } } } GraphQL
20. –THE INTERWEBS “GraphQL is the new REST”
21. Are
 REST and GraphQL even comparable?
22. Nay!
23. Comparison Phil Sturgeon wrote in “GraphQL vs REST: Overview”:
 • GraphQL isn't a magic bullet, nor is it “better" than REST • GraphQL is dope if used for the right thing https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/
24. Comparison REST and GraphQL are totally different https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/
25. REST is an architectural concept. GraphQL is a query language, specification and collection of tools.
26. What’s the problem? Performance Testing
27. The Cloud™ scales…
28. …and stuff still breaks!
29. What’s the problem? • We build complex, distributed systems • We run that stuff in “unknown” environments (a.k.a production) • We build it, so we think we know what’s going to happen there
30. What’s the problem? • We have real users in production OMG^^ • We have side-effects in production • We have bugs in production (configuration, software, OS, network)
 – You! Not me!
31. How can Performance Testing help me with all that?
32. Performance Testing Are testing methods to stimulate a system with a defined workload • Discover its behaviour • Validate and verify characteristics • Learn about and understand
 it’s behaviour
33. Performance Testing Load Testing Stress Testing Spike Testing Soak Testing Endurance Testing Resilience Testing Configuration Testing Scalability Testing
34. How to do Performance Testing?
35. How to do Performance Testing? • Set test goals • Create test definition • Execute the test • Interpretation of test results
36. How to do Performance Testing • set test goals • set non-functional requirements & SLAs • plan tests • create test definition • provision test resources
 (aka servers executing the tests) • setup and deploy the test case • execute the test • monitor the running test • tear down the test environment • gather and analyze log and other sensor data • interpretation of test results • evaluate & compare results to previous test runs • … https://stormforger.com/blog/2014/02/06/introducing-stormforger/
37. Tools • Apache JMeter (Java) • Gatling (Scala) • Tsung (Erlang) • …
38. Tools
39. Let’s do this
40. Star Wars data set
41. User Journey, Goal & NFR’s • User Journey:
 We are all fans of Han Solo.
 Get data about Han Solo,
 find four Episodes Han Solo appeared in with all detailed information • Goal:
 Do not violate of any the non-functional requirements (NFR) • Non-functional requirement #1 (NFR-1):
 Very most of all responses < 1000ms… • Non-functional requirement #2 (NFR-2):
 Very most of all responses of films < 500ms…
42. Test Case Definition
43. Test case definition for RESTful API definition.session("han-solo-user-journey", function(session) { session.get("/api/people/14/", { tag: "han-solo", extraction: { jsonpath: { firstFilm: "films[0]", secondFilm: "films[1]", thirdFilm: "films[2]", fourthFilm: "films[3]", } } }); session.get(session.matchedValue("firstFilm");, { tag: “films" }); session.get(session.matchedValue("secondFilm"), { tag: “films" }); session.get(session.matchedValue("thirdFilm"), { tag: “films" }); session.get(session.matchedValue("fourthFilm"), { tag: “films" }); });
44. Test case definition for GraphQL definition.session("han-solo-user-journey-graphql", function(session) { var graphqlQuery = ` { human(id: “14") { id name species appearsIn { id title } } } `.replace(/\n/g, '').replace(/\s{2,}/g, ' '); graphqlQuery = encodeURIComponent(graphqlQuery); session.get("/graphql?query=:query", { tag: “graphql-query", params: { query: graphqlQuery } }); });
45. Test run execution
46. Interpreting results
47. Interpreting results
48. Interpreting results
49. What to consider when Performance Testing RESTful APIs and GraphQL?
50. Differences for Performance Testing • You may need separate test cases for RESTful and GraphQL • non-functional requirements for GraphQl are different/more complicated.
51. Differences for Performance Testing • How to test fragments (e.g. Films) of a GraphQL query? • You must define a new query, which is a new “experiment”
52. Things to Consider in Terms of Performance Requests • A lot of requests in naiv implementations of RESTful • One request with GraphQL for all needed data • But you may blow up payloads
53. Things to Consider in Terms of Performance Caching • RESTful profits from all HTTP Caching mechanisms • GraphQL less
 e.g. queries can be long, what proxies don’t like • Every different query leads to a new cache bucket in the backend. Invalidation?
54. Optimizations • Optimize code in RESTful architectures seems easier
 (µ-)-service, aggregate, repository, … • At GraphQL you definitely need request tracing, to see affected components
55. Things to consider in terms of performance Isolation of expensive service requests/or versions
 (e.g. different infrastructure) • RESTful: e.g. Routing of requests • GraphQL: Acts like a gateway so routing is hard(er) ¯\_(ϑ)_/¯
56. The complete guide to Performance Testing™
57. The complete guide to Performance Testing™ 1. Define goals (Somebody has to start! Now! Really, start now!) 2. Who must be involved to match this goal from management, marketing, project management, product? 3. Is your goal still the same with all those sections? Redefine and improve it! 4. Define client journeys/ user journeys/ customer journeys. 5. Define non-functional requirements for test objects used in the journeys.
 Create and set up a PerfTest environment 6. Define the System under test (SUT). 7. Who must be involved to access and test the SUT from frontend development, backend development, QA, DBA, operations, monitoring, business analytics, business intelligence, tracking? 8. Create a WAR ROOM, meaning a (virtual) place where you’re all together. Have a chat-channel. Have regular head-ups. 9. As an agile team: create a community of practise (CoP) “Performance Testing”. 10. Know the differences between your PerfTest environment and production in terms of sizing, configuration and debug tools. 11. If you read this: yell out loud “Cheesecake” and get a StormForger Plan for free. 12. Raise figures of traffic schemas and services usage. 13. Create a model of usage and therefore created traffic in the user journey including arrival phases of new clients and concurrent clients. 14. Define a performance budgets and headrooms. 15. Start testing slowly with the most simple journey.
 16. Iterate over test case definitions. 17. Iterate over non-functional requirements. 18. Iterate over goals. 19. Attach learned performance metrics to your usage model, performance budgets, head rooms. 20. Documentation and Reporting? Yes, but no. 21. There is much, much more. 22. This ain’t complete. 23. I’m just tired. 24. Perf test early – Perf test often!

58. When to choose RESTful and/or GraphQL?
59. When to choose RESTful and/or GraphQL? • GraphQL is nice from the client/product development perspective • RESTful is nice to challenge a service oriented architectur and best practices are around for 15+ years
60. When choose RESTful and/or GraphQL? Your organization must be able to adopt the technologies • Small team with client heavy product?
 Go for GraphQL! Be ready to challenge backend problems while scaling • Heavy stuff in the backend?
 Clearly define business capabilities and go for RESTful
61. When to choose RESTful and/or GraphQL? Which problem do you need to solve?
62. When to choose RESTful and/or GraphQL? Do you have the same problems as the Facebooks and GitHubs?
63. Lars Wolff @larsvegas support@stormforger.com Thank you! Questions? We empower organizations to do continuous performance testing