Optimizing your API to Perform at Scale

Transcript

1. © AKAMAI - EDGE 2017 Optimizing your API to Perform at Scale Rigor & Akamai
2. © AKAMAI - EDGE 2017 WHOIS? Billy Hoffman Chief Technology Officer Rigor Manny Alvarez Enterprise Architect Akamai Technologies Jeffrey Costa Product Manager Akamai Technologies Chapman Lever Dir. of Alliances & Market Development Rigor @cosjef @md_a13 @zoompf @chapmanlever
3. © AKAMAI - EDGE 2017 Agenda • Consumer-focused API design • 3 Pillars of API optimization • Action items for Monday morning
4. © AKAMAI - EDGE 2017 Consumer-Focused API Design • What we call API design: • It has to work. • It has to be RESTful. • We think of our API users as developers. • We empower devs to onboard and use our API. • But we neglect to consider the ultimate consumer of our API. • We focus on consumers for websites: how much time do we spend on CSS and UI/UX... • Why build APIs differently? • Focus on how your developer’s API client is going to perform calls. • Your API consumers are your product - not the API itself.
5. © AKAMAI - EDGE 2017 What Happens If We Lose Focus On the API Client? • History repeats itself. • We make the same mistakes on APIs we made on websites 10 years ago: • Bloated response size • Too many round-trips • Not using cache layers • Ignoring low-hanging protocol fruit • Being needlessly stateful (cookies, anyone?) • Assuming that a network is reliable or has no latency • Forgetting that network performance is different for different types of clients.
6. © AKAMAI - EDGE 2017 How Did We Get Here? • Use of default API implementations (looking at you, Oracle Endeca/ATG!) • Use of out-of-the-box $Framework settings. • Thinking: “It’s just plain text. What could go wrong?” • Ignoring how changes to the API or its location, can affect users. • Not monitoring performance metrics for API traffic. • Ignoring how your API is being consumed: “I’m building an API for customers in Barcelona with a cluster located in US-WEST. Why are they complaining response time is so bad?”
7. © AKAMAI - EDGE 2017 The 3 Pillars of API Optimization 1. Account for geographical differences. 2. Review the data you are sending. 3. Negate the need to even send that data.
8. © AKAMAI - EDGE 2017 Geographic Issues
9. © AKAMAI - EDGE 2017 Ignoring the Impact of Geography • Do not ignore the situation of API client. • How far away are your origin servers from your end users? • Just like websites, the geographic distance between clients and servers can make a HUGE impact on API performance. Let’s take a look at some real world data for a modern mobile app….
10. © AKAMAI - EDGE 2017 API Client “A” in Virginia
11. © AKAMAI - EDGE 2017 API Client “B” in Chicago
12. © AKAMAI - EDGE 2017 API Client “C” in Northern California
13. © AKAMAI - EDGE 2017 Geographic Differences Creates Uneven UX • Client A - averaged 124 ms round trip time to receive a payload from the origin server in Virginia • Client B - averaged a round trip time of 223 ms or 80% slower • Client C - averaged a round trip time of 517 ms or 316% slower
14. © AKAMAI - EDGE 2017 Geographic Differences Creates Uneven UX • Client A (VA) - averaged 124 ms round trip time to receive a payload from the origin server in Virginia • Client B () - averaged a round trip time of 223 ms or 80% slower • Client C () - averaged a round trip time of 517 ms or 316% slower The Bay Area is their largest market!
15. © AKAMAI - EDGE 2017 How Did This Happen? • Business evolved, technology did not • As business expanded new customers complained via social networks about app slowness • Existing monitoring systems showed no difference in latency or API processing • Business was monitoring their system from the Southeast (origin server is in Virginia)
16. © AKAMAI - EDGE 2017 Measure as a User and Keep Data Close Action Items • You cannot fix problems that are unknown • Measure and monitor your services from the perspective of YOUR users • Optimize your infrastructure for the needs of your customers • If the majority of your customer base is in California, origin servers should be in California • Leverage a CDN to get data closer to your users if they are geographically dispersed
17. © AKAMAI - EDGE 2017 Latency Is Only ONE Component Of The User Experience • Test Case Timings: • Client A - averaged 124 ms round trip time to receive a payload from the origin server in Virginia • Client B - averaged a round trip time of 223 ms or 80% slower • Client C - averaged a round trip time of 517 ms or 316% slower • The timings above are not comprehensive of the entire UX. • These timings can be equated to the “time to first byte” of a webpage • Once an API payload is received it still has to be processed by the Client which brings us to the next problem….
18. © AKAMAI - EDGE 2017 APIs Are OBESE
19. © AKAMAI - EDGE 2017 “Fat” APIs vs. “Fat” Web Pages JSON XML (Akamai API Statistics, May 2017)
20. © AKAMAI - EDGE 2017 More than 18,000 lines of JSON in 488 Kb
21. © AKAMAI - EDGE 2017 Needless Data Bloat • Overly Verbose • Duplicate or repeated data • Large amounts of unused data • Sending NULL values
22. © AKAMAI - EDGE 2017 Optimize Response Bodies with GZIP • JSON/XML is... • Text-based • Lots of redundancy • Perfect for GZIP!
23. © AKAMAI - EDGE 2017 Original Request Without Compression https://manuel.edge.edgesuite.net/storeinfo/api/stores Add a wildcard match
24. © AKAMAI - EDGE 2017 Subsequent Request With Compression https://manuel.edge.edgesuite.net/storeinfo/api/stores
25. © AKAMAI - EDGE 2017 GZIP Action Items 1. Audit the MIME types you are using. 2. Review your Akamai config - are they all in there? 3. Review your origin config 4. Consider defining a standard used for API responses a. “Mime type shall always be ‘application/json’ henceforth”
26. © AKAMAI - EDGE 2017 HTTP/1.1 Is Suboptimal For APIs • Can’t compress request bodies • POSTS and PUTs can have big request bodies • Can’t compress headers • Cookies, other state mechanisms can be large • Resending duplicate data • Makes Poor Use of Connections • Everything happens in series • Head of line blocking
27. © AKAMAI - EDGE 2017 • Header deduplication • Binary, not textual, framing format • Multiplexable connections • Typical API request pattern is for calls to happen in parallel • Effective when processing a large number of requests. • Remember that H2 requires SSL (de-facto for most APIs today) HTTP/2: Better Use of Connections
28. © AKAMAI - EDGE 2017 HTTP 1.1 HTTP 2.0 Single Page App API On HTTP 1.1 and HTTP/2
29. © AKAMAI - EDGE 2017 HTTP 1.1 HTTP 2.0 Single Page App API On HTTP 1.1 and HTTP/2 Reduces Data Sent by 20%
30. © AKAMAI - EDGE 2017 HTTP/2 Action Items 1. Consider using it on the API clients you control (libraries) 2. Enable it at origin (NGINX, Apache) 3. Turn on in your Akamai configuration
31. © AKAMAI - EDGE 2017 APIs Are S-L-O-W
32. © AKAMAI - EDGE 2017 APIs Are S-L-O-W Cache it!
33. © AKAMAI - EDGE 2017 Why Cache? 1. Performance! • It’s faster to deliver an IMS response than to generate and deliver a new response. 2. Scalability • Most API operations are read operations. • Why go to the expensive database every time to grab a list of available countries or currencies - that almost never change? 3. Costs. • Cloud providers charge by IOPS and instance size • How much compute power (CAPEX and OPEX) could you turn off by caching? • You pay a round trip & processing cost for EVERY request/response. 4. Avoid instability due to network latencies across geographic locations. 5. Be better prepared for high traffic volumes. • Rate limiting is a legacy constraint. It continues to live on unquestioned. • Reduce your reliance on rate limiting.
34. © AKAMAI - EDGE 2017 1) Any resource accessible via GET. 2) Static or immutable data. 3) Responses used by many clients (frequently-requested data). 4) Content used by multiple partners. 5) Responses that change infrequently, or at predictable intervals. What to Cache?
35. © AKAMAI - EDGE 2017 1) Personalized data. 2) Sensitive data. 3) Personally Identifiable Information (PII). 4) Protected Health Information (PHI or HIPAA). 5) Any data held under compliance standards. What NOT to Cache?
36. © AKAMAI - EDGE 2017 How To Cache On the Client? • Ensure the API client respects the HTTP spec • Use one of these: • Cache-Control • Expires • ETag • If-Modified-Since • Vary* • Turning Cache OFF: Cache-Control: no-cache, no-store *Note: Vary makes the object not cacheable on Akamai
37. © AKAMAI - EDGE 2017 Define the TTL rules on your Akamai configuration: • Using Luna GUI • Using PAPI (Property Manager API) • Enable “serve stale” if unable to validate! Define TTL rules with your downstream cache headers • Honor Origin Cache-Control and Expire headers • Use the Edge-Control header Edge-Control: !no-store, max-age=6d, downstream-ttl=1h How To Cache On Akamai?
38. © AKAMAI - EDGE 2017 Impact of Caching at Akamai https://manuel.edge.edgesuite.net/storeinfo/api/stores No Store was above 1s
39. © AKAMAI - EDGE 2017 Content Segmentation in Cache Client URL Resp time Hit? Client A GET /turtles?fields=name,lifespan 200 ms No Client B GET /turtles?fields=name 192 ms No Client URL Resp time Hit? Client A GET /cheetah 200 ms No Client B GET /cheetah 118 ms YES! 46% speed up with cache hit! 4% speed up but cache miss Consistency means better cache-ability
40. © AKAMAI - EDGE 2017 Normalizing Cache 1. Compare your responses to different clients: • Github: api-comparison-tool • NodeJS: node-rest-diff 2. Normalize your API request at the client if possible. 3. Leverage Dynamic Page Caching (DPC) to segregate cache based on request elements. 4. Do not include timestamp on your URLs. 5. Beware use of keys/tokens in the URI that kill cacheability; e.g. client ID. 6. Don’t mix private or user specific data with public data. 7. The order of query string arguments impacts cache. Source: Phil Sturgeon
41. © AKAMAI - EDGE 2017 https://manuel.edge.edgesuite.net/premier/api/us/en/rc806881/ProductDetail s/s009l558015usr?zebra=no&octopus=yes&apple=no&cow=yes Normalizing Cache
42. © AKAMAI - EDGE 2017 Normalizing Cache
43. © AKAMAI - EDGE 2017 What Have We Learned? • The techniques that make websites fast can also make APIs fast. • API caching is a proven, well known technology to increase the scalability and performance of your APIs. • Do more with fewer resources. • This takes some investment and awareness: • Get to really know the resources you are serving up. • Understand how your consumers are putting those resources to work. • Good API design transcends code.
44. © AKAMAI - EDGE 2017
45. © AKAMAI - EDGE 2017 Appendix
46. © AKAMAI - EDGE 2017 Slow API Examples
47. © AKAMAI - EDGE 2017 What Have We Learned? • The techniques used to make websites fast can also make APIs fast. • Account for geographical differences • Figure out where your end users are coming from. • Measure from their perspective • Optimize Akamai and your infrastructure to get closer to them • Review the data you are sending • Audit your mime types • Review your Akamai and origin configs • Turn on H2, turn on GZIP • Negate the need to send data • Consider what to cache • Normalize to increase cache-abilty
48. © AKAMAI - EDGE 2017 Reducing number of API Calls • Property Manager API: creating and activating a new version Get Contracts $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/contracts/ Get Groups $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/groups Find Configuration $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/properties/ contractId==… Get Configuration $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/properties/prp_348544/versions/9/rules contractId==… Create New Version $ http -v --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/versions/ contractId==… Save the new Version $ http --auth-type edgegrid -a DevRelMD: PUT :/papi/v0/properties/prp_348544/versions/10/rules… Get Warnings $ http --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/activations/ contractId==ctr_C-1FRYVV3… Activate $ http --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/activations/ contractId==ctr_C-1FRYVV3…
49. © AKAMAI - EDGE 2017 Reducing number of API Calls • Akamai CLI: creating and activating a new version Get Configuration $ akamai property retrieve --section DevRelMD --file 'all-mainsitedemo.json' all- mainsitedemo Create New Version $ akamai property update --section DevRelMD --file 'all-mainsitedemo.json' all- mainsitedemo Activate $ akamai property activate --section DevRelMD --network STAG --propver latest all- mainsitedemo --email 'm@akamai.com'
50. © AKAMAI - EDGE 2017 ETag • ETags are unique identifiers for a particular version of a resource found by a URL. They are used for cache validation, to check for modifications quickly. • Every API response to a cacheable request SHOULD include the ETag header to identify the specific version of the returned resource. • Every API client SHOULD use the If-None-Match header whenever it is performing a cacheable request. The value of If-None-Match should be the value of the ETag header stored from a previous request. • The client must be ready to handle the 304 Not Modified response from the server to use the legal copy.
51. © AKAMAI - EDGE 2017 Consumer-Focused API Design • What we call API design: • It has to work. • It has to be RESTful. • We tend to think like engineers - not users. • We make it smooth and enjoyable for devs to consume our API. • But we often neglect to consider the ultimate consumer of our API. • Compare this to how we build websites: how much time is spent on CSS and UX/UI for the ultimate consumer? • Why build APIs differently? • Focus on how your API client is going to perform calls.