Optimizing your API to Perform at Scale

Slide 1

Slide 1 text

Slide 2

Slide 2 text

© AKAMAI - EDGE 2017 WHOIS? Billy Hoffman Chief Technology Officer Rigor Manny Alvarez Enterprise Architect Akamai Technologies Jeffrey Costa Product Manager Akamai Technologies Chapman Lever Dir. of Alliances & Market Development Rigor @cosjef @md_a13 @zoompf @chapmanlever

Slide 3

Slide 3 text

Slide 4

Slide 4 text

© AKAMAI - EDGE 2017 Consumer-Focused API Design ● What we call API design: • It has to work. • It has to be RESTful. ● We think of our API users as developers. ● We empower devs to onboard and use our API. ● But we neglect to consider the ultimate consumer of our API. ● We focus on consumers for websites: how much time do we spend on CSS and UI/UX... ● Why build APIs differently? ● Focus on how your developer’s API client is going to perform calls. ● Your API consumers are your product - not the API itself.

Slide 5

Slide 5 text

© AKAMAI - EDGE 2017 What Happens If We Lose Focus On the API Client? ● History repeats itself. ● We make the same mistakes on APIs we made on websites 10 years ago: • Bloated response size • Too many round-trips • Not using cache layers • Ignoring low-hanging protocol fruit • Being needlessly stateful (cookies, anyone?) • Assuming that a network is reliable or has no latency • Forgetting that network performance is different for different types of clients.

Slide 6

Slide 6 text

© AKAMAI - EDGE 2017 How Did We Get Here? ● Use of default API implementations (looking at you, Oracle Endeca/ATG!) ● Use of out-of-the-box $Framework settings. ● Thinking: “It’s just plain text. What could go wrong?” ● Ignoring how changes to the API or its location, can affect users. ● Not monitoring performance metrics for API traffic. ● Ignoring how your API is being consumed: “I’m building an API for customers in Barcelona with a cluster located in US-WEST. Why are they complaining response time is so bad?”

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

© AKAMAI - EDGE 2017 Ignoring the Impact of Geography ● Do not ignore the situation of API client. ● How far away are your origin servers from your end users? ● Just like websites, the geographic distance between clients and servers can make a HUGE impact on API performance. Let’s take a look at some real world data for a modern mobile app….

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

© AKAMAI - EDGE 2017 Geographic Differences Creates Uneven UX ● Client A - averaged 124 ms round trip time to receive a payload from the origin server in Virginia ● Client B - averaged a round trip time of 223 ms or 80% slower ● Client C - averaged a round trip time of 517 ms or 316% slower

Slide 14

Slide 14 text

© AKAMAI - EDGE 2017 Geographic Differences Creates Uneven UX ● Client A (VA) - averaged 124 ms round trip time to receive a payload from the origin server in Virginia ● Client B () - averaged a round trip time of 223 ms or 80% slower ● Client C () - averaged a round trip time of 517 ms or 316% slower The Bay Area is their largest market!

Slide 15

Slide 15 text

© AKAMAI - EDGE 2017 How Did This Happen? ● Business evolved, technology did not ● As business expanded new customers complained via social networks about app slowness ● Existing monitoring systems showed no difference in latency or API processing ● Business was monitoring their system from the Southeast (origin server is in Virginia)

Slide 16

Slide 16 text

© AKAMAI - EDGE 2017 Measure as a User and Keep Data Close Action Items ● You cannot fix problems that are unknown ● Measure and monitor your services from the perspective of YOUR users ● Optimize your infrastructure for the needs of your customers • If the majority of your customer base is in California, origin servers should be in California ● Leverage a CDN to get data closer to your users if they are geographically dispersed

Slide 17

Slide 17 text

© AKAMAI - EDGE 2017 Latency Is Only ONE Component Of The User Experience ● Test Case Timings: • Client A - averaged 124 ms round trip time to receive a payload from the origin server in Virginia • Client B - averaged a round trip time of 223 ms or 80% slower • Client C - averaged a round trip time of 517 ms or 316% slower ● The timings above are not comprehensive of the entire UX. • These timings can be equated to the “time to first byte” of a webpage ● Once an API payload is received it still has to be processed by the Client which brings us to the next problem….

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© AKAMAI - EDGE 2017 GZIP Action Items 1. Audit the MIME types you are using. 2. Review your Akamai config - are they all in there? 3. Review your origin config 4. Consider defining a standard used for API responses a. “Mime type shall always be ‘application/json’ henceforth”

Slide 26

Slide 26 text

© AKAMAI - EDGE 2017 HTTP/1.1 Is Suboptimal For APIs ● Can’t compress request bodies • POSTS and PUTs can have big request bodies ● Can’t compress headers • Cookies, other state mechanisms can be large • Resending duplicate data ● Makes Poor Use of Connections • Everything happens in series • Head of line blocking

Slide 27

Slide 27 text

© AKAMAI - EDGE 2017 ● Header deduplication ● Binary, not textual, framing format ● Multiplexable connections ● Typical API request pattern is for calls to happen in parallel ● Effective when processing a large number of requests. ● Remember that H2 requires SSL (de-facto for most APIs today) HTTP/2: Better Use of Connections

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

© AKAMAI - EDGE 2017 Why Cache? 1. Performance! • It’s faster to deliver an IMS response than to generate and deliver a new response. 2. Scalability • Most API operations are read operations. • Why go to the expensive database every time to grab a list of available countries or currencies - that almost never change? 3. Costs. • Cloud providers charge by IOPS and instance size • How much compute power (CAPEX and OPEX) could you turn off by caching? • You pay a round trip & processing cost for EVERY request/response. 4. Avoid instability due to network latencies across geographic locations. 5. Be better prepared for high traffic volumes. • Rate limiting is a legacy constraint. It continues to live on unquestioned. • Reduce your reliance on rate limiting.

Slide 34

Slide 34 text

© AKAMAI - EDGE 2017 1) Any resource accessible via GET. 2) Static or immutable data. 3) Responses used by many clients (frequently-requested data). 4) Content used by multiple partners. 5) Responses that change infrequently, or at predictable intervals. What to Cache?

Slide 35

Slide 35 text

© AKAMAI - EDGE 2017 1) Personalized data. 2) Sensitive data. 3) Personally Identifiable Information (PII). 4) Protected Health Information (PHI or HIPAA). 5) Any data held under compliance standards. What NOT to Cache?

Slide 36

Slide 36 text

© AKAMAI - EDGE 2017 How To Cache On the Client? ● Ensure the API client respects the HTTP spec ● Use one of these: • Cache-Control • Expires • ETag • If-Modified-Since • Vary* ● Turning Cache OFF: Cache-Control: no-cache, no-store *Note: Vary makes the object not cacheable on Akamai

Slide 37

Slide 37 text

© AKAMAI - EDGE 2017 Define the TTL rules on your Akamai configuration: ● Using Luna GUI ● Using PAPI (Property Manager API) ● Enable “serve stale” if unable to validate! Define TTL rules with your downstream cache headers ● Honor Origin Cache-Control and Expire headers ● Use the Edge-Control header Edge-Control: !no-store, max-age=6d, downstream-ttl=1h How To Cache On Akamai?

Slide 38

Slide 38 text

Slide 39

Slide 39 text

© AKAMAI - EDGE 2017 Content Segmentation in Cache Client URL Resp time Hit? Client A GET /turtles?fields=name,lifespan 200 ms No Client B GET /turtles?fields=name 192 ms No Client URL Resp time Hit? Client A GET /cheetah 200 ms No Client B GET /cheetah 118 ms YES! 46% speed up with cache hit! 4% speed up but cache miss Consistency means better cache-ability

Slide 40

Slide 40 text

© AKAMAI - EDGE 2017 Normalizing Cache 1. Compare your responses to different clients: • Github: api-comparison-tool • NodeJS: node-rest-diff 2. Normalize your API request at the client if possible. 3. Leverage Dynamic Page Caching (DPC) to segregate cache based on request elements. 4. Do not include timestamp on your URLs. 5. Beware use of keys/tokens in the URI that kill cacheability; e.g. client ID. 6. Don’t mix private or user specific data with public data. 7. The order of query string arguments impacts cache. Source: Phil Sturgeon

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

© AKAMAI - EDGE 2017 What Have We Learned? ● The techniques that make websites fast can also make APIs fast. ● API caching is a proven, well known technology to increase the scalability and performance of your APIs. ● Do more with fewer resources. ● This takes some investment and awareness: • Get to really know the resources you are serving up. • Understand how your consumers are putting those resources to work. ● Good API design transcends code.

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Slide 47

Slide 47 text

© AKAMAI - EDGE 2017 What Have We Learned? ● The techniques used to make websites fast can also make APIs fast. ● Account for geographical differences • Figure out where your end users are coming from. • Measure from their perspective • Optimize Akamai and your infrastructure to get closer to them ● Review the data you are sending • Audit your mime types • Review your Akamai and origin configs • Turn on H2, turn on GZIP ● Negate the need to send data • Consider what to cache • Normalize to increase cache-abilty

Slide 48

Slide 48 text

© AKAMAI - EDGE 2017 Reducing number of API Calls ● Property Manager API: creating and activating a new version Get Contracts $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/contracts/ Get Groups $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/groups Find Configuration $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/properties/ contractId==… Get Configuration $ http --auth-type edgegrid -a DevRelMD: :/papi/v0/properties/prp_348544/versions/9/rules contractId==… Create New Version $ http -v --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/versions/ contractId==… Save the new Version $ http --auth-type edgegrid -a DevRelMD: PUT :/papi/v0/properties/prp_348544/versions/10/rules… Get Warnings $ http --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/activations/ contractId==ctr_C-1FRYVV3… Activate $ http --auth-type edgegrid -a DevRelMD: POST :/papi/v0/properties/prp_348544/activations/ contractId==ctr_C-1FRYVV3…

Slide 49

Slide 49 text

© AKAMAI - EDGE 2017 Reducing number of API Calls ● Akamai CLI: creating and activating a new version Get Configuration $ akamai property retrieve --section DevRelMD --file 'all-mainsitedemo.json' all- mainsitedemo Create New Version $ akamai property update --section DevRelMD --file 'all-mainsitedemo.json' all- mainsitedemo Activate $ akamai property activate --section DevRelMD --network STAG --propver latest all- mainsitedemo --email '[email protected]'

Slide 50

Slide 50 text

© AKAMAI - EDGE 2017 ETag ● ETags are unique identifiers for a particular version of a resource found by a URL. They are used for cache validation, to check for modifications quickly. ● Every API response to a cacheable request SHOULD include the ETag header to identify the specific version of the returned resource. ● Every API client SHOULD use the If-None-Match header whenever it is performing a cacheable request. The value of If-None-Match should be the value of the ETag header stored from a previous request. ● The client must be ready to handle the 304 Not Modified response from the server to use the legal copy.

Slide 51

Slide 51 text

© AKAMAI - EDGE 2017 Consumer-Focused API Design ● What we call API design: • It has to work. • It has to be RESTful. ● We tend to think like engineers - not users. ● We make it smooth and enjoyable for devs to consume our API. ● But we often neglect to consider the ultimate consumer of our API. ● Compare this to how we build websites: how much time is spent on CSS and UX/UI for the ultimate consumer? ● Why build APIs differently? ● Focus on how your API client is going to perform calls.