Device Independent API design

With the explosion of consumer devices, designing flexible and device-independent optimized APIs has become increasingly challenging. This talk covers API design best practices and core principles for microservice and serverless architecture, demonstrates them using a custom PHP framework, and summarizes the lessons learnt. Additionally, the talk covers highly optimized API design using graphQL to address versioning issues and device resource constraints. Lastly, we cover key takeaways for implementing and migrating to these architectures.

Transcript

1. 1 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Device Independent API Design PRESENTED BY: AMRITA JAIN DIRECTOR, DIGITAL TECHNOLOGIES, BEACHBODY, LLC. MICROSERVICES+GRAPHQL+SERVERLESS
2. 2 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Agenda Mission Statement Milestone1: Microservices Milestone2: GraphQL Milestone3: Serverless Key Takeaways Q & A
3. 3 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Journey so far.. 1990s and earlier 2000s 2014s 2017s Monolithic Tight coupling Traditional SOA Loosely coupled Microservices Decoupled Serverless Cheap decoupling
4. 4 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Evolution of clients/consumption
5. 5 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Evolution of APIs/ Role of APIs https://www.blog.vba-market.com/2016/10/19/10-useful-real-world-api-examples/
6. 6 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN 0 2 4 6 8 10 12 14 16 18 20 Monolithic BE+FE Microservices Serverless Platforms Subscribers Team size Journey I took at Beachbody Evolution of API architecture with the growth in 1. Platforms (1 to 6) 2. Subscribers (200K to > 1.5M) 3. Team size (1 to 20) * Units in 100K 2014 2015 2016 2018
7. 7 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Design APIs that scales with traffic, platforms, team size and minimizes cost in terms of dollars, personnel and latency. Mission Statement
8. 8 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN “ ” Unix Philosophy Write programs that do one thing and do it well. Write programs to work together. Peter H. Salus Milestone: Microservices
9. 9 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN •Independently managed, clearly defined services. •Each microservice owns its data-source. •Better scaling, failover and recovery. •Flexibility in choice of language and deliverables. Microservices: Benefits
10. 10 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN •Redundant code for common functionality. •Inconsistency in common behaviors. •Harder for clients developers •Service Discovery •Data aggregation. •Performance and bandwidth constraints. •Automation, CI/CD is non negotiable. Microservices: Challenges
11. 11 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best practices: Get ahead of Heterogeneity Convention over configuration • Develop and adopt standards and conventions • Agree/Disagree but commit • Reduce decision-making for common trivial problems
12. 12 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best practices: Get ahead of Heterogeneity Custom MicroService Framework for PHP • Enforces same code flow for all MicroServices • Consistent error handling and response codes • Circuit Breaker Failover Handling • Provides utilities such as database, search engine, cache wrappers • Unifies security practices
13. 13 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best practices: Service Discovery API Gateway • Route to proper microservice • Rate Limiting • Tracing + Debugging + Monitoring • Caching + Invalidations My address ?
14. 14 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best Practice: Monitoring System-level monitoring ◦ CPU, Memory, I/O etc. ◦ StatsD, Graphite Application-level monitoring ◦ Exceptions, error logs APM ◦ Performance Issues & Bottlenecks Dashboards ◦ Graphana Which service caused failure?
15. 15 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN “ ” Evolving APIs without versions one vertex at a time Lee Byron Milestone: GraphQL
16. 16 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Device specific data • iTunes id, google account id etc. • Device-specific features eg. showing promo banner only on Android • Device specific images Response Parity
17. 17 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN { "images": { "badgeBoxArt": {}, "largeBoxArt": {}, "heroImageTablet": {}, "popularProgramMedium": {}, "devicesBannerImage": {}, "devicesStvBoxArt": {}, "devicesChromecast": {}, "devicesParallaxImage": {}, "devicesTabletHomepageLandscape1x": {}, "devicesTabletHomepageLandscape2x": {}, "devicesTabletProgramLandscape2x": {}, "devicesTabletProgramPortrait2x": {}, "devicesGlobalImage": {} }} Response Parity : Example
18. 18 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Performance tradeoffs • Clients that can parallelize prefer many calls smalls • Other clients prefer minimal data in one call. Bandwidth constraints • Prefers minimal data in one go. • Offline downloads, support batch Performance and Bandwidth
19. 19 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Always have to support all versions until all users have upgraded their apps. • Tedious & Expensive • Developer Time (∝ O(versions)) • Error Prone (∝ O(versions)) • Testing time (∝ O(versions × platforms x platformVersion)) API versioning
20. 20 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN GraphQL: In Action
21. 21 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN GraphQL: In Action
22. 22 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN ü Root node and think of data as graph. ü Implement Graphql schema. ü Define Query types: field on the query is processed ü Map the underlying API/Data ü Data manipulation can be done via resolvers ü Need help? ü Slack ü User Groups Implementation Tips
23. 23 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Implementation Tips Ref: https://github.com/webonyx/graphql-php/blob/master/examples/00-hello-world/graphql.php curl http://localhost:8080 -d '{"query": "query { echo(message: "Hello World") }" }'
24. 24 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Implementation Tips Ref: https://github.com/webonyx/graphql-php/blob/master/examples/00-hello-world/graphql.php curl http://localhost:8080 -d '{"query": "query { echo(message: "Hello World") }" }' {"data": {"echo": "You said Hello World" }}
25. 25 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best practices: Start Small Incremental Adoption • Start with one client • Start with APIs with more content- based data Image ref:https://www.youtube.com/watch?v=WQLzZf34FJ8
26. 26 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Best practices: KISS Principle Thin Interface • No extra business logic, framework stuff. • Depth and amount limiting • Query cost analysis and checks. 1 Request
27. 27 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Adding an extra layer and keeping it updated. Workarounds: • Apollo Graphql libs • Dynamic schema based on Swagger: https://github.com/amritajain/swagger-graphene • Start here: https://github.com/chentsulin/awesome-graphql Caveat: Extra Layer
28. 28 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN “ ” Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away Antoine de Saint-Exupery Milestone: Serverless
29. 29 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Serverless Benefits No Servers to provision or manage Only pay what you use for Standardization and ease of deployments Event driven Scaling, built-in availability and fault tolerance
30. 30 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Auto-scaling Websites and APIs • Serving up static content • Image and Video Manipulation • Event streaming • ETLs, Time-based batched jobs • 30 serverless architectures in 30 minutes Serverless: In Action
31. 31 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Serverless: Sample yaml file
32. 32 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Serverless: Sample yaml file serverless deploy --accountID <AWS::AccountId> --stage <dev|qa|prod> --deploymentBucket <bucketName>
33. 33 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Serverless: Sample yaml file
34. 34 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Serverless: Sample yaml file serverless deploy --accountID <AWS::AccountId> --stage <dev|qa|prod> --deploymentBucket <bucketName>
35. 35 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Avoid fat/monolithic functions • Optimize for language of choice • Cloud agnostic frameworks • Logging: Log-levels, capture device names, OS versions, and types in request headers • Store all environment variables separately and keep them encrypted. Best Practices
36. 36 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Cold starts • Invariable Demand • Local setups: 3rd-party libs that can help Common Pitfalls
37. 37 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN “ ” Nothing good comes easy because nothing easy does any good. Unknown Best Practices & Learnings
38. 38 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Follow microservice principle: One microservice, One repo, One datasource. • Clear lines of ownership • Core framework: injected via composer • Git Flow • Dev prod parity Learning: Codebase
39. 39 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Learning: Development Processes Integrated into CI process • Unittests • Integrated with Travis • Integration tests • All microservice collections include Postman tests • Used by devs to for quick validation. • Used by clients for parallel implementation
40. 40 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN • Swagger • Provide swagger documentation for every API • Baked into the framework • Lucidcharts • Sequence/Architecture diagrams in Lucidcharts. • Confluence Documentation • Must for handoff and demo. Learning: Development Processes
41. 41 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN § Setup patterns and standards on Day1. § Things change: what may seem right today might not be tomorrow, be ready to evolve. § Think of APIs as a product and make it a complete package. Key Takeaways
42. 42 DEVICE INDEPENDENT API DESIGN BY AMRITA JAIN Thank you! @amritajain973 https://joind.in/talk/a578f Currently @ Twitch!