Scaling API Reviews

How do you maintain the quality of colleague API reviews when you scale the number of feature teams? In this talk, I look at some tips on providing efficient, consistent API design reviews at scale.

Transcript

1. Scaling API reviews Ikenna Nwaiwu Presentation to Just Eat Takeaway (JET) 9th November 2023
2. Agenda Intro to APIOps Scaling API reviews API conformance Summary About me 2 3 4 5 1
3. About me 1 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1
4. APIOps Lead API DESIGN GOVERNANCE API COMMUNITY OF PRACTICE ENABLING API TEAMS AUTHOR: AUTOMATING API DELIVERY Linkedin.com/in/ikenna
5. Intro to APIOps 2 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1
6. The API delivery problem Increase speed of API design and delivery (Lower lead time) Consistency with API style guide. Quality API documentation. Well- tested, secure APIs. (Higher quality) Tension
7. What is APIOps? APIOps is the end-to-end automation of the API life cycle using DevOps and GitOps principles to help API providers deliver more value to API consumers faster
8. What is APIOps?
9. APIOps Principles
10. Process time for API reviews in APIOps
11. Example APIOps work fl ow
12. GitOps API Con fi g deployment
13. Scaling API reveiws 3 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1
14. Problem: How do you maintain the quality of colleague API reviews when you have 30 teams instead of 3?
15. Countermeasure: Move from centralize colleague reviews to federated colleague reviews
16. Centralized reviews
17. Federated API reviews
18. But how do you maintain colleague review consistency with federated reviews?
19. 3 areas of focus B. De fi ne important elements of API review process document A. De fi ne a process for becoming a reviewer C. Provide general API training
20. Focus 1: De fi ne the process for becoming a reviewer B. Training sessions A. Coaching C. Use federated and central approvals
21. Coaching Kubernetes API Review Process document https://github.com/kubernetes/community/blob/master/sig-architecture/ api-review-process.md Focus 1: De fi ne the process for becoming a reviewer
22. “To become an API reviewer on the Kubernetes project, aspiring reviewers must gain a high level of pro fi ciency in the Kubernetes API style guide. They also need to have a good understanding of the project structure and the system architecture. They are required to have participated in preliminary API reviews where mentors coach them. Mentors help train them and facilitate their inclusion as formal reviewers.” - From ‘Automating API Delivery’ by Ikenna Nwaiwu Focus 1: De fi ne the process for becoming a reviewer Coaching
23. Run training sessions for API reviewers on API style guide, API spec repo structure, API spec pipeline, API architecture, API spec publishing process, linting rules, API review document, expectations from API reviews, and so on. Training sessions Focus 1: De fi ne the process for becoming a reviewer
24. Training sessions Focus 1: De fi ne the process for becoming a reviewer Run periodic retrospectives with API reviewers.
25. Focus 1: De fi ne the process for becoming a reviewer Use federated and central PR approvals AT 10x, we have two levels of API review PR approvals: central and team level. As an Engineering Manager, you are automatically assigned as an API reviewer for your team's APIs.
26. Focus 2: De fi ne important elements of the API review process document B. Process KPIs A. Process principles C. Disagreement resolution hierarchy
27. A. Process principles Focus 2: De fi ne important elements of the API review process document API review process document should specify guiding principles so federated reviewers can refer to them. Suggested: quality, speed, dependability, fl exibility, and cost.
28. 1. Quality Improve API design quality and consistency with standards. Improve API documentation.
29. 2. Speed Reduce the time required to review and approve a design change
30. 3. Dependability To enable planning, set expectations on how long it takes to complete an API review.
31. 4. Flexibility Should explain the exception process. Di ff erent levels of reviews
32. Zulian & Bouza’s hierarchy of API design principles From ‘API Product Management’ by Andrea Zulian and Amancio Bouza
33. 5. Cost Aim to lower cost. Reviewers should suggest new automated checks where possible. Also, aim to document new ideas in the style guide.
34. B. Process KPIs Focus 2: De fi ne important elements of the API review process document Should specify API review process KPIs. 6 suggested: PR TTFRC, Number of Open PRs, PR lifetime, Review process satisfaction, and number of suggested improvements.
35. 1 B. Process KPIs PR TTFRC Time To First Review Comment or approval
36. 2 B. Process KPIs Number of open PRs
37. 3 B. Process KPIs PR lifetime from creation to merge
38. 4 B. Process KPIs API designer satisfaction using regular review process satisfaction surveys
39. 5 B. Process KPIs Number of suggested improvements to API style guide, linting rules, API process
40. These are KPIs I suggest, but a more rigorous approach is to think of KPIs that fall into the SPACE framework dimensions
41. SPACE Developer Productivity Metrics - ‘The SPACE of Developer Productivity’ by Nicole Forsgren et al. https:// queue.acm.org/detail.cfm?id=3454124 “The most important takeaway from exposing these myths is that productivity cannot be reduced to a single dimension (or metric!). The prevalence of these myths and the need to bust them motivated our work to develop a practical multidimensional framework, because only by examining a constellation of metrics in tension can we understand and in fl uence developer productivity.”
42. SPACE Developer Productivity Metrics ‘The SPACE of Developer Productivity’ by Nicole Forsgren et al. https://queue.acm.org/ detail.cfm?id=3454124
43. Should provide clear escalation process. C. Resolution hierarchy Eric Brewer https://google.aip.dev/1 Joao Prado
44. Focus 3: General API- fi rst training
45. Audience: API designers / Engrs, POs, BAs Focus 3: General API- fi rst training
46. When: on joining the company Focus 3: General API- fi rst training
47. Content: How this company builds APIs and approaches API- fi rst Focus 3: General API- fi rst training
48. API Conformance 4 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1
49. Problem? A Design-First approach to API development helps us adopt a user-centric design perspective. But how can we ensure the implemented API matches the design?
50. Countermeasure Add controls in your process to ensure that the API behaviour matches the externally published OpenAPI de fi nition fi le. Bene fi ts: Accurate documentation for better consumer dev experience and better API security.
51. 4 ways to do this
52. 1. Generate code from API spec design Openapi-generator Fern
53. 2. Generate specs from code
54. 3. Tests that API spec matches code
55. 4. Validate API spec in gateway
56. Let’s look at the testing approach: i.e OpenAPI schema testing (not integration testing!)
57. 1. Write schema test code
58. 2. Use a validating proxy with existing tests or throwaway tests https://github.com/stoplightio/prism
59. You need some form of conformance testing
60. Summary Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1 5
61. Summary APIOps is about automating the end-to-end API delivery lifecycle using DevOps and GitOps principles
62. Summary The API delivery tension is between speed and quality. And APIOps helps us achieve both for APIs.
63. Summary 3 ways of ensuring consistency with federated reviews: coaching, training sessions, and two-level approvals.
64. Summary The API review process should specify the guiding principles to help reviewers
65. Summary De fi ning API review process KPIs helps you measure the process performance. Use your API review process principles and the SPACE framework to guide you
66. Summary The API review process should de fi ne a disagreement resolution hierarchy
67. Summary API conformance controls are important and help you validate that your API implementation matches the designed spec
68. Summary A light-touch conformance testing approach using a validating proxy like Prism is a good way to improve conformance gradually
69. Automating API delivery: APIOps with OpenAPI https://www.manning.com/books/automating-api-delivery