Scaling API Reviews - Speaker Deck

Slide 1

Slide 1 text

Scaling API reviews Ikenna Nwaiwu Presentation to Just Eat Takeaway (JET) 9th November 2023

Slide 2

Slide 2 text

Agenda Intro to APIOps Scaling API reviews API conformance Summary About me 2 3 4 5 1

Slide 3

Slide 3 text

About me 1 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1

Slide 4

Slide 4 text

APIOps Lead API DESIGN GOVERNANCE API COMMUNITY OF PRACTICE ENABLING API TEAMS AUTHOR: AUTOMATING API DELIVERY Linkedin.com/in/ikenna

Slide 5

Slide 5 text

Intro to APIOps 2 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1

Slide 6

Slide 6 text

The API delivery problem Increase speed of API design and delivery (Lower lead time) Consistency with API style guide. Quality API documentation. Well- tested, secure APIs. (Higher quality) Tension

Slide 7

Slide 7 text

What is APIOps? APIOps is the end-to-end automation of the API life cycle using DevOps and GitOps principles to help API providers deliver more value to API consumers faster

Slide 8

Slide 8 text

What is APIOps?

Slide 9

Slide 9 text

APIOps Principles

Slide 10

Slide 10 text

Process time for API reviews in APIOps

Slide 11

Slide 11 text

Example APIOps work fl ow

Slide 12

Slide 12 text

GitOps API Con fi g deployment

Slide 13

Slide 13 text

Scaling API reveiws 3 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1

Slide 14

Slide 14 text

Problem: How do you maintain the quality of colleague API reviews when you have 30 teams instead of 3?

Slide 15

Slide 15 text

Countermeasure: Move from centralize colleague reviews to federated colleague reviews

Slide 16

Slide 16 text

Centralized reviews

Slide 17

Slide 17 text

Federated API reviews

Slide 18

Slide 18 text

But how do you maintain colleague review consistency with federated reviews?

Slide 19

Slide 19 text

3 areas of focus B. De fi ne important elements of API review process document A. De fi ne a process for becoming a reviewer C. Provide general API training

Slide 20

Slide 20 text

Focus 1: De fi ne the process for becoming a reviewer B. Training sessions A. Coaching C. Use federated and central approvals

Slide 21

Slide 21 text

Coaching Kubernetes API Review Process document https://github.com/kubernetes/community/blob/master/sig-architecture/ api-review-process.md Focus 1: De fi ne the process for becoming a reviewer

Slide 22

Slide 22 text

“To become an API reviewer on the Kubernetes project, aspiring reviewers must gain a high level of pro fi ciency in the Kubernetes API style guide. They also need to have a good understanding of the project structure and the system architecture. They are required to have participated in preliminary API reviews where mentors coach them. Mentors help train them and facilitate their inclusion as formal reviewers.” - From ‘Automating API Delivery’ by Ikenna Nwaiwu Focus 1: De fi ne the process for becoming a reviewer Coaching

Slide 23

Slide 23 text

Run training sessions for API reviewers on API style guide, API spec repo structure, API spec pipeline, API architecture, API spec publishing process, linting rules, API review document, expectations from API reviews, and so on. Training sessions Focus 1: De fi ne the process for becoming a reviewer

Slide 24

Slide 24 text

Training sessions Focus 1: De fi ne the process for becoming a reviewer Run periodic retrospectives with API reviewers.

Slide 25

Slide 25 text

Focus 1: De fi ne the process for becoming a reviewer Use federated and central PR approvals AT 10x, we have two levels of API review PR approvals: central and team level. As an Engineering Manager, you are automatically assigned as an API reviewer for your team's APIs.

Slide 26

Slide 26 text

Focus 2: De fi ne important elements of the API review process document B. Process KPIs A. Process principles C. Disagreement resolution hierarchy

Slide 27

Slide 27 text

A. Process principles Focus 2: De fi ne important elements of the API review process document API review process document should specify guiding principles so federated reviewers can refer to them. Suggested: quality, speed, dependability, fl exibility, and cost.

Slide 28

Slide 28 text

1. Quality Improve API design quality and consistency with standards. Improve API documentation.

Slide 29

Slide 29 text

2. Speed Reduce the time required to review and approve a design change

Slide 30

Slide 30 text

3. Dependability To enable planning, set expectations on how long it takes to complete an API review.

Slide 31

Slide 31 text

4. Flexibility Should explain the exception process. Di ff erent levels of reviews

Slide 32

Slide 32 text

Zulian & Bouza’s hierarchy of API design principles From ‘API Product Management’ by Andrea Zulian and Amancio Bouza

Slide 33

Slide 33 text

5. Cost Aim to lower cost. Reviewers should suggest new automated checks where possible. Also, aim to document new ideas in the style guide.

Slide 34

Slide 34 text

B. Process KPIs Focus 2: De fi ne important elements of the API review process document Should specify API review process KPIs. 6 suggested: PR TTFRC, Number of Open PRs, PR lifetime, Review process satisfaction, and number of suggested improvements.

Slide 35

Slide 35 text

1 B. Process KPIs PR TTFRC Time To First Review Comment or approval

Slide 36

Slide 36 text

2 B. Process KPIs Number of open PRs

Slide 37

Slide 37 text

3 B. Process KPIs PR lifetime from creation to merge

Slide 38

Slide 38 text

4 B. Process KPIs API designer satisfaction using regular review process satisfaction surveys

Slide 39

Slide 39 text

5 B. Process KPIs Number of suggested improvements to API style guide, linting rules, API process

Slide 40

Slide 40 text

These are KPIs I suggest, but a more rigorous approach is to think of KPIs that fall into the SPACE framework dimensions

Slide 41

Slide 41 text

SPACE Developer Productivity Metrics - ‘The SPACE of Developer Productivity’ by Nicole Forsgren et al. https:// queue.acm.org/detail.cfm?id=3454124 “The most important takeaway from exposing these myths is that productivity cannot be reduced to a single dimension (or metric!). The prevalence of these myths and the need to bust them motivated our work to develop a practical multidimensional framework, because only by examining a constellation of metrics in tension can we understand and in fl uence developer productivity.”

Slide 42

Slide 42 text

SPACE Developer Productivity Metrics ‘The SPACE of Developer Productivity’ by Nicole Forsgren et al. https://queue.acm.org/ detail.cfm?id=3454124

Slide 43

Slide 43 text

Should provide clear escalation process. C. Resolution hierarchy Eric Brewer https://google.aip.dev/1 Joao Prado

Slide 44

Slide 44 text

Focus 3: General API- fi rst training

Slide 45

Slide 45 text

Audience: API designers / Engrs, POs, BAs Focus 3: General API- fi rst training

Slide 46

Slide 46 text

When: on joining the company Focus 3: General API- fi rst training

Slide 47

Slide 47 text

Content: How this company builds APIs and approaches API- fi rst Focus 3: General API- fi rst training

Slide 48

Slide 48 text

API Conformance 4 Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1

Slide 49

Slide 49 text

Problem? A Design-First approach to API development helps us adopt a user-centric design perspective. But how can we ensure the implemented API matches the design?

Slide 50

Slide 50 text

Countermeasure Add controls in your process to ensure that the API behaviour matches the externally published OpenAPI de fi nition fi le. Bene fi ts: Accurate documentation for better consumer dev experience and better API security.

Slide 51

Slide 51 text

4 ways to do this

Slide 52

Slide 52 text

1. Generate code from API spec design Openapi-generator Fern

Slide 53

Slide 53 text

2. Generate specs from code

Slide 54

Slide 54 text

3. Tests that API spec matches code

Slide 55

Slide 55 text

4. Validate API spec in gateway

Slide 56

Slide 56 text

Let’s look at the testing approach: i.e OpenAPI schema testing (not integration testing!)

Slide 57

Slide 57 text

1. Write schema test code

Slide 58

Slide 58 text

2. Use a validating proxy with existing tests or throwaway tests https://github.com/stoplightio/prism

Slide 59

Slide 59 text

You need some form of conformance testing

Slide 60

Slide 60 text

Summary Intro to APIOps 2 Scaling API reviews 3 API conformance 4 Summary 5 About me 1 5

Slide 61

Slide 61 text

Summary APIOps is about automating the end-to-end API delivery lifecycle using DevOps and GitOps principles

Slide 62

Slide 62 text

Summary The API delivery tension is between speed and quality. And APIOps helps us achieve both for APIs.

Slide 63

Slide 63 text

Summary 3 ways of ensuring consistency with federated reviews: coaching, training sessions, and two-level approvals.

Slide 64

Slide 64 text

Summary The API review process should specify the guiding principles to help reviewers

Slide 65

Slide 65 text

Summary De fi ning API review process KPIs helps you measure the process performance. Use your API review process principles and the SPACE framework to guide you

Slide 66

Slide 66 text

Summary The API review process should de fi ne a disagreement resolution hierarchy

Slide 67

Slide 67 text

Summary API conformance controls are important and help you validate that your API implementation matches the designed spec

Slide 68

Slide 68 text

Summary A light-touch conformance testing approach using a validating proxy like Prism is a good way to improve conformance gradually

Slide 69

Slide 69 text

Automating API delivery: APIOps with OpenAPI https://www.manning.com/books/automating-api-delivery