Autonomous APIs (Paris 2018)

Slide 1

Slide 1 text

goodapi.co AUTONOMOUS APIS Zdenek “Z” Nemec goodapi.co Next chapter in APIs

Slide 2

Slide 2 text

goodapi.co I help businesses build APIs Zdenek “Z” Nemec @zdne zdne1

Slide 3

Slide 3 text

goodapi.co AUTONOMOUS APIS Next chapter in APIs

Slide 4

Slide 4 text

goodapi.co API CHAPTERS Number of APIs Year 1990 2000 2010 2020 1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS 4. AUTONOMOUS APIS

Slide 5

Slide 5 text

goodapi.co goodapi.co EVERY CHAPTER HAS ITS API ARCHITECTURAL STYLE–API PARADIGM REST vs. GraphQL @zdne

Slide 6

Slide 6 text

goodapi.co CUSTOMER-SPECIFIC APIS 1 LARGE ORGANIZATIONS EMBRACING IT (YEARS 1990-2000S)

Slide 7

Slide 7 text

goodapi.co CUSTOMER-SPECIFIC APIS 1 SOAP, EDI, FTP & ESB one-to-one, point-to-point

Slide 8

Slide 8 text

goodapi.co CUSTOMER-SPECIFIC APIS 1 API PROVIDERS FACE ISSUES WITH SIZE COMPLEXITY OF POINT TO POINT INTEGRATIONS AND THE ESB “FAILURE”

Slide 9

Slide 9 text

goodapi.co CUSTOMER-SPECIFIC APIS 1 NEW API CONSUMERS FIND THE APIS TOO COMPLEX AND CUMBERSOME TO USE

Slide 10

Slide 10 text

goodapi.co GENERIC APIS 2 UBIQUITOUS INTERNET & SOFTWARE DEVELOPMENT (YEARS 2005-2018+)

Slide 11

Slide 11 text

goodapi.co GENERIC APIS 2 one-to-many: many mid- or small- size customers one-to-one: few large established customers

Slide 12

Slide 12 text

goodapi.co GENERIC APIS 2 one-to-many SO-CALLED-REST, LATER ALSO GRAPHQL

Slide 13

Slide 13 text

goodapi.co GENERIC APIS 2 API PROVIDERS ARE FACING COMPLEXITY ISSUES FROM INCREASING NUMBER OF OWN APIS, CLIENT INTEGRATIONS, SERVER & CLIENT DEPLOYMENTS AND API VERSIONS

Slide 14

Slide 14 text

goodapi.co GENERIC APIS 2 NEW CONSUMERS FIND COMPLEX AND COSTLY TO INTEGRATE AND MAINTAIN THE INTEGRATIONS WITH MULTIPLE DIFFERENT APIS

Slide 15

Slide 15 text

goodapi.co HARMONIZED APIS 3 CLIENTS CONSUMING MULTIPLE APIS MICROSERVICES, FORMING API LANDSCAPES (YEAR 2017+)

Slide 16

Slide 16 text

goodapi.co API LANDSCAPES APIs are no longer in silos goodapi.co

Slide 17

Slide 17 text

goodapi.co HARMONIZED APIS 3 ORGANIZATIONS LANDSCAPES

Slide 18

Slide 18 text

goodapi.co HARMONIZED APIS 3 B C D . . . IF I AM BUILDING AN E-SHOP DO I WANT TO INTEGRATE WITH 400 LOGISTICS PROVIDERS? APIs e-shop DOMAIN LANDSCAPES

Slide 19

Slide 19 text

goodapi.co Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E API Consumer APIs Accommoda tion E 3 HARMONIZED APIS CROSS-DOMAIN LANDSCAPES

Slide 20

Slide 20 text

goodapi.co 3 HARMONIZED APIS EXAMPLES

Slide 21

Slide 21 text

goodapi.co –Radek Novotny; CEO & founder MyStay “There are few integrators for hospitality APIs consolidating as many as 40 PMS but they are either not-duplex or slow in reactions” CHALLENGE: INTEGRATION WITH MANY PROPERTY MANAGEMENT SYSTEMS (PMS) 3

Slide 22

Slide 22 text

goodapi.co HOW TO INTEGRATE WITH THOUSANDS OF IOT DEVICES FROM DIFFERENT VENDORS DEPLOYED IN DIFFERENT ENVIRONMENTS? 3

Slide 23

Slide 23 text

goodapi.co Client Client Client Client Client Client HOSPITALITY & TRAVEL Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E 3

Slide 24

Slide 24 text

goodapi.co Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E Existing Customer Existing Customer Existing Customer 3rd party New Type of Customer New Type of Customer New Type of Customer BANKING APIS Accelerated by PSD2 3

Slide 25

Slide 25 text

Slide 26

Slide 26 text

goodapi.co Website A Client Client Client Client Client Client ARE YOUR CUSTOMERS REALLY YOURS? Website B Website C Website D Website E 3

Slide 27

Slide 27 text

goodapi.co HARMONIZED APIS 3 many-to-many: clients consuming multiple APIs one-to-many: many mid- or small- size customers

Slide 28

Slide 28 text

goodapi.co HARMONIZED APIS 3 COMPLEXITY OF CONSUMING MULTIPLE DIFFERENT APIS IS TACKLED BY STANDARDIZATION, GOVERNANCE TO ACHIEVE CONSISTENCY HARMONIZATION IS ALSO OFTEN DONE WITH WRAPPER APIS

Slide 29

Slide 29 text

goodapi.co HARMONIZED APIS 3 HARMONIZED APIS PROVIDERS ARE FACING COMPLEXITY ISSUES TRYING TO NORMALIZE, GOVERN OR MAINTAINING MANY INTEGRATIONS APIS MANUALLY

Slide 30

Slide 30 text

goodapi.co HARMONIZED APIS 3 COMPLEXITY EXTERNAL API CONSUMERS ARE FACING WHEN CONSUMING MULTIPLE DOMAIN OR CROSS-DOMAIN APIS IS STILL LEFT UNCHECKED

Slide 31

Slide 31 text

goodapi.co COMPLEXITY

Slide 32

Slide 32 text

goodapi.co COMPLEXITY Complexity Task-structure Complexity How difficult it is understand how to perform a task in a distributed system Unpredictability How difficult is to predict effect of an action in distributed system, amount of entropy Size Complexity Size of the system implies cognitive complexity, large number of concepts and the knowledge required Chaotic Complexity Small variations in a certain part of the system can have large effects on overall system behavior Algorithmic Complexity Both traditional algorithm complexity in the term of time and space and cognitive complexity of understanding algorithm

Slide 33

Slide 33 text

goodapi.co COMPLEXITY Complexity API Designs x Versions x Deployment Hosts x Clients x Client Deployments IS HIGHER THAN TWO You are likely to face complexity using, maintaining and evolving your API 130 APIs 3 x 6 (environments x availability zones) 70 versions 1500 integrations own deployments: 3 x 6 on-premise deployments: 1000 apps: 3 000 000 IoT: ????? = 1,10565×10^9 IF THE NUMBER OF

Slide 34

Slide 34 text

goodapi.co SOLUTIONS

Slide 35

Slide 35 text

goodapi.co SOLUTIONS TO COMPLEXITY Complexity 1. Hire more people 2. Standardization 3. Apply massive automation 4. Autonomy of the components in API landscape

Slide 36

Slide 36 text

goodapi.co SOLUTIONS TO COMPLEXITY Complexity 1. Hire More People Mechanical turks, eyeballing documentation for changes in design, fixing clients and implementations accordingly. CUSTOMER-SPECIFIC & GENERIC APIS ROLE OF HUMANS IN APIS

Slide 37

Slide 37 text

goodapi.co HIRE MORE PEOPLE Complexity REALITY $ HUMANS INSIDE VS. • time and money spent • not scaling • chance of misinterpretation • error-prone

Slide 38

Slide 38 text

goodapi.co SOLUTIONS TO COMPLEXITY Complexity 2. Standardization HARMONIZED APIS API DESIGN, LIFECYCLE & DEVELOPMENT STANDARDIZATION

Slide 39

Slide 39 text

goodapi.co STANDARDIZATION Complexity API GUIDELINES & DESIGN CHECKS

Slide 40

Slide 40 text

goodapi.co 3. Massive Automation AUTOMATED HARMONIZED APIS, AUTONOMOUS APIS Complexity SOLUTIONS TO COMPLEXITY AUTOMATED CONTRACT-DRIVEN API LIFECYCLE

Slide 41

Slide 41 text

goodapi.co INTERFACE ACTUALLY EXPOSED INTERFACE DOCUMENTED INTERFACED USED are they the same? are they the same? are they the same? Complexity AUTOMATION KEEPING THINGS IN CHECK IS HARD

Slide 42

Slide 42 text

goodapi.co Complexity AUTOMATION CONTRACT-DRIVEN

Slide 43

Slide 43 text

goodapi.co https://goodapi.co/api-lifecycle Complexity AUTOMATION ENTIRE LIFECYCLE

Slide 44

Slide 44 text

goodapi.co 4. Autonomy AUTONOMOUS APIS SOLUTIONS TO COMPLEXITY AUTONOMY IS THE SOLUTION TO THE DIFFICULTY OF MANAGING MANY COMPONENTS CENTRALLY Components autonomy in an API Landscape reduces the complexity, enables scaling, and promotes the service commoditization. Complexity

Slide 45

Slide 45 text

goodapi.co AUTONOMY “Autonomy: freedom from external control or inﬂuence; independence; sovereignty in making decisions and changes.” –New Oxford American Dictionary Complexity

Slide 46

Slide 46 text

goodapi.co AUTONOMOUS APIS 4 MACHINE-TO-MACHINE APIS, SELF-DRIVING, NO HUMAN INTERVENTION, CROSS-DOMAIN LANDSCAPES RISE OF IOT & AI (YEAR 2020+)

Slide 47

Slide 47 text

goodapi.co Navigation through autonomic units Countries, Regions, Cities, City Quarters are autonomous regions but when you are driving from A to B you need to navigate through their roads A B Design-time & Runtime Navigation You need both to follow a map and traffic signs to navigate effectively 4 AUTONOMOUS APIS NAVIGATE SAFELY IN API LANDSCAPE

Slide 48

Slide 48 text

goodapi.co –Autonomous Client “The level of autonomy is the ability to safely navigate in the API landscape (where the constraints are time and price)” 4 LEVELS OF AUTONOMY

Slide 49

Slide 49 text

goodapi.co Brittle System Documented Brittle System Automated System Decoupled Automated System Autonomous System Autonomy Autonomous APIs 0 1 2 3 4 AUTONOMOUS APIS 4

Slide 50

Slide 50 text

goodapi.co • Tightly coupled components • No semantics, interface or lifecycle data • Pros • Time to delivery for point to point integrations when complexity is low • Cons • Everything breaks when something changes • Not scaling with growing complexity Level 0 BRITTLE SYSTEM LEVELS OF AUTONOMY

Slide 51

Slide 51 text

goodapi.co • Tightly coupled components • Interface metadata available • Human only-semantics metadata available • No lifecycle metadata (is the documentation documenting the interface implementation?) • Pros • Possible API productization • Reusability • Cons • Lock on changes, unable to evolve DOCUMENTED BRITTLE SYSTEM LEVELS OF AUTONOMY Level 1

Slide 52

Slide 52 text

goodapi.co • Mostly tightly coupled components • Metadata are verified, valid and reliable • Metadata are human or machine readable • Advertisement of the interface, semantics and lifecycle metadata • Client subscription to future changes • Pros • Explicit understanding of price and risk of making changes in the API landscape • Quality Control • Cons • Advanced mindset change involving many people • Cultural denial, hard to get buy-in AUTOMATED SYSTEM LEVELS OF AUTONOMY Level 2

Slide 53

Slide 53 text

goodapi.co • Uniform Interface • Decoupled components • Evolvability of the components in their respective environments • Share understanding-metadata at RUNTIME • Pros • Harmonized within the system • Reducing price and risk of making changes (by design) • Cons • Difficult taking everything into account, prone to chaotic & size complexity DECOUPLED AUTOMATED SYSTEM LEVELS OF AUTONOMY Level 3

Slide 54

Slide 54 text

goodapi.co • Self driving clients • Landscape discovery • Machine understanding • Vocabulary not data structure • Declarative programming • Autonomy of the components • Leveraging learned data from simulation for infrastructure • Pros • Higher level of abstraction • Implicit harmonization • Full autonomy of components • Independent evolution • Cons • Cognitive (Algorithmic) Complexity • Demanding on computational power • Higher level of abstraction AUTONOMOUS SYSTEM LEVELS OF AUTONOMY Level 4

Slide 55

Slide 55 text

goodapi.co goodapi.co Early four-engine jets required flight engineers. Later four-engine jets were designed with sufficient automation to eliminate the position. 747 FLIGHT ENGINEERS ...this will also happen in the world of APIs.

Slide 56

Slide 56 text

goodapi.co API CHAPTERS one-to-many one-to-one many-to-many machine-to-machine Drivers behind the changes API CONSUMERS EXPECTATIONS COMPLEXITY task-structure unpredictability chaotic size algorithmic + GROWING # OF API CONSUMERS + INCREASING DEMAND EVOLVING DEMAND INCREASING PROBLEMS

Slide 57

Slide 57 text

goodapi.co 1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS 4. AUTONOMOUS APIS one-to-many one-to-one many-to-many machine-to-machine API CHAPTERS

Slide 58

Slide 58 text

goodapi.co SERVICE COMMODITIZATION CUSTOMER SPECIFIC APIS (SOAP) GENERIC APIS (SO-CALLED-REST) HARMONIZED APIS AUTONOMOUS APIS Graph by Simon Wardley, “How commodity is something?” HARMONIZED APIS (AUTOMATED)

Slide 59

Slide 59 text

goodapi.co DIGITAL WHOLESALE PRESENCE & SERVICE COMMODITIZATION “Look for a service that can fulfill the shipment of 6 pallets from Prague to Paris under 100 EUR.” “Look for a service that provides information about adidas stan smith” “Is there an air condition service in my landscape?” “Look for a service that has gs1:Offer where gs1:itemOffered gs1:gpcCategoryCode is MILK.” “The level of autonomy is the ability to safely navigate in the API landscape (where the constraints are time and price)” REVOLUTION IN (API) ECONOMY