Autonomous APIs (Paris 2018)

The deck I've used at my Autonomous APIs talk, APIdays Paris 2018. https://www.apidays.co/paris

Transcript

1. goodapi.co AUTONOMOUS APIS Zdenek “Z” Nemec goodapi.co Next chapter in APIs
2. goodapi.co I help businesses build APIs Zdenek “Z” Nemec @zdne zdne1
3. goodapi.co AUTONOMOUS APIS Next chapter in APIs
4. goodapi.co API CHAPTERS Number of APIs Year 1990 2000 2010 2020 1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS 4. AUTONOMOUS APIS
5. goodapi.co goodapi.co EVERY CHAPTER HAS ITS API ARCHITECTURAL STYLE–API PARADIGM REST vs. GraphQL @zdne
6. goodapi.co CUSTOMER-SPECIFIC APIS 1 LARGE ORGANIZATIONS EMBRACING IT (YEARS 1990-2000S)
7. goodapi.co CUSTOMER-SPECIFIC APIS 1 SOAP, EDI, FTP & ESB one-to-one, point-to-point
8. goodapi.co CUSTOMER-SPECIFIC APIS 1 API PROVIDERS FACE ISSUES WITH SIZE COMPLEXITY OF POINT TO POINT INTEGRATIONS AND THE ESB “FAILURE”
9. goodapi.co CUSTOMER-SPECIFIC APIS 1 NEW API CONSUMERS FIND THE APIS TOO COMPLEX AND CUMBERSOME TO USE
10. goodapi.co GENERIC APIS 2 UBIQUITOUS INTERNET & SOFTWARE DEVELOPMENT (YEARS 2005-2018+)
11. goodapi.co GENERIC APIS 2 one-to-many: many mid- or small- size customers one-to-one: few large established customers
12. goodapi.co GENERIC APIS 2 one-to-many SO-CALLED-REST, LATER ALSO GRAPHQL
13. goodapi.co GENERIC APIS 2 API PROVIDERS ARE FACING COMPLEXITY ISSUES FROM INCREASING NUMBER OF OWN APIS, CLIENT INTEGRATIONS, SERVER & CLIENT DEPLOYMENTS AND API VERSIONS
14. goodapi.co GENERIC APIS 2 NEW CONSUMERS FIND COMPLEX AND COSTLY TO INTEGRATE AND MAINTAIN THE INTEGRATIONS WITH MULTIPLE DIFFERENT APIS
15. goodapi.co HARMONIZED APIS 3 CLIENTS CONSUMING MULTIPLE APIS MICROSERVICES, FORMING API LANDSCAPES (YEAR 2017+)
16. goodapi.co API LANDSCAPES APIs are no longer in silos goodapi.co
17. goodapi.co HARMONIZED APIS 3 ORGANIZATIONS LANDSCAPES
18. goodapi.co HARMONIZED APIS 3 B C D . . . IF I AM BUILDING AN E-SHOP DO I WANT TO INTEGRATE WITH 400 LOGISTICS PROVIDERS? APIs e-shop DOMAIN LANDSCAPES
19. goodapi.co Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E API Consumer APIs Accommoda tion E 3 HARMONIZED APIS CROSS-DOMAIN LANDSCAPES
20. goodapi.co 3 HARMONIZED APIS EXAMPLES
21. goodapi.co –Radek Novotny; CEO & founder MyStay “There are few integrators for hospitality APIs consolidating as many as 40 PMS but they are either not-duplex or slow in reactions” CHALLENGE: INTEGRATION WITH MANY PROPERTY MANAGEMENT SYSTEMS (PMS) 3
22. goodapi.co HOW TO INTEGRATE WITH THOUSANDS OF IOT DEVICES FROM DIFFERENT VENDORS DEPLOYED IN DIFFERENT ENVIRONMENTS? 3
23. goodapi.co Client Client Client Client Client Client HOSPITALITY & TRAVEL Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E 3
24. goodapi.co Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E Existing Customer Existing Customer Existing Customer 3rd party New Type of Customer New Type of Customer New Type of Customer BANKING APIS Accelerated by PSD2 3
25. goodapi.co Accommoda tion B Accommoda tion C Accommoda tion D Accommoda tion E Existing Customer Existing Customer Existing Customer 3rd party New Type of Customer New Type of Customer New Type of Customer LOGISTICS Metapack integrates over 400 carriers Over 100 (!!!) 3rd Party vendors 3
26. goodapi.co Website A Client Client Client Client Client Client ARE YOUR CUSTOMERS REALLY YOURS? Website B Website C Website D Website E 3
27. goodapi.co HARMONIZED APIS 3 many-to-many: clients consuming multiple APIs one-to-many: many mid- or small- size customers
28. goodapi.co HARMONIZED APIS 3 COMPLEXITY OF CONSUMING MULTIPLE DIFFERENT APIS IS TACKLED BY STANDARDIZATION, GOVERNANCE TO ACHIEVE CONSISTENCY HARMONIZATION IS ALSO OFTEN DONE WITH WRAPPER APIS
29. goodapi.co HARMONIZED APIS 3 HARMONIZED APIS PROVIDERS ARE FACING COMPLEXITY ISSUES TRYING TO NORMALIZE, GOVERN OR MAINTAINING MANY INTEGRATIONS APIS MANUALLY
30. goodapi.co HARMONIZED APIS 3 COMPLEXITY EXTERNAL API CONSUMERS ARE FACING WHEN CONSUMING MULTIPLE DOMAIN OR CROSS-DOMAIN APIS IS STILL LEFT UNCHECKED
31. goodapi.co COMPLEXITY
32. goodapi.co COMPLEXITY Complexity Task-structure Complexity How difficult it is understand how to perform a task in a distributed system Unpredictability How difficult is to predict effect of an action in distributed system, amount of entropy Size Complexity Size of the system implies cognitive complexity, large number of concepts and the knowledge required Chaotic Complexity Small variations in a certain part of the system can have large effects on overall system behavior Algorithmic Complexity Both traditional algorithm complexity in the term of time and space and cognitive complexity of understanding algorithm
33. goodapi.co COMPLEXITY Complexity API Designs x Versions x Deployment Hosts x Clients x Client Deployments IS HIGHER THAN TWO You are likely to face complexity using, maintaining and evolving your API 130 APIs 3 x 6 (environments x availability zones) 70 versions 1500 integrations own deployments: 3 x 6 on-premise deployments: 1000 apps: 3 000 000 IoT: ????? = 1,10565×10^9 IF THE NUMBER OF
34. goodapi.co SOLUTIONS
35. goodapi.co SOLUTIONS TO COMPLEXITY Complexity 1. Hire more people 2. Standardization 3. Apply massive automation 4. Autonomy of the components in API landscape
36. goodapi.co SOLUTIONS TO COMPLEXITY Complexity 1. Hire More People Mechanical turks, eyeballing documentation for changes in design, fixing clients and implementations accordingly. CUSTOMER-SPECIFIC & GENERIC APIS ROLE OF HUMANS IN APIS
37. goodapi.co HIRE MORE PEOPLE Complexity REALITY $ HUMANS INSIDE VS. • time and money spent • not scaling • chance of misinterpretation • error-prone
38. goodapi.co SOLUTIONS TO COMPLEXITY Complexity 2. Standardization HARMONIZED APIS API DESIGN, LIFECYCLE & DEVELOPMENT STANDARDIZATION
39. goodapi.co STANDARDIZATION Complexity API GUIDELINES & DESIGN CHECKS
40. goodapi.co 3. Massive Automation AUTOMATED HARMONIZED APIS, AUTONOMOUS APIS Complexity SOLUTIONS TO COMPLEXITY AUTOMATED CONTRACT-DRIVEN API LIFECYCLE
41. goodapi.co INTERFACE ACTUALLY EXPOSED INTERFACE DOCUMENTED INTERFACED USED are they the same? are they the same? are they the same? Complexity AUTOMATION KEEPING THINGS IN CHECK IS HARD
42. goodapi.co Complexity AUTOMATION CONTRACT-DRIVEN
43. goodapi.co https://goodapi.co/api-lifecycle Complexity AUTOMATION ENTIRE LIFECYCLE
44. goodapi.co 4. Autonomy AUTONOMOUS APIS SOLUTIONS TO COMPLEXITY AUTONOMY IS THE SOLUTION TO THE DIFFICULTY OF MANAGING MANY COMPONENTS CENTRALLY Components autonomy in an API Landscape reduces the complexity, enables scaling, and promotes the service commoditization. Complexity
45. goodapi.co AUTONOMY “Autonomy: freedom from external control or influence; independence; sovereignty in making decisions and changes.” –New Oxford American Dictionary Complexity
46. goodapi.co AUTONOMOUS APIS 4 MACHINE-TO-MACHINE APIS, SELF-DRIVING, NO HUMAN INTERVENTION, CROSS-DOMAIN LANDSCAPES RISE OF IOT & AI (YEAR 2020+)
47. goodapi.co Navigation through autonomic units Countries, Regions, Cities, City Quarters are autonomous regions but when you are driving from A to B you need to navigate through their roads A B Design-time & Runtime Navigation You need both to follow a map and traffic signs to navigate effectively 4 AUTONOMOUS APIS NAVIGATE SAFELY IN API LANDSCAPE
48. goodapi.co –Autonomous Client “The level of autonomy is the ability to safely navigate in the API landscape (where the constraints are time and price)” 4 LEVELS OF AUTONOMY
49. goodapi.co Brittle System Documented Brittle System Automated System Decoupled Automated System Autonomous System Autonomy Autonomous APIs 0 1 2 3 4 AUTONOMOUS APIS 4
50. goodapi.co • Tightly coupled components • No semantics, interface or lifecycle data • Pros • Time to delivery for point to point integrations when complexity is low • Cons • Everything breaks when something changes • Not scaling with growing complexity Level 0 BRITTLE SYSTEM LEVELS OF AUTONOMY
51. goodapi.co • Tightly coupled components • Interface metadata available • Human only-semantics metadata available • No lifecycle metadata (is the documentation documenting the interface implementation?) • Pros • Possible API productization • Reusability • Cons • Lock on changes, unable to evolve DOCUMENTED BRITTLE SYSTEM LEVELS OF AUTONOMY Level 1
52. goodapi.co • Mostly tightly coupled components • Metadata are verified, valid and reliable • Metadata are human or machine readable • Advertisement of the interface, semantics and lifecycle metadata • Client subscription to future changes • Pros • Explicit understanding of price and risk of making changes in the API landscape • Quality Control • Cons • Advanced mindset change involving many people • Cultural denial, hard to get buy-in AUTOMATED SYSTEM LEVELS OF AUTONOMY Level 2
53. goodapi.co • Uniform Interface • Decoupled components • Evolvability of the components in their respective environments • Share understanding-metadata at RUNTIME • Pros • Harmonized within the system • Reducing price and risk of making changes (by design) • Cons • Difficult taking everything into account, prone to chaotic & size complexity DECOUPLED AUTOMATED SYSTEM LEVELS OF AUTONOMY Level 3
54. goodapi.co • Self driving clients • Landscape discovery • Machine understanding • Vocabulary not data structure • Declarative programming • Autonomy of the components • Leveraging learned data from simulation for infrastructure • Pros • Higher level of abstraction • Implicit harmonization • Full autonomy of components • Independent evolution • Cons • Cognitive (Algorithmic) Complexity • Demanding on computational power • Higher level of abstraction AUTONOMOUS SYSTEM LEVELS OF AUTONOMY Level 4
55. goodapi.co goodapi.co Early four-engine jets required flight engineers. Later four-engine jets were designed with sufficient automation to eliminate the position. 747 FLIGHT ENGINEERS ...this will also happen in the world of APIs.
56. goodapi.co API CHAPTERS one-to-many one-to-one many-to-many machine-to-machine Drivers behind the changes API CONSUMERS EXPECTATIONS COMPLEXITY task-structure unpredictability chaotic size algorithmic + GROWING # OF API CONSUMERS + INCREASING DEMAND EVOLVING DEMAND INCREASING PROBLEMS
57. goodapi.co 1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS 4. AUTONOMOUS APIS one-to-many one-to-one many-to-many machine-to-machine API CHAPTERS
58. goodapi.co SERVICE COMMODITIZATION CUSTOMER SPECIFIC APIS (SOAP) GENERIC APIS (SO-CALLED-REST) HARMONIZED APIS AUTONOMOUS APIS Graph by Simon Wardley, “How commodity is something?” HARMONIZED APIS (AUTOMATED)
59. goodapi.co DIGITAL WHOLESALE PRESENCE & SERVICE COMMODITIZATION “Look for a service that can fulfill the shipment of 6 pallets from Prague to Paris under 100 EUR.” “Look for a service that provides information about adidas stan smith” “Is there an air condition service in my landscape?” “Look for a service that has gs1:Offer where gs1:itemOffered gs1:gpcCategoryCode is MILK.” “The level of autonomy is the ability to safely navigate in the API landscape (where the constraints are time and price)” REVOLUTION IN (API) ECONOMY
60. goodapi.co GOOD API INDEPENDENT API CONSULTING WE ARE HIRING
61. IMAGES BY UNSPLASH.COM ICONS BY MARTINFOWLER.COM