$30 off During Our Annual Pro Sale. View Details »

Autonomous APIs (Paris 2018)

Z
December 11, 2018

Autonomous APIs (Paris 2018)

The deck I've used at my Autonomous APIs talk, APIdays Paris 2018. https://www.apidays.co/paris

Z

December 11, 2018
Tweet

More Decks by Z

Other Decks in Programming

Transcript

  1. goodapi.co
    AUTONOMOUS
    APIS
    Zdenek “Z” Nemec
    goodapi.co
    Next chapter in APIs

    View Slide

  2. goodapi.co
    I help businesses build APIs
    Zdenek “Z” Nemec
    @zdne
    zdne1

    View Slide

  3. goodapi.co
    AUTONOMOUS APIS
    Next chapter in APIs

    View Slide

  4. goodapi.co
    API CHAPTERS
    Number of APIs
    Year
    1990 2000 2010 2020
    1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS
    4. AUTONOMOUS APIS

    View Slide

  5. goodapi.co
    goodapi.co
    EVERY CHAPTER
    HAS ITS API
    ARCHITECTURAL
    STYLE–API
    PARADIGM REST vs. GraphQL
    @zdne

    View Slide

  6. goodapi.co
    CUSTOMER-SPECIFIC APIS
    1
    LARGE ORGANIZATIONS EMBRACING IT
    (YEARS 1990-2000S)

    View Slide

  7. goodapi.co
    CUSTOMER-SPECIFIC APIS
    1
    SOAP, EDI, FTP & ESB
    one-to-one, point-to-point

    View Slide

  8. goodapi.co
    CUSTOMER-SPECIFIC APIS
    1
    API PROVIDERS FACE ISSUES WITH SIZE
    COMPLEXITY OF POINT TO POINT INTEGRATIONS
    AND THE ESB “FAILURE”

    View Slide

  9. goodapi.co
    CUSTOMER-SPECIFIC APIS
    1
    NEW API CONSUMERS FIND THE APIS TOO
    COMPLEX AND CUMBERSOME TO USE

    View Slide

  10. goodapi.co
    GENERIC APIS
    2
    UBIQUITOUS INTERNET & SOFTWARE DEVELOPMENT
    (YEARS 2005-2018+)

    View Slide

  11. goodapi.co
    GENERIC APIS
    2
    one-to-many: many mid- or
    small- size customers
    one-to-one: few large
    established customers

    View Slide

  12. goodapi.co
    GENERIC APIS
    2
    one-to-many
    SO-CALLED-REST, LATER ALSO GRAPHQL

    View Slide

  13. goodapi.co
    GENERIC APIS
    2
    API PROVIDERS ARE FACING COMPLEXITY
    ISSUES FROM INCREASING NUMBER OF OWN
    APIS, CLIENT INTEGRATIONS, SERVER & CLIENT
    DEPLOYMENTS AND API VERSIONS

    View Slide

  14. goodapi.co
    GENERIC APIS
    2
    NEW CONSUMERS FIND COMPLEX AND COSTLY
    TO INTEGRATE AND MAINTAIN THE
    INTEGRATIONS WITH MULTIPLE DIFFERENT APIS

    View Slide

  15. goodapi.co
    HARMONIZED APIS
    3
    CLIENTS CONSUMING MULTIPLE APIS
    MICROSERVICES, FORMING API LANDSCAPES
    (YEAR 2017+)

    View Slide

  16. goodapi.co
    API
    LANDSCAPES
    APIs are no longer in silos
    goodapi.co

    View Slide

  17. goodapi.co
    HARMONIZED APIS
    3
    ORGANIZATIONS LANDSCAPES

    View Slide

  18. goodapi.co
    HARMONIZED APIS
    3
    B C D . . .
    IF I AM BUILDING AN E-SHOP DO I WANT TO INTEGRATE WITH 400
    LOGISTICS PROVIDERS?
    APIs
    e-shop
    DOMAIN LANDSCAPES

    View Slide

  19. goodapi.co
    Accommoda
    tion B
    Accommoda
    tion C
    Accommoda
    tion D
    Accommoda
    tion E
    API
    Consumer
    APIs
    Accommoda
    tion E
    3 HARMONIZED APIS
    CROSS-DOMAIN LANDSCAPES

    View Slide

  20. goodapi.co
    3 HARMONIZED APIS EXAMPLES

    View Slide

  21. goodapi.co
    –Radek Novotny; CEO & founder MyStay
    “There are few integrators for
    hospitality APIs consolidating as
    many as 40 PMS but they are
    either not-duplex or slow in
    reactions”
    CHALLENGE: INTEGRATION
    WITH MANY PROPERTY
    MANAGEMENT SYSTEMS (PMS)
    3

    View Slide

  22. goodapi.co
    HOW TO INTEGRATE WITH THOUSANDS OF
    IOT DEVICES FROM DIFFERENT VENDORS
    DEPLOYED IN DIFFERENT ENVIRONMENTS?
    3

    View Slide

  23. goodapi.co
    Client Client Client Client Client Client
    HOSPITALITY & TRAVEL
    Accommoda
    tion B
    Accommoda
    tion C
    Accommoda
    tion D
    Accommoda
    tion E
    3

    View Slide

  24. goodapi.co
    Accommoda
    tion B
    Accommoda
    tion C
    Accommoda
    tion D
    Accommoda
    tion E
    Existing
    Customer
    Existing
    Customer
    Existing
    Customer
    3rd party
    New Type of
    Customer
    New Type of
    Customer
    New Type of
    Customer
    BANKING APIS
    Accelerated by PSD2
    3

    View Slide

  25. goodapi.co
    Accommoda
    tion B
    Accommoda
    tion C
    Accommoda
    tion D
    Accommoda
    tion E
    Existing
    Customer
    Existing
    Customer
    Existing
    Customer
    3rd party
    New Type of
    Customer
    New Type of
    Customer
    New Type of
    Customer
    LOGISTICS
    Metapack integrates
    over 400 carriers
    Over 100 (!!!) 3rd Party vendors
    3

    View Slide

  26. goodapi.co
    Website A
    Client Client Client Client Client Client
    ARE YOUR CUSTOMERS
    REALLY YOURS?
    Website B Website C Website D Website E
    3

    View Slide

  27. goodapi.co
    HARMONIZED APIS
    3
    many-to-many: clients
    consuming multiple APIs
    one-to-many: many mid- or
    small- size customers

    View Slide

  28. goodapi.co
    HARMONIZED APIS
    3
    COMPLEXITY OF CONSUMING MULTIPLE
    DIFFERENT APIS IS TACKLED BY
    STANDARDIZATION, GOVERNANCE TO ACHIEVE
    CONSISTENCY
    HARMONIZATION IS ALSO OFTEN DONE WITH
    WRAPPER APIS

    View Slide

  29. goodapi.co
    HARMONIZED APIS
    3
    HARMONIZED APIS PROVIDERS ARE FACING
    COMPLEXITY ISSUES TRYING TO NORMALIZE,
    GOVERN OR MAINTAINING MANY INTEGRATIONS
    APIS MANUALLY

    View Slide

  30. goodapi.co
    HARMONIZED APIS
    3
    COMPLEXITY EXTERNAL API CONSUMERS ARE
    FACING WHEN CONSUMING MULTIPLE DOMAIN
    OR CROSS-DOMAIN APIS IS STILL LEFT
    UNCHECKED

    View Slide

  31. goodapi.co
    COMPLEXITY

    View Slide

  32. goodapi.co
    COMPLEXITY
    Complexity
    Task-structure
    Complexity
    How difficult it is
    understand how to
    perform a task in a
    distributed system
    Unpredictability
    How difficult is to predict
    effect of an action in
    distributed system, amount
    of entropy
    Size Complexity
    Size of the system implies
    cognitive complexity, large
    number of concepts and
    the knowledge required
    Chaotic Complexity
    Small variations in a certain
    part of the system can have
    large effects on overall
    system behavior
    Algorithmic Complexity
    Both traditional algorithm
    complexity in the term of
    time and space and
    cognitive complexity of
    understanding algorithm

    View Slide

  33. goodapi.co
    COMPLEXITY
    Complexity
    API Designs x Versions x Deployment Hosts x Clients x Client Deployments
    IS HIGHER THAN TWO
    You are likely to face complexity using, maintaining and evolving your API
    130 APIs
    3 x 6 (environments x
    availability zones)
    70 versions 1500 integrations
    own deployments: 3 x 6
    on-premise deployments: 1000
    apps: 3 000 000
    IoT: ?????
    = 1,10565×10^9
    IF THE NUMBER OF

    View Slide

  34. goodapi.co
    SOLUTIONS

    View Slide

  35. goodapi.co
    SOLUTIONS TO COMPLEXITY
    Complexity
    1. Hire more people
    2. Standardization
    3. Apply massive automation
    4. Autonomy of the components in API landscape

    View Slide

  36. goodapi.co
    SOLUTIONS TO COMPLEXITY
    Complexity
    1. Hire More People
    Mechanical turks, eyeballing documentation for changes in
    design, fixing clients and implementations accordingly.
    CUSTOMER-SPECIFIC & GENERIC APIS
    ROLE OF HUMANS IN APIS

    View Slide

  37. goodapi.co
    HIRE MORE PEOPLE
    Complexity
    REALITY





    $
    HUMANS INSIDE
    VS.
    • time and money spent
    • not scaling
    • chance of
    misinterpretation
    • error-prone

    View Slide

  38. goodapi.co
    SOLUTIONS TO COMPLEXITY
    Complexity
    2. Standardization
    HARMONIZED APIS
    API DESIGN, LIFECYCLE &
    DEVELOPMENT STANDARDIZATION

    View Slide

  39. goodapi.co
    STANDARDIZATION
    Complexity
    API GUIDELINES & DESIGN CHECKS

    View Slide

  40. goodapi.co
    3. Massive Automation
    AUTOMATED HARMONIZED APIS, AUTONOMOUS APIS
    Complexity
    SOLUTIONS TO COMPLEXITY
    AUTOMATED CONTRACT-DRIVEN API
    LIFECYCLE

    View Slide

  41. goodapi.co



    INTERFACE
    ACTUALLY
    EXPOSED
    INTERFACE
    DOCUMENTED
    INTERFACED
    USED
    are they the same?
    are they the same?
    are they the same?
    Complexity
    AUTOMATION
    KEEPING THINGS IN CHECK IS HARD

    View Slide

  42. goodapi.co
    Complexity
    AUTOMATION
    CONTRACT-DRIVEN

    View Slide

  43. goodapi.co
    https://goodapi.co/api-lifecycle
    Complexity
    AUTOMATION
    ENTIRE LIFECYCLE

    View Slide

  44. goodapi.co
    4. Autonomy
    AUTONOMOUS APIS
    SOLUTIONS TO COMPLEXITY
    AUTONOMY IS THE SOLUTION TO THE DIFFICULTY
    OF MANAGING MANY COMPONENTS CENTRALLY
    Components autonomy in an API Landscape reduces the
    complexity, enables scaling, and promotes the service
    commoditization.
    Complexity

    View Slide

  45. goodapi.co
    AUTONOMY
    “Autonomy: freedom from external control or influence;
    independence; sovereignty in making decisions and
    changes.”
    –New Oxford American Dictionary
    Complexity

    View Slide

  46. goodapi.co
    AUTONOMOUS APIS
    4
    MACHINE-TO-MACHINE APIS, SELF-DRIVING, NO
    HUMAN INTERVENTION, CROSS-DOMAIN
    LANDSCAPES
    RISE OF IOT & AI
    (YEAR 2020+)

    View Slide

  47. goodapi.co
    Navigation through autonomic units
    Countries, Regions, Cities, City
    Quarters are autonomous regions
    but when you are driving from A to B
    you need to navigate through their
    roads
    A
    B
    Design-time & Runtime Navigation
    You need both to follow a map and
    traffic signs to navigate effectively
    4
    AUTONOMOUS APIS
    NAVIGATE SAFELY IN API LANDSCAPE

    View Slide

  48. goodapi.co
    –Autonomous Client
    “The level of autonomy is the ability to safely
    navigate in the API landscape (where the constraints
    are time and price)”
    4
    LEVELS OF AUTONOMY

    View Slide

  49. goodapi.co
    Brittle System
    Documented Brittle System
    Automated System
    Decoupled Automated System
    Autonomous System
    Autonomy
    Autonomous APIs
    0
    1
    2
    3
    4
    AUTONOMOUS APIS
    4

    View Slide

  50. goodapi.co
    • Tightly coupled components
    • No semantics, interface or
    lifecycle data
    • Pros
    • Time to delivery for point to
    point integrations when
    complexity is low
    • Cons
    • Everything breaks when
    something changes
    • Not scaling with growing
    complexity
    Level 0
    BRITTLE SYSTEM
    LEVELS OF AUTONOMY

    View Slide

  51. goodapi.co
    • Tightly coupled components
    • Interface metadata available
    • Human only-semantics
    metadata available
    • No lifecycle metadata (is the
    documentation documenting
    the interface implementation?)
    • Pros
    • Possible API productization
    • Reusability
    • Cons
    • Lock on changes, unable to evolve
    DOCUMENTED BRITTLE SYSTEM
    LEVELS OF AUTONOMY
    Level 1

    View Slide

  52. goodapi.co
    • Mostly tightly coupled
    components
    • Metadata are verified, valid and
    reliable
    • Metadata are human or machine
    readable
    • Advertisement of the interface,
    semantics and lifecycle metadata
    • Client subscription to future
    changes
    • Pros
    • Explicit understanding of price and
    risk of making changes in the API
    landscape
    • Quality Control
    • Cons
    • Advanced mindset change
    involving many people
    • Cultural denial, hard to get buy-in
    AUTOMATED SYSTEM
    LEVELS OF AUTONOMY
    Level 2

    View Slide

  53. goodapi.co
    • Uniform Interface
    • Decoupled components
    • Evolvability of the components
    in their respective environments
    • Share understanding-metadata
    at RUNTIME
    • Pros
    • Harmonized within the system
    • Reducing price and risk of making
    changes (by design)
    • Cons
    • Difficult taking everything into
    account, prone to chaotic & size
    complexity
    DECOUPLED AUTOMATED SYSTEM
    LEVELS OF AUTONOMY
    Level 3

    View Slide

  54. goodapi.co
    • Self driving clients
    • Landscape discovery
    • Machine understanding
    • Vocabulary not data structure
    • Declarative programming
    • Autonomy of the components
    • Leveraging learned data from
    simulation for infrastructure
    • Pros
    • Higher level of abstraction
    • Implicit harmonization
    • Full autonomy of components
    • Independent evolution
    • Cons
    • Cognitive (Algorithmic) Complexity
    • Demanding on computational power
    • Higher level of abstraction
    AUTONOMOUS SYSTEM
    LEVELS OF AUTONOMY
    Level 4

    View Slide

  55. goodapi.co
    goodapi.co
    Early four-engine jets required flight engineers.
    Later four-engine jets were designed with
    sufficient automation to eliminate the position.
    747 FLIGHT ENGINEERS
    ...this will also happen in the world of APIs.

    View Slide

  56. goodapi.co
    API CHAPTERS
    one-to-many
    one-to-one
    many-to-many
    machine-to-machine
    Drivers behind the changes
    API CONSUMERS EXPECTATIONS COMPLEXITY
    task-structure
    unpredictability
    chaotic
    size
    algorithmic
    +
    GROWING # OF API
    CONSUMERS
    +
    INCREASING DEMAND
    EVOLVING DEMAND INCREASING PROBLEMS

    View Slide

  57. goodapi.co
    1. CUSTOMER-SPECIFIC APIS 2. GENERIC APIS 3. HARMONIZED APIS 4. AUTONOMOUS APIS
    one-to-many
    one-to-one many-to-many machine-to-machine
    API CHAPTERS

    View Slide

  58. goodapi.co
    SERVICE COMMODITIZATION
    CUSTOMER SPECIFIC APIS (SOAP)
    GENERIC APIS (SO-CALLED-REST)
    HARMONIZED APIS
    AUTONOMOUS APIS
    Graph by Simon Wardley, “How commodity is something?”
    HARMONIZED APIS (AUTOMATED)

    View Slide

  59. goodapi.co
    DIGITAL WHOLESALE PRESENCE
    &
    SERVICE COMMODITIZATION
    “Look for a service that can fulfill the shipment of 6
    pallets from Prague to Paris under 100 EUR.”
    “Look for a service that provides information about
    adidas stan smith”
    “Is there an air condition service in my landscape?”
    “Look for a service that has gs1:Offer where
    gs1:itemOffered gs1:gpcCategoryCode is MILK.”
    “The level of autonomy is the
    ability to safely navigate in the API
    landscape (where the constraints
    are time and price)”
    REVOLUTION IN
    (API)
    ECONOMY

    View Slide

  60. goodapi.co
    GOOD
    API
    INDEPENDENT API CONSULTING
    WE ARE HIRING

    View Slide

  61. IMAGES BY UNSPLASH.COM
    ICONS BY MARTINFOWLER.COM

    View Slide