Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Coders to Builders of the Intelligent World

Jez Humble
October 12, 2018

From Coders to Builders of the Intelligent World

Jez Humble

October 12, 2018
Tweet

More Decks by Jez Humble

Other Decks in Technology

Transcript

  1. From coders to builders of the intelligent world
    software engineering for AI: invariants and unsolved problems
    Jez Humble
    CTO, DevOps Research and Assessment LLC

    View Slide

  2. @jezhumble
    abstract
    • a very short history of devops
    • a brief introduction to ml delivery lifecycle
    • what stays the same
    • what changes
    • goals and unsolved problems

    View Slide

  3. devops
    Idea Business Delivery Operation Market
    Agile
    addresses this
    DevOps
    addresses this

    View Slide

  4. @jezhumble
    devops movement
    a cross-functional community of practice dedicated to the study of
    building, evolving and operating rapidly changing, secure, resilient
    systems at scale

    View Slide

  5. @jezhumble
    the three ways

    View Slide

  6. @jezhumble
    time to restore service
    lead time for changes (checkin to release)
    deploy frequency
    change fail rate
    first way: metrics for software delivery performance

    View Slide

  7. @jezhumble
    feedback loops
    “so much of creation is discovery, and you can’t discover anything if
    you can’t see what you’re doing.”
    — Bret Victor
    Bret Victor, Inventing on Principle, http://vimeo.com/36579366

    View Slide

  8. @jezhumble
    feedback loops

    View Slide

  9. @jezhumble
    machine learning development lifecycle
    Data ETL
    Select
    algorithm
    Train model
    Validate model
    Data ETL
    Production

    View Slide

  10. @jezhumble
    mldlc vs sdlc
    sdlc mldlc
    materials code algorithms, scenarios, data
    production package model
    configuration management
    code versioning, infrastructure-as-code, api
    versioning
    data & data dictionary management, model
    & platform versioning, api versioning
    continuous integration
    constantly validating behavior of code
    against tests
    constantly validating model against
    scenarios and data
    continuous delivery
    always ready to deploy to production and
    smart devices (ios / android)
    always ready to deploy to production and
    smart devices (edge)
    observability /
    care and feeding
    instrument code, monitoring & alerting
    infrastructure
    collect model accuracy, continuous model
    training - data feeds back into training
    thanks to 勇张

    View Slide

  11. @jezhumble
    what’s the same?
    • toolchain for deployment pipeline
    • platform for testing, training, and production deployment
    • optimize for short lead times / tight feedback loops
    • tdd

    View Slide

  12. @jezhumble
    what’s different?
    • training lead times
    • data management - only possible with teams
    • data pipeline as well as delivery pipeline
    • allocate R&D time for algorithm selection and model training
    • edge: hardware heterogeneity - interface hell

    View Slide

  13. @jezhumble
    real systems

    View Slide

  14. @jezhumble
    edge: lessons from microservices
    • each model independently verifiable
    • how to avoid big up-front design for data schemas / apis?
    • how to avoid chatty, fine-grained communication?
    • security is an emergent property

    View Slide

  15. @jezhumble
    the goal
    exploit and elevate the constraints: hardware and r&d time
    • invest in ml infrastructure: toolchains and platforms
    • for developers, training and validating models, data
    pipelines / ETL, deployment to the cloud and edge,
    instrumentation and ongoing training
    • comprehensive data and configuration management
    • get visibility into - and optimize for - lead times

    View Slide

  16. @jezhumble
    unsolved problems
    • development and training lead times
    • large-scale development
    • understanding and debugging ml models
    • building and operating highly distributed systems
    • ml hype

    View Slide

  17. thank you!
    © 2016-18 Jez Humble& Associates LLC
    https://continuous-delivery.com/
    To receive the following:
    • A copy of this presentation
    • The link to the 2018 Accelerate State of DevOps Report (and previous years)
    • A 100 page excerpt from Lean Enterprise
    • Excerpts from the DevOps Handbook and Accelerate
    • 30% off my video workshop: creating high performance organizations
    • A 20m preview of my Continuous Delivery video workshop
    • Discount code for CD video + interviews with Eric Ries & more
    Just pick up your phone and send an email
    To: [email protected]
    Subject: devops

    View Slide