Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Coders to Builders of the Intelligent World

D8004857fc10614cfa4dec1bae20f874?s=47 Jez Humble
October 12, 2018

From Coders to Builders of the Intelligent World


Jez Humble

October 12, 2018


  1. From coders to builders of the intelligent world software engineering

    for AI: invariants and unsolved problems Jez Humble CTO, DevOps Research and Assessment LLC
  2. @jezhumble abstract • a very short history of devops •

    a brief introduction to ml delivery lifecycle • what stays the same • what changes • goals and unsolved problems
  3. devops Idea Business Delivery Operation Market Agile addresses this DevOps

    addresses this
  4. @jezhumble devops movement a cross-functional community of practice dedicated to

    the study of building, evolving and operating rapidly changing, secure, resilient systems at scale
  5. @jezhumble the three ways

  6. @jezhumble time to restore service lead time for changes (checkin

    to release) deploy frequency change fail rate first way: metrics for software delivery performance
  7. @jezhumble feedback loops “so much of creation is discovery, and

    you can’t discover anything if you can’t see what you’re doing.” — Bret Victor Bret Victor, Inventing on Principle, http://vimeo.com/36579366
  8. @jezhumble feedback loops

  9. @jezhumble machine learning development lifecycle Data ETL Select algorithm Train

    model Validate model Data ETL Production
  10. @jezhumble mldlc vs sdlc sdlc mldlc materials code algorithms, scenarios,

    data production package model configuration management code versioning, infrastructure-as-code, api versioning data & data dictionary management, model & platform versioning, api versioning continuous integration constantly validating behavior of code against tests constantly validating model against scenarios and data continuous delivery always ready to deploy to production and smart devices (ios / android) always ready to deploy to production and smart devices (edge) observability / care and feeding instrument code, monitoring & alerting infrastructure collect model accuracy, continuous model training - data feeds back into training thanks to 勇张
  11. @jezhumble what’s the same? • toolchain for deployment pipeline •

    platform for testing, training, and production deployment • optimize for short lead times / tight feedback loops • tdd
  12. @jezhumble what’s different? • training lead times • data management

    - only possible with teams • data pipeline as well as delivery pipeline • allocate R&D time for algorithm selection and model training • edge: hardware heterogeneity - interface hell
  13. @jezhumble real systems

  14. @jezhumble edge: lessons from microservices • each model independently verifiable

    • how to avoid big up-front design for data schemas / apis? • how to avoid chatty, fine-grained communication? • security is an emergent property
  15. @jezhumble the goal exploit and elevate the constraints: hardware and

    r&d time • invest in ml infrastructure: toolchains and platforms • for developers, training and validating models, data pipelines / ETL, deployment to the cloud and edge, instrumentation and ongoing training • comprehensive data and configuration management • get visibility into - and optimize for - lead times
  16. @jezhumble unsolved problems • development and training lead times •

    large-scale development • understanding and debugging ml models • building and operating highly distributed systems • ml hype
  17. thank you! © 2016-18 Jez Humble& Associates LLC https://continuous-delivery.com/ To

    receive the following: • A copy of this presentation • The link to the 2018 Accelerate State of DevOps Report (and previous years) • A 100 page excerpt from Lean Enterprise • Excerpts from the DevOps Handbook and Accelerate • 30% off my video workshop: creating high performance organizations • A 20m preview of my Continuous Delivery video workshop • Discount code for CD video + interviews with Eric Ries & more Just pick up your phone and send an email To: jezhumble@sendyourslides.com Subject: devops