From coders to builders of the intelligent world software engineering for AI: invariants and unsolved problems Jez Humble CTO, DevOps Research and Assessment LLC

@jezhumble abstract • a very short history of devops • a brief introduction to ml delivery lifecycle • what stays the same • what changes • goals and unsolved problems

devops Idea Business Delivery Operation Market Agile addresses this DevOps addresses this

@jezhumble devops movement a cross-functional community of practice dedicated to the study of building, evolving and operating rapidly changing, secure, resilient systems at scale

@jezhumble the three ways

@jezhumble time to restore service lead time for changes (checkin to release) deploy frequency change fail rate first way: metrics for software delivery performance

@jezhumble feedback loops “so much of creation is discovery, and you can’t discover anything if you can’t see what you’re doing.” — Bret Victor Bret Victor, Inventing on Principle, http://vimeo.com/36579366

@jezhumble feedback loops

@jezhumble machine learning development lifecycle Data ETL Select algorithm Train model Validate model Data ETL Production

@jezhumble mldlc vs sdlc sdlc mldlc materials code algorithms, scenarios, data production package model configuration management code versioning, infrastructure-as-code, api versioning data & data dictionary management, model & platform versioning, api versioning continuous integration constantly validating behavior of code against tests constantly validating model against scenarios and data continuous delivery always ready to deploy to production and smart devices (ios / android) always ready to deploy to production and smart devices (edge) observability / care and feeding instrument code, monitoring & alerting infrastructure collect model accuracy, continuous model training - data feeds back into training thanks to 勇张

@jezhumble what’s the same? • toolchain for deployment pipeline • platform for testing, training, and production deployment • optimize for short lead times / tight feedback loops • tdd

@jezhumble what’s different? • training lead times • data management - only possible with teams • data pipeline as well as delivery pipeline • allocate R&D time for algorithm selection and model training • edge: hardware heterogeneity - interface hell

@jezhumble real systems

@jezhumble edge: lessons from microservices • each model independently verifiable • how to avoid big up-front design for data schemas / apis? • how to avoid chatty, fine-grained communication? • security is an emergent property

@jezhumble the goal exploit and elevate the constraints: hardware and r&d time • invest in ml infrastructure: toolchains and platforms • for developers, training and validating models, data pipelines / ETL, deployment to the cloud and edge, instrumentation and ongoing training • comprehensive data and configuration management • get visibility into - and optimize for - lead times

@jezhumble unsolved problems • development and training lead times • large-scale development • understanding and debugging ml models • building and operating highly distributed systems • ml hype

thank you! © 2016-18 Jez Humble& Associates LLC https://continuous-delivery.com/ To receive the following: • A copy of this presentation • The link to the 2018 Accelerate State of DevOps Report (and previous years) • A 100 page excerpt from Lean Enterprise • Excerpts from the DevOps Handbook and Accelerate • 30% off my video workshop: creating high performance organizations • A 20m preview of my Continuous Delivery video workshop • Discount code for CD video + interviews with Eric Ries & more Just pick up your phone and send an email To: jezhumble@sendyourslides.com Subject: devops