Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Change Management at Scale

Change Management at Scale

Reconciling process and agility

Pierre-Yves Ritschard

May 02, 2018
Tweet

More Decks by Pierre-Yves Ritschard

Other Decks in Technology

Transcript

  1. CHANGE MANAGEMENT AT CHANGE MANAGEMENT AT SCALE SCALE STAYING ORGANIZED

    IN THE FACE OF AGILITY STAYING ORGANIZED IN THE FACE OF AGILITY 1
  2. CTO at Exoscale: The European Cloud Provider Early devops implementer:

    Orange, Lagardere Early big-data & ML implementer: paper.li @pyr 2 . 1
  3. EXOSCALE EXOSCALE provider "exoscale" { api_key = "${var.exoscale_api_key}" secret_key =

    "${var.exoscale_secret_key}" } resource "exoscale_instance" "web" { template = "Ubuntu 17.04" disk_size = "50g" profile = "medium" ssh_key = "production" } 4 . 1
  4. WHAT'S CHANGE ? WHAT'S CHANGE ? The addition, modification or

    removal of anything that could have an effect on IT services 5 . 1
  5. WHY CHANGE MATTERS WHY CHANGE MATTERS IT used to be

    a support function Inventory Payroll Accounting 6 . 1
  6. WHAT'S IN AN APP WHAT'S IN AN APP Inventory User

    preferences Real-time analytics 8 . 1
  7. AIM OF THE TALK AIM OF THE TALK Addressing the

    tension between process, security, and agility 9 . 1
  8. CANONICAL USE-CASE: ADDING CANONICAL USE-CASE: ADDING OUTBOUND MAIL OUTBOUND MAIL

    Please open outbound TCP 25 Sounds like you want TCP 587 to our internal mailers <wait 2 weeks> 13 . 1
  9. STANDARDS STANDARDS ISO 27001 and ISO27018 CSA: A good basis

    for IaaS, PaaS and SaaS vendors ITIL: Best practices 17 . 1
  10. CMDB CMDB Configuration Management Database Holds configuration items and their

    relationship Somewhat conflates Asset Management and Configuration Management 25 . 1
  11. CHANGE MANAGEMENT CHANGE MANAGEMENT Defines change lifecycle RFCs and classification

    (standard, emergency, normal) Change Acceptance Board (CAB, ECAB) Change records 26 . 1
  12. SCALING PROCESS SCALING PROCESS New objectives Fast iteration cycle Reduced

    interference Not just startups & small orgs How do we map valid ITIL concerns with agile orgs ? 28 . 1
  13. SERVICE DESIGN SERVICE DESIGN The service catalog should be self-service

    This is much easier when co-opting IaaS 29 . 1
  14. SERVICE OPERATION SERVICE OPERATION Problem and incident management are most

    likely already well addressed Work needed on reducing communication latency 30 . 1
  15. CMDB CMDB Configuration Management Standard configuration management frameworks apply Cattle

    vs Pet approach has the underlying notion of service catalog Asset Management Virtualization and Cloud provide dynamic inventory 32 . 1
  16. CHANGE MANAGEMENT: RFCS CHANGE MANAGEMENT: RFCS Useful elements Motivation &

    Objective Tentative timeframe Integration & Non-regression tests May be polymorphic Text document Configuration management update Command and control recipe 34 . 1
  17. CHANGE MANAGEMENT: CAB CHANGE MANAGEMENT: CAB Peer review of runbooks

    Breaks long release cycle Reaches CAB objectives of traceability Recurring tasks can be auto-validated (standard changes) For instance adding a vhost A good prerequisite: no manual intervention 35 . 1
  18. CHANGE MANAGEMENT: CHANGE CHANGE MANAGEMENT: CHANGE RECORDS RECORDS Text based

    log of change Accessible by all parties at time of change 36 . 1
  19. SERVICE OPERATION SERVICE OPERATION Co-opt development tools to reduce barriers

    and friction Status pages for critical services 39 . 1
  20. CMDB IS EASY CMDB IS EASY You already have configuration

    management It's already stored in git # nodes.yaml node "network-lb01" { include network::lb } node "portal-front01" { include portal::front } node "portal-front02" { include portal::front } node "portal-front03" { include portal::front } node "portal-db01" { include portal::db } 41 . 1
  21. CHANGE MANAGEMENT IS LESS CHANGE MANAGEMENT IS LESS ADRESSED ADRESSED

    No common runbook tool No standard execution logging 42 . 1
  22. COMMON THEME: REUSE YOUR EXISTING COMMON THEME: REUSE YOUR EXISTING

    INFRASTRUCTURE INFRASTRUCTURE Pull requests are great for peer review Git provides a nice way to archive text data Introduce as few new tools as possible Avoid adding bloat with process 43 . 1
  23. EXAMPLE APPROACH: EXAMPLE APPROACH: RFCs Standard Changes: Config Mgmt or

    Command-and-Control recipe Normal Changes: Doc update for the corresponding service platform CAB: Pull requests Change records: Archived chat logs Hubot is great for this 44 . 1
  24. CAREFULLY WEIGHT FRICTION TO CAREFULLY WEIGHT FRICTION TO BENEFIT RATIO

    BENEFIT RATIO Don't lose sight of the objectives 49 . 1
  25. GO STEP BY STEP GO STEP BY STEP It's ok

    to still log-in if you need it Consider it's a failure and see how you can remediate 50 . 1