$30 off During Our Annual Pro Sale. View Details »

From 100 to 1,000+ deployments a day! [Techorama Edition]

From 100 to 1,000+ deployments a day! [Techorama Edition]

At Techorama Belgium, I presented the story of Coolblue, and more importantly, a few examples of what it took for us to go from 100, to 1,000+ deployments a day; a change in mindset, responsibilities and ownership.

Pat Hermens

May 24, 2018
Tweet

More Decks by Pat Hermens

Other Decks in Technology

Transcript

  1. View Slide

  2. From 100 to 1,000+
    deployments a day
    Pat Hermens
    @phermens

    View Slide

  3. From 100 to 1,000+ deployments
    What does it take to get to the next level?

    View Slide

  4. Pat Hermens
    Currently Development Manager
    Coding for 15+ years
    Father & husband
    Rotterdam

    View Slide

  5. 5
    @phermens

    View Slide

  6. View Slide

  7. View Slide

  8. Why 1,000+ deployments/day?

    View Slide

  9. 9
    @phermens
    ¯\_(ツ)_/¯

    View Slide

  10. 10
    @phermens
    “It depends”

    View Slide

  11. 11
    @phermens
    YAGNI
    & KISS

    View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. 100 / 1,000 / 10,000; so what?

    View Slide

  20. http://www.cs.utexas.edu/users/ewd/ewd02xx/ewd249.pdf

    View Slide

  21. … Apparently we are too much trained to disregard
    differences in scale, to treat them as “gradual differences that
    are not essential.”
    We tell ourselves that what we can do once, we can also do
    twice and by induction, we fool ourselves into believing that
    we can do it as many times as needed, but this is just not true!
    ...

    View Slide

  22. ... A one-year old child will crawl on all fours with a speed of,
    say, one mile per hour. But a speed of a thousand miles per
    hour is that of a supersonic jet.
    Considered as objects with moving ability the child and the jet
    are incomparable, for whatever one can do the other cannot
    and vice versa.
    — Edsger Dijkstra, 1969

    View Slide

  23. View Slide

  24. Prerequisites...

    View Slide

  25. View Slide

  26. 26
    @phermens
    The “faster to master” checklist
    Developing
    ⬜ Coding guidelines
    ⬜ Code reviews
    ⬜ Code analysis
    ⬜ Short-lived branches
    ⬜ Feature toggles
    Monitoring
    ⬜ Defined key metrics
    ⬜ Measure – create dashboards
    ⬜ Proactivity – add alerting (& act!)
    Testing
    ⬜ Unit tests
    ⬜ Module tests
    ⬜ Integration tests
    ⬜ End-to-end tests
    ⬜ Test automation
    Deploying
    ⬜ Continuous Integration
    ⬜ Continuous Delivery
    ⬜ Infrastructure as Code

    View Slide

  27. 27
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ☑ Code analysis
    ☑ Short-lived branches
    ☑ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ☑ Measure – create dashboards
    ☑ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ☑ Module tests
    ☑ Integration tests
    ☑ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ☑ Continuous Delivery
    ☑ Infrastructure as Code
    The “faster to master” checklist

    View Slide

  28. 28
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ☑ Code analysis
    ☑ Short-lived branches
    ☑ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ☑ Measure – create dashboards
    ☑ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ☑ Module tests
    ☑ Integration tests
    ☑ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ☑ Continuous Delivery
    ☑ Infrastructure as Code
    The “faster to master” checklist

    View Slide

  29. 29
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ☑ Code analysis
    ☑ Short-lived branches
    ☑ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ☑ Measure – create dashboards
    ☑ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ☑ Module tests
    ☑ Integration tests
    ☑ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ☑ Continuous Delivery
    ☑ Infrastructure as Code
    The “faster to master” checklist

    View Slide

  30. View Slide

  31. Story time!

    View Slide

  32. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  33. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  34. Responsibility

    View Slide

  35. 35
    @phermens
    Hosting &
    Deployment
    “DevOps”

    View Slide

  36. 36
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    B
    Team
    C

    View Slide

  37. 37
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    F
    Team
    B
    Team
    C
    Team
    E

    View Slide

  38. 38
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    F
    Team
    B
    Team
    C
    Team
    E

    View Slide

  39. 39
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C

    View Slide

  40. 40
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C

    View Slide

  41. 41
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C

    View Slide

  42. 42
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C

    View Slide

  43. 43
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C

    View Slide

  44. 44
    @phermens
    Team
    H
    Team
    A
    Team
    C
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    Team
    D
    Team
    L
    Team
    K

    View Slide

  45. 45
    @phermens

    View Slide

  46. 46
    @phermens
    Sorry Kasey!

    View Slide

  47. 47
    @phermens

    View Slide

  48. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  49. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  50. Autonomy

    View Slide

  51. 51
    @phermens

    View Slide

  52. 52
    @phermens
    Log4Net + RDP

    View Slide

  53. 53
    @phermens
    Serilog + Splunk + DataDog

    View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  59. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  60. 60
    @phermens
    Ownership

    View Slide

  61. 61
    @phermens
    Build System

    View Slide

  62. 62
    @phermens

    View Slide

  63. 63
    @phermens

    View Slide

  64. 64
    @phermens
    Development team tooling choices

    View Slide

  65. 65
    @phermens
    Infrastructure team tooling choices

    View Slide

  66. 66
    @phermens

    View Slide

  67. 67
    @phermens
    What’s next?

    View Slide

  68. 68
    @phermens
    ?
    What’s next?

    View Slide

  69. 69
    @phermens
    ¯\_(ツ)_/¯

    View Slide

  70. 70
    @phermens

    View Slide

  71. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  72. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  73. 73
    @phermens
    Fail early, fail often,
    but always fail forward.

    View Slide

  74. 74
    @phermens
    Failure

    View Slide

  75. 75
    @phermens

    View Slide

  76. View Slide

  77. View Slide

  78. View Slide

  79. View Slide

  80. 80
    @phermens

    View Slide

  81. View Slide

  82. 82
    @phermens

    View Slide

  83. 83
    @phermens

    View Slide

  84. View Slide

  85. 85
    @phermens
    #war-room

    View Slide

  86. 86
    @phermens

    View Slide

  87. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  88. Responsibility Autonomy
    Failure
    Ownership

    View Slide

  89. Questions?
    If not, drinks follow this talk!

    View Slide

  90. Checklists!

    View Slide

  91. 91
    @phermens
    The “faster to master” checklist
    Developing
    ⬜ Coding guidelines
    ⬜ Code reviews
    ⬜ Code analysis
    ⬜ Short-lived branches
    ⬜ Feature toggles
    Monitoring
    ⬜ Defined key metrics
    ⬜ Measure – create dashboards
    ⬜ Proactivity – add alerting (& act!)
    Testing
    ⬜ Unit tests
    ⬜ Module tests
    ⬜ Integration tests
    ⬜ End-to-end tests
    ⬜ Test automation
    Deploying
    ⬜ Continuous Integration
    ⬜ Continuous Delivery
    ⬜ Infrastructure as Code

    View Slide

  92. 92
    @phermens
    The “100 to 1,000” checklist
    Responsibility
    ⬜ Can I control my deployments?
    ⬜ Have I removed all bottlenecks?
    ⬜ Do I have good support systems?
    ⬜ Am I collaborating with my peers?
    Autonomy
    ⬜ Freedom to innovate/improve?
    ⬜ Am I in control and independent?
    ⬜ Can I find out what went wrong?
    ⬜ Do I have the skills to fix it?
    Ownership
    ⬜ Agreed interfaces or contracts?
    ⬜ Out-of-the-box solutions?
    ⬜ Tooling is 100% automated?
    ⬜ Local build == “Production” build?
    Failure
    ⬜ Monitoring = Alerting = Action
    ⬜ Take pride in improving
    ⬜ Small deliverables, quickly
    ⬜ Non-blocking deployments

    View Slide

  93. https://sddconf.com/brands/sdd/library/DOLLARD-Debugging.pdf

    View Slide

  94. 94
    @phermens
    Thanks
    https://hermens.com.au
    @phermens

    View Slide

  95. https://hermens.com.au
    [email protected]
    Pat Hermens
    @phermens
    Thanks!

    View Slide