Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Are we there yet? Building an internal developm...

Are we there yet? Building an internal development platform

“We’ll get there.” The words you say to the preverbal “Are we there yet?” Building an internal development platform is not so much of a choice as it is an eventuality. The only question is are you intentionally building one or not? Undifferentiating engineering is a requirement for all software companies, not everything you do will set you apart from your competitors and generate value directly for your customers. The question is how do you do exactly the right amount of undifferentiating engineering to create the most value for your customers? In this talk I will discuss a decade of hard lessons I’ve learned about building internal development platforms, most recently with the Interstate Platform at Total Expert. What are the pitfalls, the fun part, the hard part, and the really really hard part that you should focus on if you want to have a successful IDP program at your company?

Andy Fleener

May 25, 2023
Tweet

More Decks by Andy Fleener

Other Decks in Technology

Transcript

  1. @andy fl [email protected] Are we there yet? Building an Internal

    Development Platform By Andy Fleener https:// fl ickr.com/photos/96dpi/4771273008/in/gallery-maven-72157622717379143/
  2. @andy fl [email protected] Let’s take a Road Trip https:// fl

    ickr.com/photos/96dpi/4796200235/in/gallery-maven-72157622717379143/
  3. @andy fl [email protected] More about the journey than the destination

    https:// fl ickr.com/photos/193347187@N08/52045942829/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  4. @andy fl [email protected] Sometimes you will fi nd things you

    didn’t anticipate https:// fl ickr.com/photos/22691568@N04/43656057080/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj- HAHtHV-2nNdaoU-DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS- QkbJ8X-2kQLJ6m-2ni7UXo/
  5. @andy fl [email protected] The anticipation of arrival always feels like

    it took longer than it did Are we there yet? 🙃 https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  6. @andy fl [email protected] Find a balance of the right number

    of stops along the way https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  7. @andy fl [email protected] What we’re going to cover today What

    is a Platform How do you know if you need one How do you go about building one How do you migrate to it And how do you evolve it
  8. @andy fl [email protected] "A platform is a foundation of self-service

    APIs tools services knowledge and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace with reduced coordination" – Evan Bottcher
  9. @andy fl [email protected] "A thick platform might consist of the

    combination of several inner platform teams providing a myriad of services. A thin platform could simply be a layer on top of a vendor-provided solution." – Team Topologies book chapter on Platform Teams
  10. @andy fl [email protected] –Steven Lohrenz on thenewstack “Platform engineering is

    the process of designing, building, and maintaining work fl ows and tools for software engineering organizations to drive consistency and speed up common tasks.“
  11. @andy fl [email protected] Undifferentiated Engineering Not everything you do will

    provide direct value to your customers Some of the most valuable work is improving the way you do work https:// fl ickr.com/photos/darea62/49395809012/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  12. @andy fl [email protected] Platforms serve at least 3 purposes Limit

    Entropy – Drive consistency Gain Ef fi ciency – Speeding up common tasks = cheaper & faster Reduce Cognitive-Load – Shrink the surface area of what an average engineer needs to know https:// fl ickr.com/photos/stillugly/52045913916/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  13. @andy fl [email protected] “Platform engineering is a new fi eld

    that has arisen to meet the growing complexity of distributed software development. As software development has started involving more tools, clouds and work fl ows, it’s become increasingly dif fi cult for software developers who are now asked to manage more different aspects that go into writing robust, secure, performant and capable applications.” –Steven Lohrenz on thenewstack
  14. @andy fl [email protected] What kind of Platform you need? 


    It depends: What is your Business Strategy? What is your Organizational Strategy? What is your Technical Strategy? https:// fl ickr.com/photos/btonevibes/22926643566/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  15. @andy fl [email protected] Remember our Goals Limit Entropy – Drive

    consistency Gain Ef fi ciency – Speeding up common tasks = cheaper & faster Reduce Cognitive-Load – Shrink the surface area of what an average engineer needs to know https:// fl ickr.com/photos/stillugly/52045913916/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  16. @andy fl [email protected] 80% talking to people to understand the

    problem https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  17. @andy fl [email protected] Rule #1 Build trust with people who

    are going to use the platform (i.e. your coworkers)
  18. @andy fl [email protected] Without established trust you will NOT get

    enough feedback to build the platform that your company actually needs
  19. @andy fl [email protected] We all have more work to do

    than time to do it Do not expect documentation to do marketing for you Do expect you will be co-creating the platform with the people who are going to use it Be AVAILABLE https:// fl ickr.com/photos/146264147@N03/34309143893/in/photolist-UgMkaF-ViNQtg-UVcKkb-UgMkzt-UgMef4-Vvna6R-ViNX88-Vvn8wt-QFj65b-UgMgG8-VrShV5-UgMeHt-ViP4Gv-UVd7Gj- UVdams-UVcK1d-ViNMFH-Vvn6Yi-ViNVAF-UgMjPa-ViP364-ViNSpR-UgMpik-VfPjYw-UVd8LJ-UVcHcJ-VfPkCY-UVcHEY-ViNGnB-ViNEfv-UVcMMf-VfPjoy-ViNXzF-ViNP8a-UgMGi2-PsLRqg- UgMzzx-UdK2nj-ViNETe-UgMEj2-Vvn8SP-ViP6bc-UdKdsL-VfPC9U-UVd5vW-UgMFfR-9SpDQo-PsLT2x-Pq5qHU-QteCR5
  20. @andy fl [email protected] 10% deciding what not to build https://

    fl ickr.com/photos/elka_cz/729827561/in/photolist-27uyiP-73hcvn-2kcGj3P-7NpeXr-2nMqn6o-4VPSMB-B8aWt-oBhfRi-2jfD6kA-paGeSy-2jPMQkp-qPB8tc-PRKjQe-2ez2Umr-CqJyt- b2D2CZ-2efotfR-8Hu1s7-DuqTmW-FE2zj-NyMkg-2mV7wM4-tEqi4-H5vn4d-J2Wafj-mH1jFr-2gBKUWY-q8C5fr-2NzaNE-5FjeTP-pbJ2tP-5bsEE2-v4NwbF-qC1ZTA-2iFWqSr-NQpRM- oyGDTd-27ZAWym-2d3FSJ-6drNuG-bpkp5K-4T2qsD-73haYc-7rJgaf-29ZSBi-8gaRmr-X1y1e7-eUgA2h-eU5dun-CmSthQ/
  21. @andy fl [email protected] Deciding what your platform won’t do –

    What constraints it’s going to have, is just as important as deciding what it will do.
  22. @andy fl [email protected] 10% crushing hopes and dreams https:// fl

    ickr.com/photos/96dpi/4796200235/in/gallery-maven-72157622717379143/
  23. @andy fl [email protected] Your stakeholders will want some things that

    are impossible to deliver https:// fl ickr.com/photos/83257111@N04/49810217188/in/photolist-2hU6yWR-2iTymiP-2iTymCB-2iTB8CW-2iTyntj-2iTBcCp-2iTynXA-2iTyod5-2iTBdFg-2m5xpiH/
  24. @andy fl [email protected] It’s more important to make good on

    the promises you make than to make promises to everyone
  25. @andy fl [email protected] Always be Migrating Start small - fi

    nd the smallest batch you can Avoid one way doors State is the worst https:// fl ickr.com/photos/joeyz51/52520157154/in/photolist-2o22wWW-adVm7U-65ZK9Q-5JfPrW-2hYoBJb-JmdEhd-65t1o5-akvxGR-5bpXHb-doCmFh-XTdQx7-5Cz9FA-d8giUE-8HD6u3-2881ZAk- aEJTVh-2RUBSs-2hHvHY5-e4WbEo-ojmgLA-6QVnkB-2oAXc9b-oCBex2-2nALF3c-jRaZaF-z6LE2k-8Y2YpP-ojmcVQ-oyPnwG-2kSn1kd-abeSDx-2jjoD7r-dyU6k9-2kSoAvs-cyx8GY- oox8Mb-9gKpNh-2kSoB2c-6FJAod-a5f6dz-2m1XcK9-9gGjKn-QZmMN9-MSTd4f-fa455V-SodF6b-2mVuyJN-fa452c-rQbjUJ-fa459D
  26. @andy fl [email protected] Small Batches Find the smallest workload you

    can to migrate fi rst Shrink the feedback loop Small things are easier to reason about https:// fl ickr.com/photos/pedrosimoes7/50111796202/in/photolist-Y1Bfvk-2jmd49f-WqdBU7-WyRgtP-26GUF8B-r9pt4m-ZnEQrY-afmChE-2gMpuj2-oEKBFS-2m6FD8T-9enLYa-2jhb4jp- N8G8jW-2jdqV6J-2jhJx6L-buBUFY-Jq2Hb6-2kZw3jf-2kZshPP-2gN7jJh-2m8qjtZ-2b9sgQA-hPywe-KgKQ9p-2m6KiBE-2hxTU6m-rd2BWJ-2iyjTpw-8RQ5s2-QGpcDm-2o3NKpa-2nmhM9X-2c3kTjd- dPoUPA-NbcvrP-dtR3cH-291vbFi-2gxSaD6-7U8GH3-bVRXUM-Rtb5ry-CzEqeh-2k62p55-ZUD4zf-H1PbWH-2kV52pF-fwdGUz-2jZRWNm-26fhaoj/
  27. @andy fl [email protected] No One-way Doors Leave yourself a way

    out Run on both platforms at the same time Make the switch a boring event https:// fl ickr.com/photos/kirinqueen/631677659/in/photolist-XPvL8-cC7KMC-WeRH21-21cyq1q-XB9tAu-Q9Mz95-GCVU8w-24tnQCe-aqAQfN-s5BESG-271PJfa-7q8u8g-teakiP-9n282i-EaVRe9-4zeuzg- H4whv2-j6iQ65-f7VbaQ-224wvSj-cLVyZL-s8GBwT-2cY2Bs4-jz3Eb9-G5Ap9-oCyzcZ-MEGyXE-RXNEDX-gwGrBB-4z5wrn-pvpmm6-awL3NP-J3qg4t-2w6wH8-8GkX5N-F2HV7s-hbZnc1-sxtJxG- Ny1jBN-2bcw1QD-7VZ7bT-27V7Bn5-EjNRg9-24pXXkN-4YXvji-LQsxkY-p94yVw-gi5EgL-Sou2BN-pZYpPK/
  28. @andy fl [email protected] State is literally the WORST Find ways

    to manage state separately from your compute Use replication and failover systems where you can Build state veri fi cation tests in places where you have concern https:// fl ickr.com/photos/193347187@N08/52076533882/in/photolist-2nkPRcQ-BcGuU3-2hzjYK3-2ixi5bd-6knt5h-HXJyhV-BUA5B1-2fuCWMm-2iGNkem-9XoPqS-2hTfFWV-2mNLJwN-2kZkcJA-2mzyNxG- LkyL1o-2oswHv1-4f9L7Q-298RPMf-QZV7M6-2jHFhc5-2jPVqZ6-2kcwDJJ-2couQTd-2h38e9p-4QtfsF-HLuGEA-2iT9one-2bNYTh-2nK99i4-8iCuT6-2mny5Py-2h3w1Dy-GFxVte-6JFVwp-2mzxz61-2jLUHpd- Edrort-x4nqDA-2iUtpPJ-Hpr6tL-3ZTPUB-p6MaX-8ievKV-SNcvbG-2kreWZf-7MH4Ug-2kaiEg7-2g8ATeN-emSbWi-dPzkVc
  29. @andy fl [email protected] Find a balance of the right number

    of stops along the way https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  30. @andy fl [email protected] The anticipation of arrival always feels like

    it took longer than it did We got there! https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  31. @andy fl [email protected] Sometimes you will fi nd things you

    didn’t anticipate Be poised to adapt https:// fl ickr.com/photos/22691568@N04/43656057080/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  32. @andy fl [email protected] More about the journey than the destination

    Invest in the process to evolve the platform https:// fl ickr.com/photos/193347187@N08/52045942829/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK-usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ- s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS- QkbJ8X-2kQLJ6m-2ni7UXo/