Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Are we there yet? Building an internal developm...

Are we there yet? Building an internal development platform

“We’ll get there.” The words you say to the preverbal “Are we there yet?” Building an internal development platform is not so much of a choice as it is an eventuality. The only question is are you intentionally building one or not? Undifferentiating engineering is a requirement for all software companies, not everything you do will set you apart from your competitors and generate value directly for your customers. The question is how do you do exactly the right amount of undifferentiating engineering to create the most value for your customers? In this talk I will discuss a decade of hard lessons I’ve learned about building internal development platforms, most recently with the Interstate Platform at Total Expert. What are the pitfalls, the fun part, the hard part, and the really really hard part that you should focus on if you want to have a successful IDP program at your company?

Andy Fleener

May 25, 2023
Tweet

More Decks by Andy Fleener

Other Decks in Technology

Transcript

  1. @andy fl eener@hachyderm.io Are we there yet? Building an Internal

    Development Platform By Andy Fleener https:// fl ickr.com/photos/96dpi/4771273008/in/gallery-maven-72157622717379143/
  2. @andy fl eener@hachyderm.io Let’s take a Road Trip https:// fl

    ickr.com/photos/96dpi/4796200235/in/gallery-maven-72157622717379143/
  3. @andy fl eener@hachyderm.io More about the journey than the destination

    https:// fl ickr.com/photos/193347187@N08/52045942829/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  4. @andy fl eener@hachyderm.io Sometimes you will fi nd things you

    didn’t anticipate https:// fl ickr.com/photos/22691568@N04/43656057080/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj- HAHtHV-2nNdaoU-DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS- QkbJ8X-2kQLJ6m-2ni7UXo/
  5. @andy fl eener@hachyderm.io The anticipation of arrival always feels like

    it took longer than it did Are we there yet? 🙃 https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  6. @andy fl eener@hachyderm.io Find a balance of the right number

    of stops along the way https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  7. @andy fl eener@hachyderm.io What we’re going to cover today What

    is a Platform How do you know if you need one How do you go about building one How do you migrate to it And how do you evolve it
  8. @andy fl eener@hachyderm.io "A platform is a foundation of self-service

    APIs tools services knowledge and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace with reduced coordination" – Evan Bottcher
  9. @andy fl eener@hachyderm.io "A thick platform might consist of the

    combination of several inner platform teams providing a myriad of services. A thin platform could simply be a layer on top of a vendor-provided solution." – Team Topologies book chapter on Platform Teams
  10. @andy fl eener@hachyderm.io –Steven Lohrenz on thenewstack “Platform engineering is

    the process of designing, building, and maintaining work fl ows and tools for software engineering organizations to drive consistency and speed up common tasks.“
  11. @andy fl eener@hachyderm.io Undifferentiated Engineering Not everything you do will

    provide direct value to your customers Some of the most valuable work is improving the way you do work https:// fl ickr.com/photos/darea62/49395809012/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  12. @andy fl eener@hachyderm.io Platforms serve at least 3 purposes Limit

    Entropy – Drive consistency Gain Ef fi ciency – Speeding up common tasks = cheaper & faster Reduce Cognitive-Load – Shrink the surface area of what an average engineer needs to know https:// fl ickr.com/photos/stillugly/52045913916/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  13. @andy fl eener@hachyderm.io “Platform engineering is a new fi eld

    that has arisen to meet the growing complexity of distributed software development. As software development has started involving more tools, clouds and work fl ows, it’s become increasingly dif fi cult for software developers who are now asked to manage more different aspects that go into writing robust, secure, performant and capable applications.” –Steven Lohrenz on thenewstack
  14. @andy fl eener@hachyderm.io What kind of Platform you need? 


    It depends: What is your Business Strategy? What is your Organizational Strategy? What is your Technical Strategy? https:// fl ickr.com/photos/btonevibes/22926643566/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  15. @andy fl eener@hachyderm.io Remember our Goals Limit Entropy – Drive

    consistency Gain Ef fi ciency – Speeding up common tasks = cheaper & faster Reduce Cognitive-Load – Shrink the surface area of what an average engineer needs to know https:// fl ickr.com/photos/stillugly/52045913916/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU- DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  16. @andy fl eener@hachyderm.io 80% talking to people to understand the

    problem https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  17. @andy fl eener@hachyderm.io Rule #1 Build trust with people who

    are going to use the platform (i.e. your coworkers)
  18. @andy fl eener@hachyderm.io Without established trust you will NOT get

    enough feedback to build the platform that your company actually needs
  19. @andy fl eener@hachyderm.io We all have more work to do

    than time to do it Do not expect documentation to do marketing for you Do expect you will be co-creating the platform with the people who are going to use it Be AVAILABLE https:// fl ickr.com/photos/146264147@N03/34309143893/in/photolist-UgMkaF-ViNQtg-UVcKkb-UgMkzt-UgMef4-Vvna6R-ViNX88-Vvn8wt-QFj65b-UgMgG8-VrShV5-UgMeHt-ViP4Gv-UVd7Gj- UVdams-UVcK1d-ViNMFH-Vvn6Yi-ViNVAF-UgMjPa-ViP364-ViNSpR-UgMpik-VfPjYw-UVd8LJ-UVcHcJ-VfPkCY-UVcHEY-ViNGnB-ViNEfv-UVcMMf-VfPjoy-ViNXzF-ViNP8a-UgMGi2-PsLRqg- UgMzzx-UdK2nj-ViNETe-UgMEj2-Vvn8SP-ViP6bc-UdKdsL-VfPC9U-UVd5vW-UgMFfR-9SpDQo-PsLT2x-Pq5qHU-QteCR5
  20. @andy fl eener@hachyderm.io 10% deciding what not to build https://

    fl ickr.com/photos/elka_cz/729827561/in/photolist-27uyiP-73hcvn-2kcGj3P-7NpeXr-2nMqn6o-4VPSMB-B8aWt-oBhfRi-2jfD6kA-paGeSy-2jPMQkp-qPB8tc-PRKjQe-2ez2Umr-CqJyt- b2D2CZ-2efotfR-8Hu1s7-DuqTmW-FE2zj-NyMkg-2mV7wM4-tEqi4-H5vn4d-J2Wafj-mH1jFr-2gBKUWY-q8C5fr-2NzaNE-5FjeTP-pbJ2tP-5bsEE2-v4NwbF-qC1ZTA-2iFWqSr-NQpRM- oyGDTd-27ZAWym-2d3FSJ-6drNuG-bpkp5K-4T2qsD-73haYc-7rJgaf-29ZSBi-8gaRmr-X1y1e7-eUgA2h-eU5dun-CmSthQ/
  21. @andy fl eener@hachyderm.io Deciding what your platform won’t do –

    What constraints it’s going to have, is just as important as deciding what it will do.
  22. @andy fl eener@hachyderm.io 10% crushing hopes and dreams https:// fl

    ickr.com/photos/96dpi/4796200235/in/gallery-maven-72157622717379143/
  23. @andy fl eener@hachyderm.io Your stakeholders will want some things that

    are impossible to deliver https:// fl ickr.com/photos/83257111@N04/49810217188/in/photolist-2hU6yWR-2iTymiP-2iTymCB-2iTB8CW-2iTyntj-2iTBcCp-2iTynXA-2iTyod5-2iTBdFg-2m5xpiH/
  24. @andy fl eener@hachyderm.io It’s more important to make good on

    the promises you make than to make promises to everyone
  25. @andy fl eener@hachyderm.io Always be Migrating Start small - fi

    nd the smallest batch you can Avoid one way doors State is the worst https:// fl ickr.com/photos/joeyz51/52520157154/in/photolist-2o22wWW-adVm7U-65ZK9Q-5JfPrW-2hYoBJb-JmdEhd-65t1o5-akvxGR-5bpXHb-doCmFh-XTdQx7-5Cz9FA-d8giUE-8HD6u3-2881ZAk- aEJTVh-2RUBSs-2hHvHY5-e4WbEo-ojmgLA-6QVnkB-2oAXc9b-oCBex2-2nALF3c-jRaZaF-z6LE2k-8Y2YpP-ojmcVQ-oyPnwG-2kSn1kd-abeSDx-2jjoD7r-dyU6k9-2kSoAvs-cyx8GY- oox8Mb-9gKpNh-2kSoB2c-6FJAod-a5f6dz-2m1XcK9-9gGjKn-QZmMN9-MSTd4f-fa455V-SodF6b-2mVuyJN-fa452c-rQbjUJ-fa459D
  26. @andy fl eener@hachyderm.io Small Batches Find the smallest workload you

    can to migrate fi rst Shrink the feedback loop Small things are easier to reason about https:// fl ickr.com/photos/pedrosimoes7/50111796202/in/photolist-Y1Bfvk-2jmd49f-WqdBU7-WyRgtP-26GUF8B-r9pt4m-ZnEQrY-afmChE-2gMpuj2-oEKBFS-2m6FD8T-9enLYa-2jhb4jp- N8G8jW-2jdqV6J-2jhJx6L-buBUFY-Jq2Hb6-2kZw3jf-2kZshPP-2gN7jJh-2m8qjtZ-2b9sgQA-hPywe-KgKQ9p-2m6KiBE-2hxTU6m-rd2BWJ-2iyjTpw-8RQ5s2-QGpcDm-2o3NKpa-2nmhM9X-2c3kTjd- dPoUPA-NbcvrP-dtR3cH-291vbFi-2gxSaD6-7U8GH3-bVRXUM-Rtb5ry-CzEqeh-2k62p55-ZUD4zf-H1PbWH-2kV52pF-fwdGUz-2jZRWNm-26fhaoj/
  27. @andy fl eener@hachyderm.io No One-way Doors Leave yourself a way

    out Run on both platforms at the same time Make the switch a boring event https:// fl ickr.com/photos/kirinqueen/631677659/in/photolist-XPvL8-cC7KMC-WeRH21-21cyq1q-XB9tAu-Q9Mz95-GCVU8w-24tnQCe-aqAQfN-s5BESG-271PJfa-7q8u8g-teakiP-9n282i-EaVRe9-4zeuzg- H4whv2-j6iQ65-f7VbaQ-224wvSj-cLVyZL-s8GBwT-2cY2Bs4-jz3Eb9-G5Ap9-oCyzcZ-MEGyXE-RXNEDX-gwGrBB-4z5wrn-pvpmm6-awL3NP-J3qg4t-2w6wH8-8GkX5N-F2HV7s-hbZnc1-sxtJxG- Ny1jBN-2bcw1QD-7VZ7bT-27V7Bn5-EjNRg9-24pXXkN-4YXvji-LQsxkY-p94yVw-gi5EgL-Sou2BN-pZYpPK/
  28. @andy fl eener@hachyderm.io State is literally the WORST Find ways

    to manage state separately from your compute Use replication and failover systems where you can Build state veri fi cation tests in places where you have concern https:// fl ickr.com/photos/193347187@N08/52076533882/in/photolist-2nkPRcQ-BcGuU3-2hzjYK3-2ixi5bd-6knt5h-HXJyhV-BUA5B1-2fuCWMm-2iGNkem-9XoPqS-2hTfFWV-2mNLJwN-2kZkcJA-2mzyNxG- LkyL1o-2oswHv1-4f9L7Q-298RPMf-QZV7M6-2jHFhc5-2jPVqZ6-2kcwDJJ-2couQTd-2h38e9p-4QtfsF-HLuGEA-2iT9one-2bNYTh-2nK99i4-8iCuT6-2mny5Py-2h3w1Dy-GFxVte-6JFVwp-2mzxz61-2jLUHpd- Edrort-x4nqDA-2iUtpPJ-Hpr6tL-3ZTPUB-p6MaX-8ievKV-SNcvbG-2kreWZf-7MH4Ug-2kaiEg7-2g8ATeN-emSbWi-dPzkVc
  29. @andy fl eener@hachyderm.io Find a balance of the right number

    of stops along the way https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  30. @andy fl eener@hachyderm.io The anticipation of arrival always feels like

    it took longer than it did We got there! https:// fl ickr.com/photos/schepers/48761144222/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  31. @andy fl eener@hachyderm.io Sometimes you will fi nd things you

    didn’t anticipate Be poised to adapt https:// fl ickr.com/photos/22691568@N04/43656057080/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK- usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ-s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr- AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS-QkbJ8X-2kQLJ6m-2ni7UXo/
  32. @andy fl eener@hachyderm.io More about the journey than the destination

    Invest in the process to evolve the platform https:// fl ickr.com/photos/193347187@N08/52045942829/in/photolist-2ni84xT-dL6Wum-2iLqcqq-2kXTYuw-7sx6st-2jSEu8L-sTFLqQ-2nfnL5x-Mx37rR-2gD4qCb-PkrUDQ-2kAu5CK-usFoPi-2hDGVmT-28xRKD4-2iehVxy-2icwqUa-28d3ZbC-2gmwYVa-2idL4uJ- s72gC7-2kdeMzz-PFngiZ-2gNwhcX-XKSRWE-aC76sF-2hFeFou-ZMisnR-2kyWFjb-29vJGPj-HAHtHV-2nNdaoU-DL1yYc-2jWbmpr-AVX3o3-2hhRBn9-27mUZUU-2ifWr6w-2k7BA45-23Y6CHz-2eykGTk-LjwXFK-2kJUEcT-2i4BtkS-dqfMuW-zafnSp-Qg8wYS- QkbJ8X-2kQLJ6m-2ni7UXo/