Upgrade to Pro — share decks privately, control downloads, hide ads and more …

那些年我們做過的 DevOps Pipeline

Cheng-Wei Chen
September 17, 2022

那些年我們做過的 DevOps Pipeline

DevOps 這麼多年了,你當了多少年的水管(Pipeline)工人了呢?是否經歷了從地端到雲端、VM 到 Container、老爺爺到太空章魚⋯⋯等多個不一樣的年代?

在這場分享中,講者將回顧過往,看一看這些年來 DevOps Pipeline 的歷史軌跡,放眼過去、展望未來,如果再來一次,我們該如何開始一條新的 DevOps Pipeline,如何向著水管工人大師之路邁進。

Cheng-Wei Chen

September 17, 2022
Tweet

More Decks by Cheng-Wei Chen

Other Decks in Technology

Transcript

  1. Photo by Simon Stratford on Unsplash Cheng Wei Chen @

    DevOpsDays Taipei 2022 那些年我們做過的 DevOps Pipeline
  2. Cheng Wei Chen 陳正瑋 Athemaster 炬識科技 / Technology Consultant 《Effective

    DevOps 中⽂版》譯者、GitLab Hero、DevOps Taiwan Community 志⼯ https://chengweichen.com
  3. 當有人跟你說: 「嘿,我們來導入 
 DevOps 吧!」 你覺得他打算⋯⋯? Photo by Camylla Battani

    on Unsplash
  4. 1 CI / CD Pipeline 2 講幹話 3 ⾃動化部署 4

    ⾃動化測試 5 IaC
  5. 1 2 3 4 5

  6. QRcode 涉及:⼯具、架構、流程 Photo by Renel Wackett on Unsplash 你調整過多少次 Pipeline?

    Jenkins GitLab CI 1 VM 2 VM / Container 3 Stage 5 Stage QRcode 已關閉,現場收集到的數據在下⼀⾴。
  7. None
  8. 2007 - CI (Book) 2010 - CD (Book) 1990 2000

    2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄ 我們在這裡
  9. 1994 - Origin of Continuous integration by Grady Booch 2007

    - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄ 我們在這裡
  10. 2009 - 1st DevOpsDays 1999 – Continuous Integration 1999 –

    eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡
  11. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 1990 2000 2015 1995 2010 2005 2020 2025
  12. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent
  13. 2005 - Git 1999 – VMware 2007 - Bamboo 2007

    - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 2000 - Apache Subversion (SVN)
  14. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2011 - GitLab 2008 - GitHub 2005 - Git 1990 2000 2015 1995 2010 2005 2020 2025
  15. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub
  16. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub 第 2 代 CI / CD Pipeline Pipeline VCS YAML CI Service CI Job
  17. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025
  18. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container
  19. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - Spinnaker 2015 - Buddy 2015 - K8s 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025
  20. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 2021 - Pulumi v2.0.0 2015 - K8s
  21. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s
  22. QRcode Photo by PAUL SMITH on Unsplash 你現在是哪一個世代的 CI/CD Pipeline?

    QRcode 已關閉,現場收集到的數據在下⼀⾴。
  23. None
  24. 好的程式與架構是 
 持續演化來的~ Photo by Fancycrave on Unsplash 江湖傳⾔ 你會說

    Pipeline嗎? 你會說 Pipeline 嗎?
  25. 好的程式與架構是 
 持續演化來的~ 組織架構、文化、流程、Tech Stack 也是 
 Photo by Fancycrave

    on Unsplash 江湖傳⾔ DevOps Pipeline 當然也是
  26. 想像與思考 PHP DigitalOcean GitLab CI Monolith Members: 3 Laravel 這些條件會產生何種

    Pipeline? GitLab.com NAS run Docker
  27. 想像與思考 2 JAVA AWS Jenkins Monolith Members: 10 Spring 這些條件會產生何種

    Pipeline? Bitbucket VM
  28. 想像與思考 3 Golang AWS GitLab CI Microservice Dev Team: 30

    Go Micro 這些條件會產生何種 Pipeline? 自架 GitLab GCP Azure Argo CD K8s Platform Team: 6
  29. 想像與思考 4 JAVA AWS Jenkins Monolith Members: 10 Spring 如果加上更多的條件呢?

    Bitbucket VM 軟硬整合? 金融業? 多產品彼此相依? APP + Web?
  30. QRcode Photo by Joshua J. Cotten on Unsplash 有哪些「因素」會影響 CI/CD

    Pipeline 的規劃? QRcode 已關閉,現場收集到的數據在下⼀⾴。
  31. None
  32. 程式語言 部署環境 可用 IT 資源 組織架構 工具決策 產品交付方式 資安規範 團隊文化

    軟體架構 系統架構 如何 build? 如何 test? 產業別 產品類型 團隊人數 產品的品質要求 產品迭代速度 產品交付速度
  33. 全都是相依關係 Photo by Fancycrave on Unsplash 有辦法只靠⾃⼰搞定?

  34. Photo by Fancycrave on Unsplash 建議?

  35. 圖片來源:網路迷因 以下分享的內容 可能不適⽤於你、你的公司或團隊 請⼩⼼服⽤。

  36. Mindset 圖片來源:https://www.ghibli.jp/works/laputa/ • 是⼀趟漫長的旅程 太漫長了~ 長到都⽣青苔了~

  37. Mindset 圖片來源:https://www.ghibli.jp/works/laputa/ • 持續改善、迭代是常態 第三個 Job 
 壞了誒? 沒事~ 正常迭代嘍~

  38. Mindset 圖片來源:https://www.ghibli.jp/works/laputa/ • 跨職能、跨部⾨的溝通協調無可避免 部署失敗 Infra Ops Dev 炸鍋的 Prod

    環境
  39. The Question Behind the Question Photo by Fancycrave on Unsplash

    • 退後⼀步,了解全貌
  40. The Question Behind the Question Photo by Fancycrave on Unsplash

    • 退後⼀步,了解全貌 • ⼈、流程、⼯具、資源限制, 
 權衡多項變因的多重相依關係
  41. The Question Behind the Question Photo by Fancycrave on Unsplash

    • 退後⼀步,了解全貌 • ⼈、流程、⼯具、資源限制, 
 權衡多項變因的多重相依關係 • 找出關鍵瓶頸點, 
 安排超前 2~3 步的規劃
  42. Communicate, collaborate, align goals 圖片來源:https://www.ghibli.jp/works/kazetachinu/ • 持續溝通不能少 有在⽤ Docker 的舉⼿!

  43. Communicate, collaborate, align goals 圖片來源:https://www.ghibli.jp/works/kazetachinu/ • 對⿑團隊對於 Pipeline 的期待 以後

    K8s 就歸你管吧! ……
  44. Communicate, collaborate, align goals 圖片來源:https://www.ghibli.jp/works/kazetachinu/ • 建立共同的 Pipeline 使⽤原則 哪個蠢貨在週五按下

    Deploy 的! 你老闆啊
  45. Communicate, collaborate, align goals 圖片來源:https://www.ghibli.jp/works/kazetachinu/ • 避免落入無效的⾃動化 沒有測試,就全⾃動 部署,錯了嗎?

  46. First success, Small Wins Photo by Fancycrave on Unsplash •

    搶下灘頭堡 • 借⼒使⼒ • 越過死亡⼭⾕ 死亡山谷 灘頭堡 Small Win Small Win
  47. Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 •

    保存 • 交流 • 傳承
  48. Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 •

    保存 • 交流 • 傳承
  49. Photo by Fancycrave on Unsplash 更實務或技術⼀點的建議?

  50. 圖片來源:網路迷因 以下分享的內容 可能不適⽤於你、你的公司或團隊 請⼩⼼服⽤。

  51. 釐清是哪個世代的 Pipeline Photo by Fancycrave on Unsplash • 1 •

    2 • 2.5 • 3
  52. 建立管理相依關係的原則 Photo by Fancycrave on Unsplash • Dependency • Package

    • Release • Artifact • Container • Secret • Infrastructure • Environment management
  53. ⼩處著⼿,持續迭代 Photo by Fancycrave on Unsplash • 不要妄想⼀條龍,CI、CD 是可以拆開的 •

    Everything as Code • 良好的軟⼯技巧、架構設計 • 模板、模組、繼承 • 測試再測試
  54. 好 多 書不讀嗎? Docker 或 Container Books More…

  55. Photo by Fancycrave on Unsplash 總結 & 回顧

  56. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent
  57. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub 第 2 代 CI / CD Pipeline Pipeline VCS YAML CI Service CI Job
  58. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container
  59. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999

    – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s
  60. 程式語言 部署環境 可用 IT 資源 組織架構 工具決策 產品交付方式 資安規範 團隊文化

    軟體架構 系統架構 如何 build? 如何 test? 產業別 產品類型 團隊人數 產品的品質要求 產品迭代速度 產品交付速度 全都是相依關係 有辦法只靠⾃⼰搞定?
  61. 實踐 GitOps 的難題? GitOps 解決了什麼新問題? Photo by Camylla Battani on

    Unsplash 兩個延伸題目,留給大家回去思考看看
  62. • Mindset • The Question Behind the Question • First

    success, Small Wins • Communicate, collaborate, align goals • Knowledge management Photo by Fancycrave on Unsplash 個⼈⼼得
  63. • 釐清是哪個世代的 Pipeline • 建立管理相依關係的原則 • ⼩處著⼿,持續迭代 Photo by Fancycrave

    on Unsplash 個⼈⼼得
  64. 好 多 書不讀嗎? Docker 或 Container Books More…

  65. Photo by Clark Tibbs on Unsplash Thank You