那些年我們做過的 DevOps Pipeline

DevOps 這麼多年了，你當了多少年的水管（Pipeline）工人了呢？是否經歷了從地端到雲端、VM 到 Container、老爺爺到太空章魚⋯⋯等多個不一樣的年代？

在這場分享中，講者將回顧過往，看一看這些年來 DevOps Pipeline 的歷史軌跡，放眼過去、展望未來，如果再來一次，我們該如何開始一條新的 DevOps Pipeline，如何向著水管工人大師之路邁進。

Transcript

1. Photo by Simon Stratford on Unsplash Cheng Wei Chen @ DevOpsDays Taipei 2022 那些年我們做過的 DevOps Pipeline
2. Cheng Wei Chen 陳正瑋 Athemaster 炬識科技 / Technology Consultant 《Effective DevOps 中⽂版》譯者、GitLab Hero、DevOps Taiwan Community 志⼯ https://chengweichen.com
3. 當有人跟你說： 「嘿，我們來導入 
 DevOps 吧！」 你覺得他打算⋯⋯？ Photo by Camylla Battani on Unsplash
4. 1 CI / CD Pipeline 2 講幹話 3 ⾃動化部署 4 ⾃動化測試 5 IaC
5. 1 2 3 4 5
6. QRcode 涉及：⼯具、架構、流程 Photo by Renel Wackett on Unsplash 你調整過多少次 Pipeline？ Jenkins GitLab CI 1 VM 2 VM / Container 3 Stage 5 Stage QRcode 已關閉，現場收集到的數據在下⼀⾴。
8. 2007 - CI (Book) 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄ 我們在這裡
9. 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄ 我們在這裡
10. 2009 - 1st DevOpsDays 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡
11. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 1990 2000 2015 1995 2010 2005 2020 2025
12. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent
13. 2005 - Git 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 2000 - Apache Subversion (SVN)
14. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2011 - GitLab 2008 - GitHub 2005 - Git 1990 2000 2015 1995 2010 2005 2020 2025
15. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub
16. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub 第 2 代 CI / CD Pipeline Pipeline VCS YAML CI Service CI Job
17. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025
18. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container
19. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - Spinnaker 2015 - Buddy 2015 - K8s 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025
20. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 2021 - Pulumi v2.0.0 2015 - K8s
21. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s
22. QRcode Photo by PAUL SMITH on Unsplash 你現在是哪一個世代的 CI/CD Pipeline？ QRcode 已關閉，現場收集到的數據在下⼀⾴。
24. 好的程式與架構是 
 持續演化來的～ Photo by Fancycrave on Unsplash 江湖傳⾔ 你會說 Pipeline嗎？ 你會說 Pipeline 嗎？
25. 好的程式與架構是 
 持續演化來的～ 組織架構、文化、流程、Tech Stack 也是 
 Photo by Fancycrave on Unsplash 江湖傳⾔ DevOps Pipeline 當然也是
26. 想像與思考 PHP DigitalOcean GitLab CI Monolith Members: 3 Laravel 這些條件會產生何種 Pipeline？ GitLab.com NAS run Docker
27. 想像與思考 2 JAVA AWS Jenkins Monolith Members: 10 Spring 這些條件會產生何種 Pipeline？ Bitbucket VM
28. 想像與思考 3 Golang AWS GitLab CI Microservice Dev Team: 30 Go Micro 這些條件會產生何種 Pipeline？ 自架 GitLab GCP Azure Argo CD K8s Platform Team: 6
29. 想像與思考 4 JAVA AWS Jenkins Monolith Members: 10 Spring 如果加上更多的條件呢？ Bitbucket VM 軟硬整合？ 金融業？ 多產品彼此相依？ APP + Web？
30. QRcode Photo by Joshua J. Cotten on Unsplash 有哪些「因素」會影響 CI/CD Pipeline 的規劃？ QRcode 已關閉，現場收集到的數據在下⼀⾴。
32. 程式語言 部署環境 可用 IT 資源 組織架構 工具決策 產品交付方式 資安規範 團隊文化 軟體架構 系統架構 如何 build？ 如何 test？ 產業別 產品類型 團隊人數 產品的品質要求 產品迭代速度 產品交付速度
33. 全都是相依關係 Photo by Fancycrave on Unsplash 有辦法只靠⾃⼰搞定？
34. Photo by Fancycrave on Unsplash 建議？
35. 圖片來源：網路迷因 以下分享的內容 可能不適⽤於你、你的公司或團隊 請⼩⼼服⽤。
36. Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 是⼀趟漫長的旅程 太漫長了～ 長到都⽣青苔了~
37. Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 持續改善、迭代是常態 第三個 Job 
 壞了誒？ 沒事～ 正常迭代嘍～
38. Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 跨職能、跨部⾨的溝通協調無可避免 部署失敗 Infra Ops Dev 炸鍋的 Prod 環境
39. The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌
40. The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌 • ⼈、流程、⼯具、資源限制， 
 權衡多項變因的多重相依關係
41. The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌 • ⼈、流程、⼯具、資源限制， 
 權衡多項變因的多重相依關係 • 找出關鍵瓶頸點， 
 安排超前 2~3 步的規劃
42. Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 持續溝通不能少 有在⽤ Docker 的舉⼿！
43. Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 對⿑團隊對於 Pipeline 的期待 以後 K8s 就歸你管吧！ ……
44. Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 建立共同的 Pipeline 使⽤原則 哪個蠢貨在週五按下 Deploy 的！ 你老闆啊
45. Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 避免落入無效的⾃動化 沒有測試，就全⾃動 部署，錯了嗎？
46. First success, Small Wins Photo by Fancycrave on Unsplash • 搶下灘頭堡 • 借⼒使⼒ • 越過死亡⼭⾕ 死亡山谷 灘頭堡 Small Win Small Win
47. Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 • 保存 • 交流 • 傳承
48. Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 • 保存 • 交流 • 傳承
49. Photo by Fancycrave on Unsplash 更實務或技術⼀點的建議？
50. 圖片來源：網路迷因 以下分享的內容 可能不適⽤於你、你的公司或團隊 請⼩⼼服⽤。
51. 釐清是哪個世代的 Pipeline Photo by Fancycrave on Unsplash • 1 • 2 • 2.5 • 3
52. 建立管理相依關係的原則 Photo by Fancycrave on Unsplash • Dependency • Package • Release • Artifact • Container • Secret • Infrastructure • Environment management
53. ⼩處著⼿，持續迭代 Photo by Fancycrave on Unsplash • 不要妄想⼀條龍，CI、CD 是可以拆開的 • Everything as Code • 良好的軟⼯技巧、架構設計 • 模板、模組、繼承 • 測試再測試
54. 好 多 書不讀嗎？ Docker 或 Container Books More…
55. Photo by Fancycrave on Unsplash 總結 & 回顧
56. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent
57. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub 第 2 代 CI / CD Pipeline Pipeline VCS YAML CI Service CI Job
58. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container
59. 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s
60. 程式語言 部署環境 可用 IT 資源 組織架構 工具決策 產品交付方式 資安規範 團隊文化 軟體架構 系統架構 如何 build？ 如何 test？ 產業別 產品類型 團隊人數 產品的品質要求 產品迭代速度 產品交付速度 全都是相依關係 有辦法只靠⾃⼰搞定？
61. 實踐 GitOps 的難題？ GitOps 解決了什麼新問題？ Photo by Camylla Battani on Unsplash 兩個延伸題目，留給大家回去思考看看
62. • Mindset • The Question Behind the Question • First success, Small Wins • Communicate, collaborate, align goals • Knowledge management Photo by Fancycrave on Unsplash 個⼈⼼得
63. • 釐清是哪個世代的 Pipeline • 建立管理相依關係的原則 • ⼩處著⼿，持續迭代 Photo by Fancycrave on Unsplash 個⼈⼼得
64. 好 多 書不讀嗎？ Docker 或 Container Books More…
65. Photo by Clark Tibbs on Unsplash Thank You