那些年我們做過的 DevOps Pipeline

Slide 1

Slide 1 text

Photo by Simon Stratford on Unsplash Cheng Wei Chen @ DevOpsDays Taipei 2022 那些年我們做過的 DevOps Pipeline

Slide 2

Slide 2 text

Cheng Wei Chen 陳正瑋 Athemaster 炬識科技 / Technology Consultant 《Effective DevOps 中⽂版》譯者、GitLab Hero、DevOps Taiwan Community 志⼯ https://chengweichen.com

Slide 3

Slide 3 text

當有人跟你說：「嘿，我們來導入   DevOps 吧！」你覺得他打算⋯⋯？ Photo by Camylla Battani on Unsplash

Slide 4

Slide 4 text

1 CI / CD Pipeline 2 講幹話 3 ⾃動化部署 4 ⾃動化測試 5 IaC

Slide 5

Slide 5 text

1 2 3 4 5

Slide 6

Slide 6 text

QRcode 涉及：⼯具、架構、流程 Photo by Renel Wackett on Unsplash 你調整過多少次 Pipeline？ Jenkins GitLab CI 1 VM 2 VM / Container 3 Stage 5 Stage QRcode 已關閉，現場收集到的數據在下⼀⾴。

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

2007 - CI (Book) 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄我們在這裡

Slide 9

Slide 9 text

1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 CI/CD Pipeline 是個老東⻄我們在這裡

Slide 10

Slide 10 text

2009 - 1st DevOpsDays 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡

Slide 11

Slide 11 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 1990 2000 2015 1995 2010 2005 2020 2025

Slide 12

Slide 12 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent

Slide 13

Slide 13 text

2005 - Git 1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 2000 - Apache Subversion (SVN)

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2008 - Bitbucket 2016 - Gitea 2005 - Git 2011 - Travis CI 2011 - CircleCI 2012 - GitLab CI 2018 - GitHub Action 1990 2000 2015 1995 2010 2005 2020 2025 2011 - GitLab 2008 - GitHub 第 2 代 CI / CD Pipeline Pipeline VCS YAML CI Service CI Job

Slide 17

Slide 17 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025

Slide 18

Slide 18 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container

Slide 19

Slide 19 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - Spinnaker 2015 - Buddy 2015 - K8s 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025

Slide 20

Slide 20 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 2021 - Pulumi v2.0.0 2015 - K8s

Slide 21

Slide 21 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s

Slide 22

Slide 22 text

QRcode Photo by PAUL SMITH on Unsplash 你現在是哪一個世代的 CI/CD Pipeline？ QRcode 已關閉，現場收集到的數據在下⼀⾴。

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

好的程式與架構是   持續演化來的～ Photo by Fancycrave on Unsplash 江湖傳⾔你會說 Pipeline嗎？你會說 Pipeline 嗎？

Slide 25

Slide 25 text

好的程式與架構是   持續演化來的～組織架構、文化、流程、Tech Stack 也是   Photo by Fancycrave on Unsplash 江湖傳⾔ DevOps Pipeline 當然也是

Slide 26

Slide 26 text

想像與思考 PHP DigitalOcean GitLab CI Monolith Members: 3 Laravel 這些條件會產生何種 Pipeline？ GitLab.com NAS run Docker

Slide 27

Slide 27 text

想像與思考 2 JAVA AWS Jenkins Monolith Members: 10 Spring 這些條件會產生何種 Pipeline？ Bitbucket VM

Slide 28

Slide 28 text

想像與思考 3 Golang AWS GitLab CI Microservice Dev Team: 30 Go Micro 這些條件會產生何種 Pipeline？自架 GitLab GCP Azure Argo CD K8s Platform Team: 6

Slide 29

Slide 29 text

想像與思考 4 JAVA AWS Jenkins Monolith Members: 10 Spring 如果加上更多的條件呢？ Bitbucket VM 軟硬整合？金融業？多產品彼此相依？ APP + Web？

Slide 30

Slide 30 text

QRcode Photo by Joshua J. Cotten on Unsplash 有哪些「因素」會影響 CI/CD Pipeline 的規劃？ QRcode 已關閉，現場收集到的數據在下⼀⾴。

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

程式語言部署環境可用 IT 資源組織架構工具決策產品交付方式資安規範團隊文化軟體架構系統架構如何 build？如何 test？產業別產品類型團隊人數產品的品質要求產品迭代速度產品交付速度

Slide 33

Slide 33 text

全都是相依關係 Photo by Fancycrave on Unsplash 有辦法只靠⾃⼰搞定？

Slide 34

Slide 34 text

Photo by Fancycrave on Unsplash 建議？

Slide 35

Slide 35 text

圖片來源：網路迷因以下分享的內容可能不適⽤於你、你的公司或團隊請⼩⼼服⽤。

Slide 36

Slide 36 text

Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 是⼀趟漫長的旅程太漫長了～長到都⽣青苔了~

Slide 37

Slide 37 text

Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 持續改善、迭代是常態第三個 Job   壞了誒？沒事～正常迭代嘍～

Slide 38

Slide 38 text

Mindset 圖片來源：https://www.ghibli.jp/works/laputa/ • 跨職能、跨部⾨的溝通協調無可避免部署失敗 Infra Ops Dev 炸鍋的 Prod 環境

Slide 39

Slide 39 text

The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌

Slide 40

Slide 40 text

The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌 • ⼈、流程、⼯具、資源限制，   權衡多項變因的多重相依關係

Slide 41

Slide 41 text

The Question Behind the Question Photo by Fancycrave on Unsplash • 退後⼀步，了解全貌 • ⼈、流程、⼯具、資源限制，   權衡多項變因的多重相依關係 • 找出關鍵瓶頸點，   安排超前 2~3 步的規劃

Slide 42

Slide 42 text

Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 持續溝通不能少有在⽤ Docker 的舉⼿！

Slide 43

Slide 43 text

Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 對⿑團隊對於 Pipeline 的期待以後 K8s 就歸你管吧！ ……

Slide 44

Slide 44 text

Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 建立共同的 Pipeline 使⽤原則哪個蠢貨在週五按下 Deploy 的！你老闆啊

Slide 45

Slide 45 text

Communicate, collaborate, align goals 圖片來源：https://www.ghibli.jp/works/kazetachinu/ • 避免落入無效的⾃動化沒有測試，就全⾃動部署，錯了嗎？

Slide 46

Slide 46 text

First success, Small Wins Photo by Fancycrave on Unsplash • 搶下灘頭堡 • 借⼒使⼒ • 越過死亡⼭⾕死亡山谷灘頭堡 Small Win Small Win

Slide 47

Slide 47 text

Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 • 保存 • 交流 • 傳承

Slide 48

Slide 48 text

Knowledge management Photo by Fancycrave on Unsplash • 關鍵知識、決策、技能的 • 保存 • 交流 • 傳承

Slide 49

Slide 49 text

Photo by Fancycrave on Unsplash 更實務或技術⼀點的建議？

Slide 50

Slide 50 text

圖片來源：網路迷因以下分享的內容可能不適⽤於你、你的公司或團隊請⼩⼼服⽤。

Slide 51

Slide 51 text

釐清是哪個世代的 Pipeline Photo by Fancycrave on Unsplash • 1 • 2 • 2.5 • 3

Slide 52

Slide 52 text

建立管理相依關係的原則 Photo by Fancycrave on Unsplash • Dependency • Package • Release • Artifact • Container • Secret • Infrastructure • Environment management

Slide 53

Slide 53 text

⼩處著⼿，持續迭代 Photo by Fancycrave on Unsplash • 不要妄想⼀條龍，CI、CD 是可以拆開的 • Everything as Code • 良好的軟⼯技巧、架構設計 • 模板、模組、繼承 • 測試再測試

Slide 54

Slide 54 text

好多書不讀嗎？ Docker 或 Container Books More…

Slide 55

Slide 55 text

Photo by Fancycrave on Unsplash 總結 & 回顧

Slide 56

Slide 56 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 1990 2000 2015 1995 2010 2005 2020 2025 我們在這裡 2000 - Apache Subversion (SVN) 2005 - Git 第 1 代 CI / CD Pipeline CI Server GUI VM SSH VM Agent

Slide 57

Slide 57 text

Slide 58

Slide 58 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2015 - K8s 1990 2000 2015 1995 2010 2005 2020 2025 第 2.5 代 CI / CD Pipeline VM VM Container

Slide 59

Slide 59 text

1999 – VMware 2007 - Bamboo 2007 - Cruise 1999 – Continuous Integration 1999 – eXtreme Programming 1994 - Origin of Continuous integration by Grady Booch 2007 - CI (Book) 2010 - CD (Book) 2011 - Jenkins 2006 - TeamCity 2000 - Apache Subversion (SVN) 2010 - GoCD 2005 - Huden 2009 - 1st DevOpsDays 2007 – VirtualBox 2016 - Gitea 2012 - GitLab CI 2008 - Bitbucket 2005 - Git 2011 - Travis CI 2011 - CircleCI 2018 - GitHub Action 2011 - GitLab 2008 - GitHub 資料來源: https://en.wikipedia.org/wiki/Continuous_integration 2014 - Docker (v1.0) 2016 - IaC (Book, 1st) 2020 - IaC (Book, 2nd) 2015 - Spinnaker 2015 - Buddy 2019 - Argo CD 2018 - Jenkins X 1990 2000 2015 1995 2010 2005 2020 2025 2017 - GitOps 2021 - Terraform v1.0.0 第 3 代 CI / CD Pipeline Container VCS YAML Container Automation 2021 - Pulumi v2.0.0 2015 - K8s

Slide 60

Slide 60 text

程式語言部署環境可用 IT 資源組織架構工具決策產品交付方式資安規範團隊文化軟體架構系統架構如何 build？如何 test？產業別產品類型團隊人數產品的品質要求產品迭代速度產品交付速度全都是相依關係有辦法只靠⾃⼰搞定？

Slide 61

Slide 61 text

實踐 GitOps 的難題？ GitOps 解決了什麼新問題？ Photo by Camylla Battani on Unsplash 兩個延伸題目，留給大家回去思考看看

Slide 62

Slide 62 text

• Mindset • The Question Behind the Question • First success, Small Wins • Communicate, collaborate, align goals • Knowledge management Photo by Fancycrave on Unsplash 個⼈⼼得

Slide 63

Slide 63 text

• 釐清是哪個世代的 Pipeline • 建立管理相依關係的原則 • ⼩處著⼿，持續迭代 Photo by Fancycrave on Unsplash 個⼈⼼得