$30 off During Our Annual Pro Sale. View Details »

Testing in Production, Deploy on Fridays

Yi-Feng Tzeng
November 14, 2020

Testing in Production, Deploy on Fridays

本議題是去年 ModernWeb'19 「Progressive Deployment & NoDeploy」的延伸。雖然已提倡 Testing in Production 多年,但至今願意或敢於實踐的團隊並不多,背後原因多是與文化及態度有些關係。

此次主要分享推廣過程中遇到的苦與甜,以及自己親力操刀幾項達成 Testing in Production, Deploy on Fridays 成就的產品。

Yi-Feng Tzeng

November 14, 2020
Tweet

More Decks by Yi-Feng Tzeng

Other Decks in Technology

Transcript

  1. 技術在我們手上 世界就在我們手上
    Testing in Production,
    Deploy on Fridays
    曾義峰 (Ant)
    [email protected]

    View Slide

  2. 曾義峰 (aka Ant)

    LeadBest Consulting Group 首席執行顧問

    TGO 台北分會學習委員

    臺灣資安社群 CHROOT 成員

    曾任資安顧問及電子票證公司顧問

    開源人年會 (COSCUP) 2009 及 2012 講師

    臺灣駭客年會 (HITCON) 2008 及 2009 講師

    臺灣 Modern Web 2015/2016/2017/2018/2019 講師

    View Slide

  3. 3/111
    Deploy on ... Fridays
    技術在我們手上 世界就在我們手上

    View Slide

  4. 4/111
    Deploy on ... Fridays
    No
    NoNoNO
    Don't
    Do
    tha
    t
    Image Credit : https://kknews.cc/pet/pqbxb8.html
    Font Credit : https://www.dafont.com/theme.php?cat=109 (Defused)
    技術在我們手上 世界就在我們手上

    View Slide

  5. 5/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.getdigital.eu/deploy-on-friday.html
    我應該在星期五下午 5 點部署嗎?
    我只是想要 ( 做一點小變更 )...
    不行
    不行就是不行 ~

    View Slide

  6. 6/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://medium.com/openclassrooms-product-design-and-engineering/do-not-deploy-on-friday-92b1b46ebfe6

    View Slide

  7. 7/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://medium.com/openclassrooms-product-design-and-engineering/do-not-deploy-on-friday-92b1b46ebfe6
    他們說在星期五進行部署一切都會沒事的 ...

    View Slide

  8. 8/111
    技術在我們手上 世界就在我們手上
    Credit : https://twitter.com/kvlly/status/1116672656781266944

    View Slide

  9. 9/111
    技術在我們手上 世界就在我們手上
    Credit : https://twitter.com/kvlly/status/1116672656781266944
    I am not to blame, it is about (team) culture

    View Slide

  10. 10/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday
    Team Discipline

    View Slide

  11. 11/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday

    View Slide

  12. 12/111
    Why
    技術在我們手上 世界就在我們手上

    View Slide

  13. 13/111
    技術在我們手上 世界就在我們手上
    Confidence vs. Risk
    &Trust
    Why

    View Slide

  14. 14/111
    技術在我們手上 世界就在我們手上
    Confidence vs. Risk
    &Trust
    Why
    如果團隊害怕在星期五部署,是不是意謂著星期一、二、三、四的部署也同樣令人不安
    如果你提供的是 24/7 全天候服務,那麼星期五與其他時間應該沒有什麼不同!

    View Slide

  15. 15/111
    技術在我們手上 世界就在我們手上
    Confidence vs. Risk
    &Trust
    Why
    如果團隊害怕在星期五部署,是不是意謂著星期一、二、三、四的部署也同樣令人不安
    如果你提供的是 24/7 全天候服務,那麼星期五與其他時間應該沒有什麼不同!
    團隊對部署有沒有信心,這才是重要的問題

    View Slide

  16. Progressive Deployment & NoDeploy
    曾義峰 (Ant)
    [email protected]
    2019-08-28
    Credit : https://www.slideshare.net/yftzeng/progressive-deployment-nodeploy

    View Slide

  17. 17/111
    技術在我們手上 世界就在我們手上
    Credit : https://www.slideshare.net/yftzeng/progressive-deployment-nodeploy
    13,692 views

    View Slide

  18. 18/111
    ~Dark 1980s 2008 2014
    ITIL
    Design for Anti-Fragility
    Design for Robustness Design for Resilience
    DevOps
    堅實性設計 抗脆性設計 復原性設計
    精心設計。
    嚴格管控。
    故障被設計為需要異常
    處理。
    將開發及維運為一體。
    視故障為正常狀態。
    為故障發生設計隔離。
    將維運視為工程。
    最小化人為錯誤。
    為故障復原至正常狀態。
    轉型
    轉型
    進化
    SRE Dev(ops)
    Design for Deployless
    Partial Credit : https://noops.connpass.com
    NoDeploy
    2019
    想像
    想像
    進化
    無部署設計

    View Slide

  19. 19/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    Commit Dev Test Staging Test Deploy Prod
    Continuous Deployment
    Local Dev Staging Prod
    Environments
    Chaos
    Test

    View Slide

  20. 20/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    Image Credit : https://software.af.mil/training/devops/
    技術在我們手上 世界就在我們手上

    View Slide

  21. 21/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    Deploy on Fridays
    Image Credit : https://software.af.mil/training/devops/
    技術在我們手上 世界就在我們手上

    View Slide

  22. 22/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    Image Credit : https://software.af.mil/training/devops/
    技術在我們手上 世界就在我們手上

    View Slide

  23. 23/111
    Image Credit : https://www.freepik.com/free-icon/silo_869863.htm
    Sales
    Marketing
    Dev Ops Compliance
    (Security)
    Value
    DevOps
    Cost Cost
    技術在我們手上 世界就在我們手上

    View Slide

  24. 24/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday

    View Slide

  25. 25/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday

    View Slide

  26. 26/111
    技術在我們手上 世界就在我們手上
    Weekends
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday

    View Slide

  27. 27/111
    技術在我們手上 世界就在我們手上
    Weekends
    20% of the week
    Image Credit : https://thecodinglove.com/wishing-you-all-a-nice-friday

    View Slide

  28. 28/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays

    View Slide

  29. 29/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays

    View Slide

  30. 30/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆

    View Slide

  31. 31/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆
    Trunk
    Trunk
    PUSH
    ⬇️
    No-Merge-Fridays
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    PUSH
    ⬇️
    time

    View Slide

  32. 32/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆
    Image Credit : https://www.git-tower.com/help/guides/branches-and-tags/merge-conflicts/windows

    View Slide

  33. 33/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆ Lead Time ⬆

    View Slide

  34. 34/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆ Lead Time ⬆
    Image Credit : https://www.qualysoft.com/en/blog/tech-blog-introduction-part-2-cicd-patterns-and-best-practices

    View Slide

  35. 35/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆ Lead Time ⬆
    Image Credit : https://www.qualysoft.com/en/blog/tech-blog-introduction-part-2-cicd-patterns-and-best-practices

    View Slide

  36. 36/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆ Lead Time ⬆ Problems Debug
    ⬆ ⬇

    View Slide

  37. 37/111
    技術在我們手上 世界就在我們手上
    No-Deploy-Fridays
    No-Merge-Fridays
    Conflicts ⬆ Lead Time ⬆ Problems Debug
    ⬆ ⬇
    Small
    Commits
    Release
    Big
    Commits
    Release
    Vs.

    View Slide

  38. 38/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://doc.oroinc.com/cloud/environments/
    Image Credit : https://oroinc.com/b2b-ecommerce/blog/testing-and-staging-environments-in-ecommerce-implementation/

    View Slide

  39. 39/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://doc.oroinc.com/cloud/environments/
    Image Credit : https://oroinc.com/b2b-ecommerce/blog/testing-and-staging-environments-in-ecommerce-implementation/

    View Slide

  40. 40/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.slideshare.net/cPrime/kanban-testing-54239643
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.slideshare.net/LaunchDarkly/the-staging-server-is-dead-long-live-the-staging-server

    View Slide

  41. 41/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.slideshare.net/cPrime/kanban-testing-54239643
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.slideshare.net/LaunchDarkly/the-staging-server-is-dead-long-live-the-staging-server

    View Slide

  42. 42/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.slideshare.net/cPrime/kanban-testing-54239643
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.slideshare.net/LaunchDarkly/the-staging-server-is-dead-long-live-the-staging-server

    View Slide

  43. 43/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.slideshare.net/cPrime/kanban-testing-54239643
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.slideshare.net/LaunchDarkly/the-staging-server-is-dead-long-live-the-staging-server
    這些網站都告訴我們應該這麼做,好像世界就應該要長成這個樣子

    View Slide

  44. 44/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.slideshare.net/cPrime/kanban-testing-54239643
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.slideshare.net/LaunchDarkly/the-staging-server-is-dead-long-live-the-staging-server
    這些網站都告訴我們應該這麼做,好像世界就應該要長成這個樣子
    缺點是什麼?

    View Slide

  45. 45/111
    Staging
    Staging 環境必須與 Production 環境幾乎
    相同,這十分困難,否則測試就不準了。
    Trying to mirror your staging
    environment to production is a fool’s
    errand. Just give up.
    嘗試將 Staging 視為 Production 是一個愚蠢的事。放棄吧。
    Partial Credit : https://thenewstack.io/honeycombs-charity-majors-go-ahead-test-in-production/

    View Slide

  46. 46/111
    Credit : https://www.infoq.cn/article/b9lG4O8oJ8bWcFFQfFKE
    Staging

    View Slide

  47. 47/111
    Image Credit : Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf
    Microsoft/Azure

    View Slide

  48. 48/111
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.reddit.com/r/ProgrammerHumor/comments/ing3jx/works_on_my_machine/
    Image Credit : https://www.pngitem.com/middle/hwwmRmi_dove-clipart-flame-clipart-flame-png-transparent-png/
    技術在我們手上 世界就在我們手上
    在我電腦環境就沒問題

    View Slide

  49. 49/111
    大膽的想法

    View Slide

  50. 50/111
    大膽的想法
    純真的想法

    View Slide

  51. 51/111
    Testing in Production
    在 Microservices 環境中,尤其是 FaaS 環境中,
    Testing in production 是一種實際的需要。

    View Slide

  52. 52/111
    Chaos Engineering @ Netflix
    這些是需要整合測試的故障模式示例,因為它們涉及服務之間的互動。在某
    些情況下,可能只能在 Production 進行完全整合測試。在 Netflix ,根本不
    可能完全重現整個架構並運行端到端測試。
    當可以在 Testing 環境中重現整個系統時,我們仍然相信需要在 Production
    運行實驗,因為在 Testing 環境中永遠不可能完全重現系統的所有方面。總
    是存在差異,例如合併客戶端與真實客戶端相比的行為方式或 DNS 配置問
    題。
    Credit : Chaos Engineering IEEE Software (2016-05-05).pdf
    Testing in Production

    View Slide

  53. 53/111
    技術在我們手上 世界就在我們手上

    If you aren't testing in prod
    you aren't testing in reality

    如果不在正式環境測試,
    你的測試就不會是真實的
    Credit : https://twitter.com/mipsytipsy/status/1147863838647185408
    Image Credit : https://store.steampowered.com/app/561970/REALITY/

    View Slide

  54. 54/111
    技術在我們手上 世界就在我們手上

    Dev will match Prod,
    if you develop on production

    如果在正式環境上進行開發,
    開發環境就會與正式環境相稱
    Credit : Blue is the new green (2019-03-26).pdf
    Image Credit : https://tw.voicetube.com/videos/62711

    View Slide

  55. 55/111
    Your aim won’t be perfect,
    control over the blast radius.
    你的瞄準並不完美,
    請控制好爆炸半徑。
    Image Credit : https://www.behance.net/gallery/57558081/FREE-BUSINESS-ICONS
    Progressive Deployment

    View Slide

  56. 56/111
    Credit : https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/
    Image Credit : https://www.trisignup.com/Race/24568/Sponsor/94608
    Progressive deployment is continuous deploy
    with fine-grained control over the blast radius.
    漸進式部署是對爆破半徑進行細部控制的連續部署。
    Progressive Deployment

    View Slide

  57. 57/111
    Credit : https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/
    Image Credit : https://support.google.com/optimize/answer/7012154?hl=en
    Deploying a service is not that same as activating it for all users.
    部署一個服務,未必需要讓所有使用者皆同時生效。
    Progressive Deployment

    View Slide

  58. 58/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    技術在我們手上 世界就在我們手上

    View Slide

  59. 59/111
    Code Test Acceptance Deploy Operate
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    DevOps / SRE
    技術在我們手上 世界就在我們手上

    View Slide

  60. 60/111
    Code Test Acceptance Deploy Release
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    Continuous Deployment + Feature Flag / 持續部署 + 功能標誌
    技術在我們手上 世界就在我們手上
    Operate
    DevOps / SRE

    View Slide

  61. 61/111
    Code Test Acceptance Deploy Release
    Continuous Integration / 持續整合
    Continuous Delivery / 持續交付
    Continuous Deployment / 持續部署
    Continuous Deployment + Feature Flag / 持續部署 + 功能標誌
    技術在我們手上 世界就在我們手上
    Operate
    DevOps / SRE
    Code Push ➡
    Deployment
    Developer
    Production
    Server User
    Feature Flaged ➡
    Release

    View Slide

  62. 62/111
    Feature Flag
    01 所有程式皆為部署,並可控制開關,減少整合問題
    All code is deployed, but control exposure, reduces integration issue.
    02 控制細緻度可至特定層級或是開發分支
    Flags provide runtime control down to individual user.
    03 開發者可以增加或刪除功能,無需再次部署
    Users can be added or removed with no redeployment
    04 暗啟動
    Enables dark launch
    Credit : Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf
    Image Credit : https://opensource.com/article/18/2/feature-flags-ring-deployment-model

    View Slide

  63. Cost in Time
    時間成本
    hours
    mins ~ hour
    secs
    ~
    mins
    Applicability
    適用性
    2%
    8%
    90%
    Toggle
    Feature Flag
    ReDeploy
    Previous Version
    Rollback
    Revert Commit
    Incident Recovery
    事件恢復

    View Slide

  64. 64/111
    Why

    How
    技術在我們手上 世界就在我們手上

    View Slide

  65. 65/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  66. 66/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  67. 67/111
    技術在我們手上 世界就在我們手上
    Group = Beta
    Group = Normal
    If ( group = “beta” ) {
    ...
    }
    else {
    ...
    }
    Beta
    Normal
    By User Group
    Time
    URI
    IP Address
    User Group

    View Slide

  68. 68/111
    技術在我們手上 世界就在我們手上
    IP = o.o.o.o
    IP = x.x.x.x
    If ( IP = “o.o.o.o” ) {
    ...
    }
    else {
    ...
    }
    Beta
    By IP Address
    Time
    URI
    IP Address
    User Group
    Normal

    View Slide

  69. 69/111
    技術在我們手上 世界就在我們手上
    URI ~ ?flag=1
    If ( URI ~ “flag=1” ) {
    ...
    }
    else {
    ...
    }
    Beta
    Normal
    By URI
    Time
    URI
    IP Address
    User Group

    View Slide

  70. 70/111
    技術在我們手上 世界就在我們手上
    URI ~ 20201005
    If ( URI ~“20201005”) {
    ...
    }
    else {
    ...
    }
    Beta
    Normal
    By Time
    Time
    URI
    IP Address
    User Group

    View Slide

  71. 71/111
    技術在我們手上 世界就在我們手上
    Story Mapping
    Code Test Acceptance Deploy Release Done
    #19
    Fix
    Login Bug
    #20
    Fix
    Email Bug
    #21
    Perf.
    Tuning
    #17
    Sort By
    Date
    #18
    Filter By
    Group
    #22
    German
    i18n

    View Slide

  72. 72/111
    技術在我們手上 世界就在我們手上
    Story Mapping
    Code Test Acceptance Deploy Release Done
    #19
    Fix
    Login Bug
    #20
    Fix
    Email Bug
    #21
    Perf.
    Tuning
    #17
    Sort By
    Date
    Flag#3
    #18
    Filter By
    Group
    Flag#4
    #22
    German
    i18n
    Flag#5

    View Slide

  73. 73/111
    技術在我們手上 世界就在我們手上
    Story Mapping
    Code Test Acceptance Deploy Release Done
    #19
    Fix
    Login Bug
    #20
    Fix
    Email Bug
    #21
    Perf.
    Tuning
    #17
    Sort By
    Date
    Flag#3
    #18
    Filter By
    Group
    Flag#4
    #22
    German
    i18n
    Flag#5
    #17
    Sort By
    Date
    Flag#3

    View Slide

  74. 74/111
    #22
    German
    i18n
    Flag#5
    技術在我們手上 世界就在我們手上
    Story Mapping
    Code Test Acceptance Deploy Release Done
    #19
    Fix
    Login Bug
    #20
    Fix
    Email Bug
    #21
    Perf.
    Tuning
    #17
    Sort By
    Date
    Flag#3
    #18
    Filter By
    Group
    Flag#4

    View Slide

  75. 75/111
    01 提交程式至主幹可能會影響其他功能
    Commit code on trunk may affects other features.
    02 未完成的功能可能也會部署,如果配置有誤可能將未完成的功能開啟
    Unfinished features may be deployed, and turned on if the configuration is incorrect.
    Image Credit : https://opensource.com/article/18/2/feature-flags-ring-deployment-model
    Feature Flag

    View Slide

  76. 76/111
    #22
    German
    i18n
    Flag#5
    技術在我們手上 世界就在我們手上
    Story Mapping
    Code Test Acceptance Deploy Release Done
    #19
    Fix
    Login Bug
    #20
    Fix
    Email Bug
    #21
    Perf.
    Tuning
    #17
    Sort By
    Date
    Flag#3
    #18
    Filter By
    Group
    Flag#4

    View Slide

  77. 77/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  78. 78/111
    Facebook
    Image Credit : https://engineering.fb.com/web/rapid-release-at-massive-scale (2017-08-31)
    技術在我們手上 世界就在我們手上

    View Slide

  79. 79/111
    Microsoft/Azure
    Image Credit : Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf
    技術在我們手上 世界就在我們手上

    View Slide

  80. 80/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  81. 81/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.codonomics.com/2020/08/multi-tenant-architectures.html
    Multi-tenant Architectures ( 多租戶架構 )
    Shared Database Separate Database

    View Slide

  82. 82/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.codonomics.com/2020/08/multi-tenant-architectures.html
    Multi-tenant Architectures ( 多租戶架構 )
    Shared Database Separate Database

    View Slide

  83. 83/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.codonomics.com/2020/08/multi-tenant-architectures.html
    Multi-tenant Architectures ( 多租戶架構 )
    Shared Database Separate Database

    View Slide

  84. 84/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.codonomics.com/2020/08/multi-tenant-architectures.html
    Multi-tenant Architectures ( 多租戶架構 )
    Shared Database Separate Database
    Disable DROP/TRUNCATE/DELETE
    Soft Delete

    View Slide

  85. 85/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.codonomics.com/2020/08/multi-tenant-architectures.html
    Multi-tenant Architectures ( 多租戶架構 )
    Shared Database Separate Database

    View Slide

  86. 86/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Code Push ➡
    5 mins
    ⬅ Data Sync
    ? mins
    Code Push ➡
    15 mins
    Code Push ➡
    15 mins
    ⬅ Data Sync
    ? mins
    ⬅ Data Sync
    ? mins
    Easy
    Hard

    View Slide

  87. 87/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://memezila.com/Pizza-rolls-in-the-oven-vs-in-the-microwave-meme-3854

    View Slide

  88. 88/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://twitter.com/GitPushAll/status/1309533288567058434

    View Slide

  89. 89/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  90. 90/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.atlassian.com/blog/bitbucket/5-pull-request-must-haves
    Pull Requests

    View Slide

  91. 91/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.atlassian.com/blog/bitbucket/5-pull-request-must-haves
    Pull Requests
    Code Reviews are great
    But slow pipeline

    View Slide

  92. 92/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://www.atlassian.com/blog/bitbucket/5-pull-request-must-haves
    Pull Requests
    Code Reviews are great
    But slow pipeline
    Code Reviews Latency

    View Slide

  93. 93/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.submain.com/code-review-vs-pair-programming-2/
    Pair Programming

    View Slide

  94. 94/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://blog.submain.com/code-review-vs-pair-programming-2/
    Pair Programming
    Code Reviews Inline

    View Slide

  95. 95/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://team-coder.com/mob-programming/
    Mob Programming

    View Slide

  96. 96/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://team-coder.com/mob-programming/
    Mob Programming
    Continuous Code Reviews Inline

    View Slide

  97. 97/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  98. 98/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://docs.microsoft.com/zh-tw/dotnet/architecture/cloud-native/service-mesh-communication-infrastructure
    Service Mesh

    View Slide

  99. 99/111
    技術在我們手上 世界就在我們手上
    01 Feature Flag with Story Mapping
    功能標誌及使用者故事對照
    02 Progressive Deployment
    漸進式部署
    03 Data(base) Synchronization Strategy
    資料 ( 庫 ) 同步策略
    04 Pull Requests to Pair/Mob Programming
    從 Pull Requests 轉為 Pair/Mob Programming
    05 Service Mesh
    服務網格
    Testing in Production, Deploy on Fridays

    View Slide

  100. 100/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Testing in Production

    View Slide

  101. 101/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Testing in Production

    View Slide

  102. 102/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Testing in Production
    Reality ⬆
    Cost ⬇
    Maintenance ⬇

    View Slide

  103. 103/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://twitter.com/reillyusa/status/1260272851778768903

    View Slide

  104. 104/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://twitter.com/reillyusa/status/1260272851778768903

    View Slide

  105. 105/111
    技術在我們手上 世界就在我們手上
    Image Credit : https://twitter.com/reillyusa/status/1260272851778768903
    SRE≠ 呂布 (Reboot)

    View Slide

  106. 106/111
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.reddit.com/r/ProgrammerHumor/comments/ing3jx/works_on_my_machine/
    Image Credit : https://www.pngitem.com/middle/hwwmRmi_dove-clipart-flame-clipart-flame-png-transparent-png/
    技術在我們手上 世界就在我們手上
    在我電腦環境就沒問題

    View Slide

  107. 107/111
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.reddit.com/r/ProgrammerHumor/comments/ing3jx/works_on_my_machine/
    Image Credit : https://www.pngitem.com/middle/hwwmRmi_dove-clipart-flame-clipart-flame-png-transparent-png/
    技術在我們手上 世界就在我們手上
    在我電腦環境就沒問題

    View Slide

  108. 108/111
    Image Credit : https://jaxenter.com/deployment-best-practices-172662.html
    Image Credit : https://www.reddit.com/r/ProgrammerHumor/comments/ing3jx/works_on_my_machine/
    Image Credit : https://www.pngitem.com/middle/hwwmRmi_dove-clipart-flame-clipart-flame-png-transparent-png/
    技術在我們手上 世界就在我們手上
    在我電腦環境就沒問題
    Confidence & Trust ⬆

    View Slide

  109. 109/111
    Deploy on ... Fridays
    Image Credit : https://memes.tw/collection/388
    技術在我們手上 世界就在我們手上

    View Slide

  110. 110/111
    [email protected]
    https://www.facebook.com/yftzeng.tw
    https://twitter.com/yftzeng
    曾義峰 (Ant)

    View Slide

  111. 技術在我們手上 世界就在我們手上
    曾義峰 (Ant)
    LeadBest Consulting Group 首席執行顧問
    [email protected]

    View Slide