Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to build deppbot

Juanito Fatas
September 24, 2016

How to build deppbot

@ RubyConf China 2016

Juanito Fatas

September 24, 2016
Tweet

More Decks by Juanito Fatas

Other Decks in Technology

Transcript

  1. RubyConf China 2016
    How to build
    deppbot
    Stories from building https:/
    /www.deppbot.com

    View full-size slide

  2. ౯ݝฎ...
    ኼ睞
    I only know a little

    View full-size slide

  3. deppbot core team
    @JuanitoFatas

    View full-size slide

  4. DANGER CONTRIBUTOR
    danger.systems

    View full-size slide

  5. Rails Guides by @AndorChen

    View full-size slide

  6. You may know me from

    View full-size slide

  7. First Time
    in ౮᮷

    View full-size slide

  8. ౯犥獮犖ฎ㮆嘨蜦
    聲硬襑穩䋿ࣁݑ犋ԧ
    ݝঅ㬵樄敋蝍穩ᛔኧ
    ᄆᄆݪ秚

    View full-size slide

  9. 匍ࣁ౯ᛔ૩樄敋మ
    䓄ࠨ疰䓄ࠨ疰ᓒ

    View full-size slide

  10. ॠሴ聲ৼ犖
    犋胼瞲ֵ౯

    View full-size slide

  11. ᮎ䒍猆
    讔簡蝡㱾
    瞤硛ک瞨

    View full-size slide

  12. ౮᮷Ӟଷ㬵ԧ
    疰犋మ
    櫝樄ጱउ૱

    View full-size slide

  13. ࢯࣁ泷໲Ӥ೉஑
    ૪妿睲綡Ոኞԧ
    礬๜蚎犋ԧ

    View full-size slide

  14. ᯿碝ਧ嬝ԧ
    犋ᬘ疰ฎᬘ
    ᬘ疰ฎ粁ᬘ

    View full-size slide

  15. How often
    do you
    update?
    !!

    View full-size slide

  16. LATER
    EQUALS
    NEVER

    View full-size slide

  17. Winston Teo
    Practice of updating all
    dependencies to
    newer versions several
    times a month.
    Founder, Jolly Good Code

    Continuous Updates

    View full-size slide

  18. 論持續更新
    時常更新項⽬
    的 RubyGems
    乃最佳實踐也
    — 胡適之

    View full-size slide

  19. Benefits of
    Continuous Updates

    View full-size slide

  20. INCREMENTAL
    IMPROVEMENTS

    View full-size slide

  21. FIX SECURITY
    VULNERABILITIES

    View full-size slide

  22. REDUCE

    TECHNICAL DEBT

    View full-size slide

  23. MAKE FUTURE
    UPGRADE EASIER

    View full-size slide

  24. DEVELOPER
    DISLIKE
    LEGACY GEMS

    View full-size slide

  25. MAINTAINER

    LOVE BUGS FROM
    NEW RELEASES

    View full-size slide

  26. CONTINUOUS
    LEARNINGS FROM
    GEM UPDATES

    View full-size slide

  27. SHIP LATEST
    SOFTWARES

    View full-size slide

  28. deppbot is a…
    Automated
    Updates
    Service

    View full-size slide

  29. deppbot is a…
    Security
    Updates
    Service

    View full-size slide

  30. deppbot is a…
    Dependency
    Updates
    Service

    View full-size slide

  31. Why built
    deppbot?

    View full-size slide

  32. Worked at
    Consultancy

    View full-size slide

  33. Client
    Projects

    View full-size slide

  34. Minimum
    Value
    Product

    View full-size slide

  35. Ship
    Latest
    Gems

    View full-size slide

  36. I like to
    keep my Gems
    updated

    View full-size slide

  37. HOWTO
    USE deppbot

    View full-size slide

  38. 1. Sign Up / Sign In

    View full-size slide

  39. 2. Subscribe

    View full-size slide

  40. deppbot adds herself to your repo
    Run Automated Updates
    every 1.hour do
    if need_to_update?
    Run Automated Updates
    end
    end

    View full-size slide

  41. git clone

    works for project hosts on bitbucket, submodules too

    View full-size slide

  42. bundle update

    Updates Gemfile.lock and install gems

    View full-size slide

  43. Travis CI?
    Not all kinds of gems can be installed on a single VPS

    View full-size slide

  44. bundle lock --update

    Updates Gemfile.lock without installing
    This command re-introduced in bundler v1.10 #3439

    View full-size slide

  45. diff -u

    Diff of Gemfile.lock (before / after)

    View full-size slide

  46. Delete Repo on VPS

    immediately when we got the diff

    View full-size slide

  47. Cook Pull Request

    gem links, compare views, changelogs, time savings

    View full-size slide

  48. Pull Request
    &YBN
    QMF

    View full-size slide

  49. nokogiri

    Query RubyGems.org API
    Gem authors, please fill in your metadata

    View full-size slide

  50. nokogiri

    Find GitHub URL from RubyGems data
    Gem authors, please fill in your metadata

    View full-size slide

  51. 1.6.6.4…1.6.7

    Parse diff and link_to repository compare view for code review
    Gem authors, please push your tags when release a gem

    View full-size slide

  52. CHANGELOG

    Query GitHub API, jollygoodcode/whatsnew
    Don’t let your friends dump git logs into CHANGELOGs

    View full-size slide

  53. Time Savings

    Sum every Pull Request processed time

    View full-size slide

  54. Send Pull Request

    Merged and keep up-to-date

    View full-size slide

  55. Bundler
    uses 250MB
    v1.7.2

    View full-size slide

  56. Some Bundler
    features only
    available at v1.9.x

    View full-size slide

  57. Fork
    buildpack for
    Custom Bundler
    version

    View full-size slide

  58. Fork
    buildpack for
    Custom Bundler
    version
    Digital
    Ocean

    View full-size slide

  59. Digital Ocean 1CPU
    Production*1 (2GB)
    Staging*1 (1GB)
    Amazon RDS
    http:/
    /stackshare.io/deppbot/deppbot

    View full-size slide

  60. 2150 commits
    515 Pull Requests

    View full-size slide

  61. deppbot uses
    deppbot to
    build deppbot

    View full-size slide

  62. FAST TEST SUITE
    FAST FEEDBACK

    View full-size slide

  63. https://www.deppbot.com
    8629
    Pull Requests Sent
    944
    hours
    Engineering Time Saved

    View full-size slide

  64. The Birth of
    deppbot
    2015.09.03
    https:/
    /github.com/jollygoodcode/jollygoodcode.github.io/issues/3

    View full-size slide

  65. Automated
    Bundle
    Updates

    View full-size slide

  66. Automated
    Security
    Updates
    2015.12.25
    https:/
    /github.com/jollygoodcode/jollygoodcode.github.io/issues/15

    View full-size slide

  67. 2015.12.25
    https:/
    /github.com/jollygoodcode/jollygoodcode.github.io/issues/15

    View full-size slide

  68. Small Features
    Improvements
    Bug fixes
    Refactorings

    View full-size slide

  69. Not only
    deppbot

    View full-size slide

  70. Other
    Services

    View full-size slide

  71. https:/
    /libraries.io
    https:/
    /gemnasium.com
    https:/
    /requires.io
    https:/
    /david-dm.org
    Notification
    Only notify you

    View full-size slide

  72. Actionable
    https:/
    /deppbot.com
    http:/
    /pyup.io
    https:/
    /greenkeeper.io
    http:/
    /tachikoma.io
    Do, don’t tell

    View full-size slide

  73. From idea
    to product

    View full-size slide

  74. bundle update
    add, commit, push
    open a new PR on GitHub.com
    Issue the Pull Request
    Manually

    View full-size slide

  75. today = Time.current.strftime("%F")
    new_branch = "bundle-update-#{today}"
    `git checkout master`
    `git pull`
    `git checkout -b #{new_branch}`
    `bundle update`
    `git add Gemfile.lock`
    `git commit -m ‘Bundle Updates’
    `git push origin #{new_branch}`
    `git pull-request -m "Updates #{today}"
    Script

    View full-size slide

  76. today = Time.current.strftime("%F")
    new_branch = "bundle-update-#{today}"
    `git checkout master`
    `git pull`
    `git checkout -b #{new_branch}`
    `bundle update`
    `git add Gemfile.lock`
    `git commit -m ‘Bundle Updates’
    `git push origin #{new_branch}`
    `git pull-request -m "Updates #{today}"
    15
    m
    ins

    View full-size slide

  77. Discover
    tachikoma.io

    View full-size slide

  78. Too
    Expensive
    $49/mo

    View full-size slide

  79. Some clients
    are also
    interested

    View full-size slide

  80. Decided to
    create a
    SaaS

    View full-size slide

  81. Hence
    deppbot

    View full-size slide

  82. Dependency
    Bot = depbot

    View full-size slide

  83. depbot is
    taken

    View full-size slide

  84. Johnny depp
    is cool

    View full-size slide

  85. How does it
    work?

    View full-size slide

  86. ! "
    Your Project deppbot GitHub
    Subscribe Automated
    Updates

    View full-size slide

  87. Normal Updates
    Security Updates
    Automated Updates

    View full-size slide

  88. Normal Updates
    1. Need update?
    2. bundle update
    3. Send Pull Request

    View full-size slide

  89. Normal Updates
    1. Need update?
    2. bundle update
    3. Send Pull Request

    View full-size slide

  90. bundle update
    1. clone & setup
    2. start new build
    3. bundle update
    4. store diff

    View full-size slide

  91. bundle update

    View full-size slide

  92. bundle update

    View full-size slide

  93. Normal Updates
    1. Need update?
    2. bundle update
    3. Send Pull Request

    View full-size slide

  94. Send Pull Request
    1. Check if can send?
    2. Send it
    3. Finish build

    View full-size slide

  95. Send Pull Request

    View full-size slide

  96. Send Pull Request

    View full-size slide

  97. Send Pull Request

    View full-size slide

  98. Security Updates
    1. Need update?
    2. security update
    3. Send Pull Request

    View full-size slide

  99. Security Updates
    1. Need update?
    2. security update
    3. Send Pull Request

    View full-size slide

  100. GitHub
    Organisations API
    GitHub permissions
    https:/
    /github.com/jollygoodcode/jollygoodcode.github.io/issues/11

    View full-size slide

  101. GitHub API
    limit

    View full-size slide

  102. ݶӞ㮆Ӯኴ
    ݶӞ㮆瓵మ

    View full-size slide

  103. 1 PR
    at a time

    View full-size slide

  104. Listen to PR events
    Track if you merged
    Don’t send PR if open
    Webhook

    View full-size slide

  105. GitHub is
    down

    View full-size slide

  106. RubyGems.org
    Downtime

    View full-size slide

  107. Many other
    tricky cases

    View full-size slide

  108. Idea is
    CHEAP

    View full-size slide

  109. Create value
    for users

    View full-size slide

  110. Marketing
    is HARD

    View full-size slide

  111. Bugs are
    unpredictable

    View full-size slide

  112. Ruby is
    Elegant and
    Beautiful

    View full-size slide

  113. Open Source
    from real app

    View full-size slide

  114. Share what
    you learned

    View full-size slide

  115. Code is
    useless
    till shipped

    View full-size slide

  116. Embrace
    Changes

    View full-size slide

  117. Raises
    Awareness
    of gems

    View full-size slide

  118. Continuous
    Learnings

    View full-size slide

  119. Continuous
    Updates

    View full-size slide

  120. Update Early

    View full-size slide

  121. Update Often

    View full-size slide

  122. YES WE
    CAN DO IT!!

    View full-size slide

  123. #MakeRubyGreatAgain
    http://blog.testdouble.com/posts/2016-05-09-make-ruby-great-again.html
    Hopefully

    View full-size slide

  124. I have
    deppbot
    stickers

    View full-size slide

  125. Special Thanks
    @huacnlee
    @lgn21st

    View full-size slide

  126. Any
    Questions?

    View full-size slide