Upgrade to Pro — share decks privately, control downloads, hide ads and more …

State of the RubyGems 2023

State of the RubyGems 2023

Given at RubyConf 2023 in San Diego, CA.

Samuel E. Giddins

November 14, 2023
Tweet

More Decks by Samuel E. Giddins

Other Decks in Programming

Transcript

  1. State of the
    RubyGems
    2023
    1 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  2. Your intrepid presenter
    @segiddins
    → Samuel Giddins
    → RubyGems, Bundler, RubyGems.org maintainer
    → Security lead
    → 10+ year bug contributor
    2 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  3. Where did we come from?
    RubyGems was created at this very conference 19
    years ago
    3 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  4. Where did we come from?
    Growth of the Ruby ecosystem made it hard to
    maintain many gems on a system by hand
    4 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  5. Where did we come from?
    Bundler came in 2009 to solve the problem of
    maintaining gems by hand.
    Increased usage made it difficult for volunteers to
    keep up with the growing number of Bundler &
    RubyGems.org users
    5 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  6. Enter Ruby Together Central
    6 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  7. Enter Ruby Central
    7 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  8. RubyGems statistics
    8 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  9. RubyGems statistics
    As of October 2023, our
    major numbers are
    → 181,745 users
    → 192,825 gems
    → 1,555,458 versions of gems
    → 147,326,326,048 total gem
    downloads
    → average 20,000 requests/
    second
    → average 2 billion
    requests/weekday
    → maximum 225,000
    requests/second
    → 7.5 TB/hour, 185 TB/day
    → 4.6 PB/month, 54 PB/year
    9 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  10. RubyGems statistics
    That’s a lot of gems!
    10 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  11. Ruby Central’s Open
    Source team
    11 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  12. Ruby Central’s Open Source team
    → a 24/7 on-call team
    → the SRE team, configuring and operating Fastly, AWS, Kubernetes,
    Postgres, OpenSearch, Redis, and more
    → the RubyGems.org team, maintaining the Rails application
    → the RubyGems team, maintaining the gem command
    → the Bundler team, maintaining the bundle command
    → the Ruby Toolbox team
    → the RubyAPI team
    → the Gemstash team
    12 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  13. Ruby Central’s Open Source team
    Thank you to each & every contributor
    13 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  14. Ruby Central’s Open Source team
    Here at RubyConf
    Colby Martin
    14 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  15. What have we done?
    15 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  16. What have we done?
    Kept the lights on
    16 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  17. What have we done?
    Made many releases
    17 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  18. What have we done?
    Lots of good stuff
    18 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  19. Running RubyGems is
    challenging
    19 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  20. Running RubyGems is challenging
    Incidents
    → Dependency API deprecation caused a 25x increase
    in traffic, from ~10k rps to ~225k rps
    → Migrating from unicorn to puma (and renaming the
    k8s deployment) lost our horizontal pod autoscaler
    rule, causing the single pod to get overwhelmed
    → Early May pager storm
    20 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  21. Running RubyGems is challenging
    Incidents
    Weekly AWS costs Weekly bytes served by
    the Rails app
    21 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  22. Running RubyGems is challenging
    Recurring security issues
    → Dependency confusion
    → Account takeover
    → Good maintainer gone bad
    22 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  23. Running RubyGems is challenging
    Service uptime
    → DDoS
    → Bitrot
    → New security vulns
    → Deprecated infrastructure
    23 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  24. Running RubyGems is challenging
    Support requests
    → Lost account access
    → Abandoned (sometimes empty) projects
    → Limited supply of names for gems
    → Higher security means more customer service
    →Requiring MFA means many more manual MFA resets by staff
    → Ownership disputes
    →Multiple people claim ownership
    →Companies claim trademark or IP
    24 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  25. Running RubyGems is
    challenging
    Security researcher reports
    25 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  26. Running RubyGems is
    challenging
    Infrastructure is precarious
    26 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  27. What about 2023?
    27 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  28. What about 2023?
    Improved MFA and added support for passkeys
    instead of OTP codes
    28 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  29. What about 2023?
    Better uptime and stability by removing the
    dependency API
    29 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  30. What about 2023?
    → Implemented a PubGrub-based dependency
    resolver for Bundler
    → Added gem exec to RubyGems
    → Shipped the bundle compose beta
    30 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  31. What about 2023?
    Merged hundreds of pull requests across 19 versions of
    Bundler & RubyGems
    → Significant performance improvements for large application
    bundles
    → ruby(file: ".ruby-version") support for Gemfiles to re-use
    version files
    → fully allowlist-based safe loading for Marshal files,
    completely removing a repeated source of security issues
    31 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  32. What about 2023?
    Made maintaining RubyGems.org significantly easier
    → Continued migration to infrastructure as code
    → Multiple environments for testing changes
    → Web-based admin tools to reduce SSH & console access
    → Backend support for gem contents and OIDC auth
    → User-facing features based on these improvements
    coming soon
    32 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  33. What about 2023?
    It's been a productive year!
    33 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  34. How much does this
    cost???
    34 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  35. How much does this cost?
    Organizational overhead
    35 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  36. How much does this cost?
    AWS
    $20,000 / month
    36 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  37. How much does this cost?
    Fastly
    $500,000 / month
    37 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  38. How much does this cost?
    Developers
    $45,000 - $55,000 / month
    38 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  39. How much does this cost?
    39 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  40. Funding sources
    40 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  41. Funding sources
    Donated services
    → DNSimple
    → DataDog
    → Honeybadger
    → Mend.io
    → AWS
    → Fastly
    41 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  42. Funding sources
    Donated services
    42 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  43. Funding sources
    Ruby Central Memberships & Partnerships
    43 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  44. Funding sources
    Memberships
    rubycentral.org
    → Single developers
    → Small companies
    → Large corporations
    44 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  45. Funding sources
    Partnerships
    → Shopify
    → German Sovereign Tech Fund
    45 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  46. What we fund right now
    46 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  47. What we fund right now
    24/7/365 On-Call Rotation
    47 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  48. What we fund right now
    Infrastructural work
    → SRE work
    → Gem transparency
    48 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  49. What we fund right now
    Industry-wide collaboration
    → Languages
    → Companies
    49 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  50. What we fund right now
    Gem clients
    → RubyGems
    → Bundler
    50 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  51. What we fund right now
    Slow & steady progress
    51 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  52. We would like to fund more
    52 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  53. Our goals
    53 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  54. Our goals
    Full-time security role
    → Trusted publishing
    → Checksum verification
    → Full passkey support
    → TUF (the update framework)
    → SLSA-compliant builders
    → Sigstore support
    54 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  55. Our goals
    Ecosystem quality-of-life improvements
    → More gem information
    → Browsable gem contents
    → Downloads over time
    → Hosted & searchable documentation
    → In-browser gem playgrounds
    55 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  56. How to support us
    56 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  57. How to support us
    Contribute code
    github.com/rubygems
    57 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  58. How to support us
    Read RFCs and give us feedback.
    github.com/rubygems/rfcs
    58 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  59. How to support us
    Share this info
    59 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  60. How to support us
    Become a member
    rubycentral.org
    60 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  61. How to support us
    Become a partner to Ruby Central
    Adarsh
    [email protected]
    61 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  62. Give us feedback
    62 — State of the RubyGems 2023 @ RubyConf

    View full-size slide

  63. Thank you
    63 — State of the RubyGems 2023 @ RubyConf

    View full-size slide