Laying the Cultural and Technical Foundation for Big Rails

As applications built on Rails get larger and larger, and more and more engineers work in the same monolith, our community needs to think more about what sort of tooling and architectural changes will help us continue to scale. This talk shares ideas around a toolchain, and more importantly, the social and cultural programs needed to support that toolchain, that can be used to help engineers in an ever-growing Rails codebase continue to have high velocity, manage their complexity, and claim ownership over their own business subdomains.

Transcript

1. Laying the Cultural and Technical Foundation for Big Rails RailsConf, May 2022 Alex Evanczuk
2. Thank you to those who have helped maintain and contribute to Rails. Thank you to those who have helped put together this conference.
3. About Me We’re hiring! https://gusto.com/careers For questions, feedback, or info on working at Gusto: alex@gusto.com
5. A Story of Growing Complexity at Gusto
6. Is this story familiar to you? There’s something we can do about it.
7. Big Rails
8. What is Big Rails? A system of sociotechnical tools, practices, and conventions that scale Rails development in terms of lifespan, number of contributors, and complexity. A Definition Accountability and Ownership Clear Boundaries Thoughtful Dependency Management Gradual Adoption Intentionally Curated and Sustainable Feedback Loops Major Principles
9. Accountability and Ownership
10. module PayrollSyncer def self.sync_deductions! raise "I am an error" end end
11. github.com/bigrails/code_ownership github.com/bigrails/code-ownership-vscode github.com/bigrails/bigrails-teams # config/teams/payroll.yml name: Payroll slack: handle: '@payroll-team' room_for_robots: '#payroll-notifications' room_for_humans: '#payroll’ github: team: '@Gusto/payroll' owned_globs: - app/services/payroll_syncer.rb team = Teams.find(‘Payroll') CodeOwnership.for_file('app/services/payroll_syncer.rb') == team CodeOwnership.for_class(PayrollSyncer) == team begin PayrollSyncer.sync_deductions! rescue => ex CodeOwnership.for_backtrace(ex.backtrace) == team end
12. Clear Boundaries and Dependency Management
13. app/ models/ benefits/ payroll/ hr/ views/ benefits/ payroll/ hr/ controllers/ benefits/ payroll/ hr/ services/ benefits/ payroll/ hr/
14. app/ models/ benefits/ payroll/ hr/ views/ benefits/ payroll/ hr/ controllers/ benefits/ payroll/ hr/ services/ benefits/ payroll/ hr/
15. app/ models/ benefits/ payroll/ hr/ views/ benefits/ payroll/ hr/ controllers/ benefits/ payroll/ hr/ services/ benefits/ payroll/ hr/ github.com/bigrails/stimpack packs/ benefits/ models/ views/ controllers/ services/ payroll/ models/ views/ controllers/ services/ hr/ models/ views/ controllers/ services/
16. With domain-based organization, how can we now systematically manage the relationship between those domains?
17. Packwerk
18. packs/ benefits/ models/ views/ controllers/ services/ + package.yml payroll/ models/ views/ controllers/ services/ + package.yml hr/ models/ views/ controllers/ services/ + package.yml # packs/benefits/package.yml enforce_dependencies: true enforce_privacy: true dependencies: - packs/payroll metadata: owner: Enrollments github.com/shopify/packwerk
19. packs/ benefits/ models/ views/ controllers/ services/ + public/ + package.yml payroll/ models/ views/ controllers/ services/ + public/ + package.yml hr/ models/ views/ controllers/ services/ + public/ + package.yml # packs/benefits/package.yml enforce_dependencies: true enforce_privacy: true dependencies: - packs/payroll metadata: owner: Enrollments github.com/shopify/packwerk
26. # packs/benefits/app/services/benefits.rb module Benefits def self.some_method HrHelper.inform_administrator! end end # packs/benefits/deprecated_references.yml --- packs/hr: "::HrHelper": violations: - privacy - dependency files: - packs/benefits/app/services/benefits.rb
27. Okay… but what about gems and engines? github.com/bigrails/package_protections Gems/Engines Packages Supports gradual modularity ❌ ✅ Inexpensive to change boundaries ❌ ✅ Supports distribution ✅ ❌ Supports versioning ✅ ❌ Fast tests ✅ ✅* Supports strict boundaries ✅ ✅** Engine features ✅ ✅*** * With spring and bootsnap ** With package_protections *** With stimpack
28. To start off, 100% of packages do not enforce privacy or dependency protections. The challenge: Push every team to turn on privacy and dependency enforcement Gradual Adoption
29. github.com/bigrails/modularization_statistics
30. github.com/bigrails/modularization_statistics
31. Feedback loops to focus on… Dev writes code Dev commits code Dev’s packwerk build step fails Dev updates the packwerk TODO list Dev merges PRs to the main branch ? ? ? ? ?
32. Dev writes code Dev commits code Dev’s packwerk build step fails Dev updates the packwerk TODO list Dev merges PRs to the main branch ? ? ? ‼‼‼ ? Feedback loops to focus on…
34. Dev writes code Dev commits code Dev’s packwerk build step fails Dev updates the packwerk TODO list Dev merges PRs to the main branch VSCode Extension Pre-commit Hooks Inline Automated PR Comments Inline Automated PR Comments Datadog Metrics github.com/bigrails/packwerk-vscode github.com/bigrails/danger-packwerk github.com/bigrails/modularization_statistics Feedback loops to focus on…
35. github.com/bigrails/packwerk-vscode github.com/bigrails/danger-packwerk github.com/bigrails/modularization_statistics
37. What’s next?
38. In what ways can Ruby and Rails continue to offer great tools and cultural norms that help users create well-modularized systems?
39. What can the different conventions of packwerk packages, gemspecs, and other packaging systems learn from each other?
40. We’re hiring! https://gusto.com/careers Feedback and Communities - Emailing me: alex@gusto.com or just starting a conversation - Creating issues, starting discussions, or opening PRs in any repository within https://github.com/bigrails - Joining us in the Ruby/Rails Modularity Slack Server: https://rubymod.slack.com - Invite link: https://tinyurl.com/ruby-modularity - Catch me after the talk and let’s pair! Other Resources and Tools - “Gradual Modularization for Ruby and Rails” by Stephan Hagemann - “Thinking in Systems” by Donella Meadows - “Boundaries” by Gary Bernhardt - https://engineering.gusto.com (Gusto’s Engineering Blog) - Just released: Blog post version of this talk! Your feedback is important and appreciated!