Slide 1
Slide 1 text
Modularizing with Packwerk
Diego Algorta
oboxodo
Ruby Montevideo Meetup 2024-04-24
Slide 2
Slide 2 text
Who am I?
Slide 3
Slide 3 text
I read these books in 2003 and fell in ♥ with Ruby
Slide 4
Slide 4 text
First Ruby Meetup in UY? - UYLUG 2007-07-28
Slide 5
Slide 5 text
Oh, the youth…
Slide 6
Slide 6 text
Oh, the youth…
so full of black hair.
Slide 7
Slide 7 text
so full of black hair.
so full of hair.
Oh, the youth…
Slide 8
Slide 8 text
Who am I?
In relation to Ruby…
● Learnt Ruby in 2003 but it
didn’t pay the bills
● Working remotely with Rails
since 2007 for various clients
● Cubox 2009-2014
● RubyConf Uruguay 2010-2014
● First tech hire at Quimbee.com
2014-...
Slide 9
Slide 9 text
RubyConf Uruguay 2010, 2011, 2012, 2013, 2014
Slide 10
Slide 10 text
Let’s get serious
Slide 11
Slide 11 text
Example app
Slide 12
Slide 12 text
Example app - just models for brevity
Classic patterns from a Rails monolith that has
grown over time:
● Started as a simple learning system with
users taking courses and keeping track of
their progress.
Slide 13
Slide 13 text
Example app - just models for brevity
Classic patterns from a Rails monolith that has
grown over time:
● Started as a simple learning system with
users taking courses and keeping track of
their progress.
● After some time a blogging system is
added but commenting is implemented as
a generic feature for any model
(polymorphic).
● User ends up being a “God object”.
● Namespacing as unenforced
modularization.
● Original models are not namespaced.
Slide 14
Slide 14 text
What is Packwerk?
Slide 15
Slide 15 text
What is Packwerk? - From Shopify - Straight from their readme file
Ruby gem used to enforce boundaries and modularize Rails applications.
Packwerk can be used to:
● Combine groups of files into packages
● Define package-level constant visibility (i.e. have publicly accessible
constants)
● Help existing codebases to become more modular without obstructing
development
Source: https://github.com/Shopify/packwerk
Slide 16
Slide 16 text
Combine groups of
files into packages
(aka packs)
Slide 17
Slide 17 text
Combine groups of
files into packages
(aka packs)
…progressively ♥
Slide 18
Slide 18 text
Ruby at Scale
Set of gems built around packwerk
● packs
● packs-rails
● packs-specification
● code_ownership
● visualize_packs
● …and a bunch more…
https://github.com/rubyatscale
Tools from gusto.com
Slide 19
Slide 19 text
Stuff you can define per pack
Built-in:
● Dependency
Extensions:
● Privacy
● Layer (formerly “Architecture”)
● Ownership
● …and more.
# packs/my_pack/package.yml
enforce_dependencies : true
enforce_privacy : false
enforce_layers : true
owner: Team A
layer: Feature
dependencies:
- packs/foo
- packs/bar
- packs/baz
Slide 20
Slide 20 text
Example from visualize_packs gem
Slide 21
Slide 21 text
How does a modularized monolith look like?
.
├── app
│ ├── controllers
│ ├── models
│ ├── views
│ └── ...
├── lib
└── spec
Slide 22
Slide 22 text
How does a modularized monolith look like?
.
├── app
│ ├── controllers
│ ├── models
│ ├── views
│ └── ...
├── lib
└── spec
.
├── package.yml
├── app
│ └── ...as empty as possible...
├── lib
├── packs
│ ├── foo
│ │ ├── package.yml
│ │ ├── package_todo.yml
│ │ ├── app
│ │ │ ├── controllers
│ │ │ ├── models
│ │ │ └── views
│ │ ├── config
│ │ ├── lib
│ │ └── spec
│ └── bar
│ ├── package.yml
│ └── ...
└── spec
Slide 23
Slide 23 text
How does a modularized monolith look like?
.
├── app
│ ├── controllers
│ ├── models
│ ├── views
│ └── ...
├── lib
└── spec
.
├── package.yml
├── app
│ └── ...as empty as possible...
├── lib
├── packs
│ ├── foo
│ │ ├── package.yml
│ │ ├── package_todo.yml
│ │ ├── app
│ │ │ ├── controllers
│ │ │ ├── models
│ │ │ └── views
│ │ ├── config
│ │ ├── lib
│ │ └── spec
│ └── bar
│ ├── package.yml
│ └── ...
└── spec
Slide 24
Slide 24 text
Most important commands
# Validates configuration correctness. Run it on the CI.
bin/packs validate
# Checks code against package_todo.yml files. Run it on the CI!
bin/packs check
# Updates package_todo.yml files from current code. Be careful!
bin/packs update
# Help... duh.
bin/packs help
Slide 25
Slide 25 text
Splitting
strategies
You can mix these as needed
Vertical
● One pack per functionality with
all its files (controllers, models,
views, etc)
Horizontal
● Separate packs for Models,
Views, Controllers, etc.
Slide 26
Slide 26 text
Original
Slide 27
Slide 27 text
Original Adding Packs and Layers
Slide 28
Slide 28 text
Just a few patterns to fix violations/to-dos
Slide 29
Slide 29 text
⚠
None of these aim for
complete decoupling
(like microservices)
Slide 30
Slide 30 text
Aim for “I could
eventually extract this
as a gem or engine” if
I really wanted
Slide 31
Slide 31 text
Break belongs_to & has_one/many circular dependency
# Given an inter-packs `belongs_to` definition
# packs/learning/app/models/enrollment.rb
class Enrollment < ApplicationRecord
# Adds dependency from Enrollment to User.
# Necessary.
belongs_to :user
end
# Bad ❌
# packs/users/app/models/user.rb
class User < ApplicationRecord
# Adds dependency from User to Enrollment
# Necessary? (most of the time, not really).
has_many :enrollments, dependent: :destroy
end
Slide 32
Slide 32 text
Break belongs_to & has_one/many circular dependency
# Given an inter-packs `belongs_to` definition
# packs/learning/app/models/enrollment.rb
class Enrollment < ApplicationRecord
# Adds dependency from Enrollment to User.
# Necessary.
belongs_to :user
end
# Bad ❌
# packs/users/app/models/user.rb
class User < ApplicationRecord
# Adds dependency from User to Enrollment
# Necessary? (most of the time, not really).
has_many :enrollments, dependent: :destroy
end
# Good ✅
# Replace `user.enrollments` with:
Enrollment.where(user: user)
#
packs/learning/config/initializers/learning.rb
Rails.application.configure do
config.to_prepare do
# Subscribe to destroy notifications
User.before_destroy do |o|
Enrollment.where(user: o).destroy_all
end
end
end
Slide 33
Slide 33 text
belongs_to
Avoid referring to a model in a
higher-layer pack…
unless it’s polymorphic which
inverts the dependency.
<>
Commenting::Commentable
Commenting::Comment
Post
Lesson
Slide 34
Slide 34 text
belongs_to
Avoid referring to a model in a
higher-layer pack…
unless it’s polymorphic which
inverts the dependency.
<>
Commenting::Commentable
Commenting::Comment
Post
Lesson
Slide 35
Slide 35 text
belongs_to
Avoid referring to a model in a
higher-layer pack…
unless it’s polymorphic which
inverts the dependency.
<>
Commenting::Commentable
Commenting::Comment
Post
Lesson
Slide 36
Slide 36 text
Polymorphic belongs_to inverts dependency
# packs/blogging/app/models/post.rb
class Post < ApplicationRecord
# Adds dependency from blogging to commenting
# pack, which is in lower layer ✅.
include Commenting::Commentable
end
Slide 37
Slide 37 text
Polymorphic belongs_to inverts dependency
# packs/blogging/app/models/post.rb
class Post < ApplicationRecord
# Adds dependency from blogging to commenting
# pack, which is in lower layer ✅.
include Commenting::Commentable
end
# packs/commenting/app/models/concerns/commenting/commentable.rb
class Commenting::Commentable <
ApplicationRecord
extend ActiveSupport::Concern
included do
has_many :comments, as: :commentable,
dependent: :destroy
end
end
# packs/commenting/app/models/commenting/comment.rb
class Comment < ApplicationRecord
# DOESN’T add dependency on Post which is in
# a higher layer ✅.
belongs_to :commentable, polymorphic: true
end
Slide 38
Slide 38 text
Remove knowledge from God objects
# Bad ❌
# app/models/user.rb
class User < ApplicationRecord
# Adds unnecessary dependency from User to
# Post. It’s only important when the user
# actually uses the blogging feature.
has_many :posts
validates :name, presence: true
validates :blogger_alias, presence: true
end
Slide 39
Slide 39 text
Remove knowledge from God objects
# Bad ❌
# app/models/user.rb
class User < ApplicationRecord
# Adds unnecessary dependency from User to
# Post. It’s only important when the user
# actually uses the blogging feature.
has_many :posts
validates :name, presence: true
validates :blogger_alias, presence: true
end
# Good ✅
# packs/users/app/models/user.rb
class User < ApplicationRecord
# This model being in a pack on the lowest layer
# can be referred to from anywhere but must
# be very light on dependencies of its own.
validates :name, presence: true
end
# packs/blogging/app/models/blogging/profile.rb
class Blogging::Profile < ApplicationRecord
# Acts as the central point of contact from the
# blogging pack to the User.
belongs_to :user
delegate :name, to: :user
has_many :posts
validates :blogger_alias, presence: true
end
Slide 40
Slide 40 text
Avoid has_many :through through models in lower layers
# Bad ❌
class User
has_one :profile,
class_name: "Blogging::Profile"
has_many :enrollments
end
class Blogging::Profile
belongs_to :user
# This forces User to know about Enrollment
has_many :enrollments, through: :user
end
class Enrollment
belongs_to :user
end
Slide 41
Slide 41 text
Avoid has_many :through through models in lower layers
# Bad ❌
class User
has_one :profile,
class_name: "Blogging::Profile"
has_many :enrollments
end
class Blogging::Profile
belongs_to :user
# This forces User to know about Enrollment
has_many :enrollments, through: :user
end
class Enrollment
belongs_to :user
end
Slide 42
Slide 42 text
Avoid has_many :through through models in lower layers
# Bad ❌
class User
has_one :profile,
class_name: "Blogging::Profile"
has_many :enrollments
end
class Blogging::Profile
belongs_to :user
# This forces User to know about Enrollment
has_many :enrollments, through: :user
end
class Enrollment
belongs_to :user
end
Slide 43
Slide 43 text
Avoid has_many :through through models in lower layers
# Bad ❌
class User
has_one :profile,
class_name: "Blogging::Profile"
has_many :enrollments
end
class Blogging::Profile
belongs_to :user
# This forces User to know about Enrollment
has_many :enrollments, through: :user
end
class Enrollment
belongs_to :user
end
Slide 44
Slide 44 text
Avoid has_many :through through models in lower layers
# Bad ❌
class User
has_one :profile,
class_name: "Blogging::Profile"
has_many :enrollments
end
class Blogging::Profile
belongs_to :user
# This forces User to know about Enrollment
has_many :enrollments, through: :user
end
class Enrollment
belongs_to :user
end
# Good ✅
class User; end
class Blogging::Profile
belongs_to :user
# Join directly using data in common (user_id),
# but avoiding the need for User model to know
# about the Enrollment model.
has_many :enrollments,
foreign_key: :user_id, primary_key: :user_id
end
class Enrollment
belongs_to :user
end
Slide 45
Slide 45 text
Use configuration when needed - contrived example, I know
# Bad ❌
Commenter: <%= user_display_name(@comment.user) %>
# packs/commenting/app/helpers/commenting/comments_helper.rb module Commenting::CommentsHelper def user_display_name(user) "#{user.name} (#{user.posts.count})" end endSlide 46
Slide 46 text
Use configuration when needed - contrived example, I know
# Bad ❌
Commenter: <%= user_display_name(@comment.user) %>
# packs/commenting/app/helpers/commenting/comments_helper.rb module Commenting::CommentsHelper def user_display_name(user) "#{user.name} (#{user.posts.count})" end end # Good ✅Commenter: <%= Commenting.user_display_name[@comment.user] %>
# packs/commenting/app/models/commenting.rb module Commenting mattr_accessor :user_display_name, default: ->(user) { user.name } end # config/initializers/commenting.rb Rails.application.configure do config.to_prepare do Commenting.user_display_name = ->(user) { profile = Profile.find_by(user: user) "#{user.name} (#{profile.posts.count})" } end endSlide 47
Slide 47 text
How much should we modularize?
Slide 48
Slide 48 text
Let’s define a “Maturity Framework”
Idea from: https://rubyrailsmodularity.com/t/seeking-advice-on-implementing-a-package-level-guide-to-categorize-and-track-progress/799
Slide 49
Slide 49 text
Milestone / Level 0 1 2 3 4
Packs exist ✅ ✅ ✅ ✅ ✅
All ruby files exist within a pack ✅ ✅ ✅ ✅
Checks are integrated into the build pipeline ✅ ✅ ✅ ✅
There are no architectural violations ✅ ✅ ✅
Packs have code owner and basic documentation ✅ ✅ ✅
Packs have an initial public API ✅ ✅
Packs are well-documented ✅ ✅
There are no dependency violations ✅ ✅
Packs interfaces don’t use ActiveRecord objects ✅
There are no privacy violations ✅
Maturity Framework
Slide 50
Slide 50 text
One size doesn’t fit all
Slide 51
Slide 51 text
Pareto principle
(aka 80/20 rule)
Slide 52
Slide 52 text
Maturity Framework
Milestone / Level 0 1 ⭐ 3 4
Packs exist ✅ ✅ ✅ ✅ ✅
All ruby files exist within a pack ✅ ✅ ✅ ✅
Checks are integrated into the build pipeline ✅ ✅ ✅ ✅
There are no architectural violations ✅ ✅ ✅
Packs have code owner and basic documentation ✅ ✅ ✅
Packs have an initial public API ✅ ✅
Packs are well-documented ✅ ✅
There are no dependency violations ✅ ✅
Packs interfaces don’t use ActiveRecord objects ✅
There are no privacy violations ✅
Slide 53
Slide 53 text
We found great value on Layers
Focus on layers dependencies first
● Higher layer ➡ lower layer ✅
● Lower layer ➡ higher layer ❌
Then:
● A pack on Application layer can
depend on packs on ANY other
layer.
● A pack on Infrastructure layer
can ONLY depend on packs on
the same layer.
# example layers in packwerk.yml
layers:
- Application # Rails app itself
- Products # Cohesive “sub-apps”
- Features # Shareable or small
- Infrastructure # Globals
Slide 54
Slide 54 text
Measuring progress over time
Slide 55
Slide 55 text
Measuring progress over time
$ bin/packs get_info --format csv --types layer --include-date
Date,Pack name,Owned by,Size,Public API,Inbound layer
violations,Outbound layer violations
2025-04-20,.,No one,95,app/public,0,0
2025-04-20,packs/blogging,No one,13,packs/blogging/app/public,1,0
2025-04-20,packs/learning,No one,12,packs/learning/app/public,4,0
2025-04-20,packs/content,No one,12,packs/content/app/public,0,3
2025-04-20,packs/users,No one,12,packs/users/app/public,0,3
2025-04-20,packs/commenting,No one,8,packs/commenting/app/public,1,0
2025-04-20,packs/rails_shims,No one,5,packs/rails_shims/app/public,0,0
Slide 56
Slide 56 text
Date Pack name
Owned
by Size Public API
Inbound
layer
violations
Outbound
layer
violations
2025-04-20 . No one 95 app/public 0 0
2025-04-20 packs/blogging No one 13 packs/blogging/app/public 1 0
2025-04-20 packs/learning No one 12 packs/learning/app/public 4 0
2025-04-20 packs/content No one 12 packs/content/app/public 0 3
2025-04-20 packs/users No one 12 packs/users/app/public 0 3
2025-04-20 packs/commenting No one 8 packs/commenting/app/public 1 0
2025-04-20 packs/rails_shims No one 5 packs/rails_shims/app/public 0 0
Measuring progress over time
Slide 57
Slide 57 text
Real Quimbee modularization progress pivot table sample
Slide 58
Slide 58 text
Real Quimbee modularization progress pivot table sample
Decided to stop
bleeding!
Slide 59
Slide 59 text
Real Quimbee modularization progress pivot table sample
Real push during
Jan-Apr 2024
Slide 60
Slide 60 text
Real Quimbee modularization progress pivot table sample
Reduced our
layer violations by
almost 60%!
Slide 61
Slide 61 text
Real Quimbee modularization progress pivot table sample
Damn it! We lost
momentum and
started bleeding a
bit again…
Slide 62
Slide 62 text
Real Quimbee modularization progress pivot table sample
But we’ve done a
decent job at keeping
the absolute number
of problem packs
reduced
Slide 63
Slide 63 text
The End
No time for more… 🫤
Slide 64
Slide 64 text
Questions?
Diego Algorta
@oboxodo
https://ob.oxo.do
[email protected]
Some resources:
https://leanpub.com/package-based-rails-applications
Stephan Hagemann
https://rubyrailsmodularity.com
https://github.com/rubyatscale
https://railsatscale.com