Slide 1
Slide 1 text
BETWEEN
Vladimir Dementyev
Evil Martians
Microservices
Monoliths
Slide 2
Slide 2 text
2
rails new monolith
Slide 3
Slide 3 text
3
Slide 4
Slide 4 text
The Big Four
vs
Others
4
Slide 5
Slide 5 text
We are
doomed
5
Slide 6
Slide 6 text
Can you escape
your destiny?
6
V.Yurlov, for “The war of the worlds” by H.G.Wells
Slide 7
Slide 7 text
7
Slide 8
Slide 8 text
8
A software system is called
“monolithic” if it has a monolithic
architecture, in which functionally
distinguishable aspects are all
interwoven, rather than containing
architecturally separate
components.—Wiki
Slide 9
Slide 9 text
Components
9
Slide 10
Slide 10 text
Monolith
10
Slide 11
Slide 11 text
11
Monoliths Microservices
Modular monoliths
Slide 12
Slide 12 text
Shopify
12
engineering.shopify.com/blogs/engineering/deconstructing-monolith-designing-software-maximizes-developer-productivity
Slide 13
Slide 13 text
Shopify
13
Source: speakerdeck.com/ssnickolay/evolution-of-rails-application-architecture-14-years-in-production
Slide 14
Slide 14 text
Hanami
14
Slide 15
Slide 15 text
Elixir
15
Slide 16
Slide 16 text
Rails
16
Slide 17
Slide 17 text
About me
Or how I ended up here
17
github.com/palkan
Slide 18
Slide 18 text
evilmartians.com
18
Slide 19
Slide 19 text
evilmartians.com
19
Ruby Next
Transpiler for
Ruby
Slide 20
Slide 20 text
evl.ms/blog
20
Slide 21
Slide 21 text
evilmartians.com
21
Slide 22
Slide 22 text
What we’ve
done
Or how we decided to
go with engines
22
Slide 23
Slide 23 text
Co-living rental service
23
Slide 24
Slide 24 text
Our mission
• Given a property and lease
management application
• Build a community portal for users
within the same Rails app
24
Slide 25
Slide 25 text
Community
• Events
• Perks
• Chat
• Billing
25
Slide 26
Slide 26 text
Phase #1: Namespaces
26
app/
controllers/
chat/…
models/
chat/…
jobs/
chat/…
Slide 27
Slide 27 text
Namespaces
• Quick start
• Fake isolation
27
Slide 28
Slide 28 text
• Prepare the code to re-use in the
future spinoff projects
• Given a property and lease
management application
• Build a community portal for users
within the same Rails app
Our mission evolves
28
Slide 29
Slide 29 text
Phase #2: Engines & Gems
29
app/…
engines/
chat/
app/
controllers/…
…
lib/…
gems/…
Slide 30
Slide 30 text
The Modular Monolith:
Rails Architecture
30
medium.com/@dan_manges/the-modular-monolith-rails-architecture-fb1023826fc4
Slide 31
Slide 31 text
Engems
Building Rails apps from engines
and gems
31
rails plugin new \
my_engine --mountable
Slide 32
Slide 32 text
Crash Course in Engines
$ rails plugin new my_engine \
--mountable # or --full
create
create README.md
create Rakefile
create Gemfile
...
create my_engine.gemspec
32
Engine is a gem
Slide 33
Slide 33 text
Crash Course in Engines
33
my_engine/
app/
controllers/ ...
...
config/routes.rb
lib/
my_engine/engine.rb
my_engine.rb
test/ ...
Added to autoload paths
Isolated tests
Slide 34
Slide 34 text
Crash Course in Engines
34
# my_engine/lib/my_engine/engine.rb
module MyEngine
class Engine < ::Rails ::Engine
# only in mountable engines
isolate_namespace MyEngine
end
end
Slide 35
Slide 35 text
Crash Course in Engines
35
# my_engine/config/routes.rb
MyEngine ::Engine.routes.draw do
get "/railsconf",
to: redirect("/stay_home")
end
Slide 36
Slide 36 text
Crash Course in Engines
36
# /Gemfile
gem "my_engine", path: "my_engine"
# /config/routes.rb
Rails.application.routes.draw do
mount MyEngine ::Engine => "/my_engine"
end
Slide 37
Slide 37 text
Popular Engines
37
actioncable/
actionmailbox/
actiontext/
activestorage/
railtie/
Slide 38
Slide 38 text
38
gem "devise"
Slide 39
Slide 39 text
The end
39
Slide 40
Slide 40 text
40
Slide 41
Slide 41 text
41
Slide 42
Slide 42 text
Common Engines
42
connect_by
meet_by
chat_by
manage_by
perks_by
main app
active-storage-proxy
common-events
Slide 43
Slide 43 text
43
Slide 44
Slide 44 text
Dependencies
• How to use non-Rubygems deps in
gems?
• How to share common deps?
• How to sync versions?
44
Slide 45
Slide 45 text
Path/Git problem
45
# engines/my_engine/Gemfile
gem "local-lib", path: " ../local-lib"
gem "git-lib", github: "palkan/git-lib"
# /Gemfile
gem "my_engine", path: "engines/my_engine"
gem "local-lib", path: " ../local-lib"
gem "git-lib", github: "palkan/git-lib"
Slide 46
Slide 46 text
eval_gemfile
46
# engines/my_engine/Gemfile
eval_gemfile "./Gemfile.runtime"
# engines/my_engine/Gemfile.runtime
gem "local-lib", path: " ../local-lib"
gem "git-lib", github: "palkan/git-lib"
# /Gemfile
gem "my_engine", path: "engines/my_engine"
eval_gemfile "engines/my_engine/Gemfile.runtime"
Slide 47
Slide 47 text
eval_gemfile
47
Slide 48
Slide 48 text
Shared Gemfiles
48
gemfiles/
profilers.gemfile
rails.gemfile
gem "stackprof", "0.2.12"
gem "ruby-prof", "0.17.0"
gem "memory_profiler", "0.9.12"
gem "rails", "6.0.0.rc1"
Slide 49
Slide 49 text
Shared Gemfiles
49
# engines/my_engine/Gemfile
eval_gemfile " ../gemfiles/rails.runtime"
eval_gemfile " ../gemfiles/profilers.runtime"
# /Gemfile
eval_gemfile "gemfiles/rails.runtime"
eval_gemfile "gemfiles/profilers.runtime"
Slide 50
Slide 50 text
Keep Versions N’Sync
50
gem "transdeps"
A gem to find inconsistent dependency
versions in component-based Ruby apps.
Slide 51
Slide 51 text
transdeps
51
action_policy-graphql (0.3.2) != action_policy-graphql (0.4.0) (chat_by )
faraday (0.17.3) != faraday (1.0.1) (chat_by )
graphql (1.10.4) != graphql (1.10.5) (chat_by )
parser (2.7.0.3) != parser (2.7.0.4) (chat_by )
pg (1.2.2) != pg (1.2.3) (chat_by )
rspec-expectations (3.9.0) != rspec-expectations (3.9.1) (chat_by )
rspec-rails (4.0.0.beta4) != rspec-rails (4.0.0.rc1) (chat_by )
thor (0.20.3) != thor (1.0.1) (chat_by )
twilio-ruby (5.31.1) != twilio-ruby (5.32.0) (chat_by )
faraday (0.17.3) != faraday (1.0.0) (manage_by )
parser (2.7.0.3) != parser (2.7.0.4) (manage_by )
pg (1.2.2) != pg (1.2.3) (manage_by )
rspec-expectations (3.9.0) != rspec-expectations (3.9.1) (manage_by )
rspec-rails (4.0.0.beta4) != rspec-rails (4.0.0.rc1) (manage_by )
thor (0.20.3) != thor (1.0.1) (manage_by )
twilio-ruby (5.31.1) != twilio-ruby (5.32.0) (manage_by )
action_policy-graphql (0.3.2) != action_policy-graphql (0.4.0) (core_by )
faraday (0.17.3) != faraday (1.0.1) (core_by )
graphql (1.10.4) != graphql (1.10.5) (core_by )
parser (2.7.0.3) != parser (2.7.0.4) (core_by )
pg (1.2.2) != pg (1.2.3) (core_by )
rspec-expectations (3.9.0) != rspec-expectations (3.9.1) (core_by )
rspec-rails (4.0.0.beta4) != rspec-rails (4.0.0.rc1) (core_by )
thor (0.20.3) != thor (1.0.1) (core_by )
twilio-ruby (5.31.1) != twilio-ruby (5.32.0) (core_by )
action_policy-graphql (0.3.2) != action_policy-graphql (0.4.0) (meet_by )
faraday (0.17.3) != faraday (1.0.1) (meet_by )
graphql (1.10.4) != graphql (1.10.5) (meet_by )
parser (2.7.0.3) != parser (2.7.0.4) (meet_by )
pg (1.2.2) != pg (1.2.3) (meet_by )
rspec-expectations (3.9.0) != rspec-expectations (3.9.1) (meet_by )
rspec-rails (4.0.0.beta4) != rspec-rails (4.0.0.rc1) (meet_by )
thor (0.20.3) != thor (1.0.1) (meet_by )
twilio-ruby (5.31.1) != twilio-ruby (5.32.0) (meet_by )
Slide 52
Slide 52 text
transdeps: cons
• Does not take into account env: it is
impossible to ignore development, test
deps
• Non-configurable: it is impossible to
ignore patch level differences
52
Slide 53
Slide 53 text
Syncing Versions 2.0
53
BUNDLE_GEMFILE=" ../ ../Gemfile" bundle install
BUNDLE_GEMFILE=" ../ ../Gemfile" bundle exec rspec
Use the main app's Gemfile in engines!
Slide 54
Slide 54 text
BUNDLE_GEMFILE: Caveats
• How to define engine's dev and test
deps?
• How to avoid loading everything in
dummy apps (in engine tests)?
54
Slide 55
Slide 55 text
component
55
# /Gemfile
class Bundler ::Dsl
def component(name)
group :default, name.to_sym do
gem name, path: "engines/ #{name}"
eval_gemfile "engines/ #{name}/Gemfile.runtime"
end
group name.to_sym do
dev_deps_from_gemspec("engines/ #{name}.gemspec").each do
gem _1.name, _1.requirement
end
end
end
end
component "connect_by"
Slide 56
Slide 56 text
Bundler.require
56
# /test/dummy/config/application.rb
# ...
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
- Bundler.require(*Rails.groups)
+ Bundler.require(:connect_by)
Slide 57
Slide 57 text
BUNDLE_GEMFILE: Tips
57
bundle config --local gemfile=" ../ ../Gemfile"
Slide 58
Slide 58 text
58
Slide 59
Slide 59 text
DB vs. Engines
• How to manage migrations?
• How to load seeds?
59
Slide 60
Slide 60 text
Migrations. Option #1
Install migrations:
rails my_engine:install:migrations
60
Slide 61
Slide 61 text
Migrations. Option #2
“Mount“ migrations:
61
# engines/my_engine/lib/my_engine/engine.rb
initializer "my_engine.migrations" do |app|
app.config.paths["db/migrate"].concat(
config.paths["db/migrate"].expanded
)
# For checking pending migrations
ActiveRecord ::Migrator.migrations_paths +=
config.paths["db/migrate"].expanded.flatten
end
Slide 62
Slide 62 text
Seeds
62
# /db/seed.rb
ActiveRecord ::Base.transaction do
ConnectBy ::Engine.load_seed
PerksBy ::Engine.load_seed
ChatBy ::Engine.load_seed
MeetBy ::Engine.load_seed
end
Slide 63
Slide 63 text
How to test
engines?
63
Slide 64
Slide 64 text
Testing. Option #1
Using a full-featured Dummy app:
64
spec/
dummy/
app/
controllers/
…
config/
db/
test/
…
Slide 65
Slide 65 text
Testing. Option #2
65
gem "combustion"
A library to help you test your Rails Engines in
a simple and effective manner, instead of
creating a full Rails application in your spec or
test folder.
Slide 66
Slide 66 text
Combustion
66
Combustion.initialize! :active_record, :active_job do
config.logger = Logger.new(nil)
config.log_level = :fatal
config.autoloader = :zeitwerk
config.active_storage.service = :test
config.active_job.queue_adapter = :test
end
Slide 67
Slide 67 text
Combustion
67
spec/
internal/
config/
storage.yml
db/
schema.rb
Slide 68
Slide 68 text
Combustion
• Load only required Rails frameworks
• No boilerplate, only the files you need
• Automatically re-runs migrations for
every test run
68
Slide 69
Slide 69 text
Engines on CI
• Run tests separately for each engine
• Run tests only for dirty engines 㱺
decease build times!
69
Slide 70
Slide 70 text
pattern = File.join( __dir __, " ../{engines,gems}/*/*.gemspec")
gemspecs = Dir.glob(pattern).map(Gem ::Specification.:load)
# Parses git diff $(git merge-base origin/master HEAD) --name-only
dirty = fetch_dirty_libraries
index = build_inverted_index(gemspecs)
lib_name = ARGV[0]
if (index[lib_name] & dirty).empty?
$stdout.puts "[Dirty Check] No changes for #{lib_name}. Skip"
exit(0)
else
exit(1
end
.ci/is-dirty
70
Slide 71
Slide 71 text
CI
71
.ci/is-dirty connect_by || \
(cd engines/connect_by && bundle exec rspec)
Slide 72
Slide 72 text
72
Slide 73
Slide 73 text
Base & Behaviour
• Base Rails classes within an engine
MUST be configurable
• They also MAY require to have a certain
“interface”
73
Slide 74
Slide 74 text
Base & Behaviour
74
module ConnectBy
class ApplicationController < Engine.config.application_controller.constantize
raise "Must include ConnectBy ::ControllerBehaviour" unless
self < ConnectBy ::ControllerBehaviour
raise "Must implement #current_user method" unless
instance_methods.include?(:current_user)
end
end
Slide 75
Slide 75 text
Base & Behaviour
75
module ConnectBy
class ApplicationController < Engine.config.application_controller.constantize
raise "Must include ConnectBy ::ControllerBehaviour" unless
self < ConnectBy ::ControllerBehaviour
raise "Must implement #current_user method" unless
instance_methods.include?(:current_user)
end
end
Configurable
Slide 76
Slide 76 text
Base & Behaviour
76
# config/application.rb
config.connect_by.application_controller =
"MyApplicationController"
Slide 77
Slide 77 text
Base & Behaviour
77
module ConnectBy
class ApplicationController < Engine.config.application_controller.constantize
raise "Must include ConnectBy ::ControllerBehaviour" unless
self < ConnectBy ::ControllerBehaviour
raise "Must implement #current_user method" unless
instance_methods.include?(:current_user)
end
end
“Interface”
Slide 78
Slide 78 text
class ApplicationController < ActionController ::API
include ConnectBy ::ControllerBehaviour
include JWTAuth
include RavenContext if defined?(Raven)
end
Base & Behaviour
78
Slide 79
Slide 79 text
Modify Models
• Prefer extensions over patches
• Use load hooks (to support autoload/
reload)
79
Slide 80
Slide 80 text
Extension
80
# engines/meet_by/app/models/ext/connect_by/city.rb
module MeetBy
module Ext
module ConnectBy
module City
extend ActiveSupport ::Concern
included do
has_many :events, class_name: "MeetBy ::Events ::Member",
inverse_of: :user,
foreign_key: :user_id,
dependent: :destroy
end
end
end
end
end
Slide 81
Slide 81 text
Load Hooks
81
# engines/connect_by/app/models/connect_by/city.rb
module ConnectBy
class City < ActiveRecord ::Base
# ...
ActiveSupport.run_load_hooks(
"connect_by/city",
self
)
end
end
Slide 82
Slide 82 text
Load Hooks
82
# engines/meet_by/lib/meet_by/engine.rb
initializer "meet_by.extensions" do
ActiveSupport.on_load("connect_by/city") do
include ::MeetBy ::Ext ::ConnectBy ::City
end
end
Slide 83
Slide 83 text
How to communicate
between engines?
83
Slide 84
Slide 84 text
84
Every time new users register in the app
we want them to automatically join the
default city chat
Example
Slide 85
Slide 85 text
Problem
85
connect_by
chat_by
depends on
User Registration
???
Slide 86
Slide 86 text
Solution
• Events
• Events
• Events
86
Slide 87
Slide 87 text
Tools
• Hanami Events
• Rails Events Store
• Wisper (?)
• dry-events (?)
87
Slide 88
Slide 88 text
88
railseventstore.org
Slide 89
Slide 89 text
89
gem "active_event_store"
Wrapper over Rails Event Store with
conventions and transparent Rails integration
Slide 90
Slide 90 text
AES vs. RES
• Class-independent event types
• Uses RES as interchangeable adapter
• Better testing tools
• Less verbose API and convention over
configuration
90
Slide 91
Slide 91 text
Active Event Store
91
class ProfileCompleted < ActiveEventStore ::Event
# (optional) event identifier is used for transmitting events
# to subscribers.
#
# By default, identifier is equal to `name.underscore.gsub('/', '.')`.
self.identifier = "profile_completed"
# Add attributes accessors
attributes :user_id
# Sync attributes only available for sync subscribers
sync_attributes :user
end
Slide 92
Slide 92 text
92
event = ProfileCompleted.new(user_id: user.id)
# or with metadata
event = ProfileCompleted.new(
user_id: user.id,
metadata: { ip: request.remote_ip }
)
# then publish the event
ActiveEventStore.publish(event)
Active Event Store
Slide 93
Slide 93 text
93
initializer "my_engine.subscribe_to_events" do
ActiveSupport.on_load :active_event_store do |store|
# async subscriber is invoked from background job,
# enqueued after the current transaction commits
store.subscribe MyEventHandler, to: ProfileCreated
# anonymous handler (could only be synchronous)
store.subscribe(to: ProfileCreated, sync: true) do |event|
# do something
end
# subscribes to ProfileCreated automatically
store.subscribe OnProfileCreated ::DoThat
end
end Convention over
configuration
Active Event Store
Slide 94
Slide 94 text
engines/
connect_by/
app/
events/
connect_by/
users/
registered.rb
chat_by/
app/
subscribers/
connect_by/
users/
on_registered/
create_chat_account.rb
94
Active Event Store
Slide 95
Slide 95 text
What should we leave
in the main app?
95
Slide 96
Slide 96 text
Extreme Case
96
app/
bin/
config/
db/
engines/
gems/
lib/
spec/
Slide 97
Slide 97 text
Common Case
97
app/
controllers/
application_controller.rb
sessions_controller.rb
bin/
config/
db/
engines/
gems/
lib/
spec/
Slide 98
Slide 98 text
Main App
• Feature/system tests
• Locales and templates overrides
• Instrumentation and exception
handling
• Configuration
98
Slide 99
Slide 99 text
Gems
Or stop putting everything into
lib/ext folder
99
Slide 100
Slide 100 text
Gems
• Organized lib/ext
• Shared non-application specific code
• Isolated tests
• Ability to share between applications in the future
(e.g., GitHub Package Registry)
• Ability to open-source in the future
100
Slide 101
Slide 101 text
Gems
101
gems/
common-rubocop/
common-testing/
common-graphql/
…
Slide 102
Slide 102 text
Why
engines?
And why not
102
Slide 103
Slide 103 text
Why engines?
• Modular monolith instead of monolith
monsters or distributed monoliths
• Code (and tests) isolation
• Loose coupling (no more spaghetti
logic)
103
Slide 104
Slide 104 text
Why engines?
• Easy to upgrade frameworks (e.g.,
Rails) component by component
• Easy to re-use logic in another app
• …and we can migrate to microservices
in the future (if we’re brave enough)
104
Slide 105
Slide 105 text
Why not engines?
• Not so good third-party gems support
• Not so good support within Rails itself
105
Slide 106
Slide 106 text
github.com/palkan/engems
106
Rails component-based architecture on top of
engines and gems showroom
Slide 107
Slide 107 text
Thank you!
Vladimir Dementyev
Evil Martians
@palkan
@palkan_tula
evilmartians.com
@evilmartians