About Me ● Writer, programmer, translator ● @shioyama on github, twitter, etc ● I program in Ruby ● I live in Tokyo ● I am the author of Mobility ● I blog at dejimata.com ● I coined “Module Builder Pattern”
One Problem, Many Solutions ● How many languages? ● How many translated models? ● Growing application? Adding new models? ● What’s your database? Are your translations in your database? ● What are your performance requirements? ● How good are you at Arel?
Plan for Today’s Talk 1. Translation Interface – How do you read and write translations? – What other features are supported? 2. Storage Strategies – How are translations actually stored? – Good and bad of each strategy 3. Mobility – Translated Attributes, Backends and Plugins 4. Looking Ahead – Mobility as Translation Layer
Common Features ● Translations are fetched and stored using (virtual) attribute accessors ● Language fetched or stored is the value of the global locale (I18n.locale or similar) ● Models can be queried using translated attributes (like normal column-based attributes)
One Attribute, Many Languages class Post < ApplicationRecord translates :title end post = Post.new I18n.locale = :en post.title = "Translating with Mobility" I18n.locale = :fr post.title = "Traduction avec Mobilité" post.save post = Post.first I18n.locale = :en post.title #=> "Translating with Mobility" I18n.locale = :fr post.title #=> "Traduction avec Mobilité" class Post < ApplicationRecord translates :title end post = Post.new I18n.locale = :en post.title = "Translating with Mobility" I18n.locale = :fr post.title = "Traduction avec Mobilité" post.save post = Post.first I18n.locale = :en post.title #=> "Translating with Mobility" I18n.locale = :fr post.title #=> "Traduction avec Mobilité"
Bonus Points: Query by Translation I18n.locale = :en Post.find_by(title: "Translating with Mobility") #=> (post with English title "Translating...") I18n.locale = :fr Post.find_by(title: "Traduction avec Mobilité") #=> (returns post with French title "Traduction...") I18n.locale = :en Post.find_by(title: "Translating with Mobility") #=> (post with English title "Translating...") I18n.locale = :fr Post.find_by(title: "Traduction avec Mobilité") #=> (returns post with French title "Traduction...") ● Override query methods to return records by translated attribute ● Hides details of storage strategy from query interface
What is a Storage Strategy? translated attribute translated attribute Persistence Layer describes a mapping between translated attributes and persistence layer
Translatable Columns ● Good – Easy to understand – Easy to construct queries – Good performance (no joins) ● Bad – Every new language requires a migration for every translated model – Columns created for every locale even if there is no translation for a locale
Model Translation Tables ● Translation table for each model, with: – locale column – foreign key pointing to model table (post_id) – one or more translated columns ● has_many association between model and translations ● Translation table records are created/updated when attribute is set and model is saved has_many :translations, class_name: PostTranslation has_many :translations, class_name: PostTranslation
Model Translation Tables ● Good – Easily scales with number of languages (no new migrations to add locale) – Space usage is more efficient (only create record if translation is stored) ● Bad – Querying may be less performant (need to join translation table) – Code is more complex due to joining – Migration to create translation table/column for every model/attribute
Shared Translation Tables ● One or more translation tables, with: – locale column – foreign key pointing to model id (translatable_id) – column holding the model class (translatable_type) – column holding the name of the attribute (key) – column holding the value of the attribute (value) ● One translation table for each value type (string, text, ...) has_many :translations, as: :translatable, class_name: ::Mobility::ActiveRecord::StringTranslation has_many :translations, as: :translatable, class_name: ::Mobility::ActiveRecord::StringTranslation
Shared Translation Tables ● Good – No additional migrations at all, so easy to scale to new locales + new models – Space usage is relatively efficient, scales with number of locales + models ● Bad – Very complex querying (know your Arel?) – All translations in the same table, can be difficult to maintain a consistent database state
Serialized Translations ● Store translations as serialized hash on a single database column class Post < ApplicationRecord serialize :title_translations, Hash end post = Post.new post.title_translations = { "en" => "Translating with Mobility" } post.save class Post < ApplicationRecord serialize :title_translations, Hash end post = Post.new post.title_translations = { "en" => "Translating with Mobility" } post.save
Serialized Translations ● Good – Scales well with locales (no additional migrations) – Easy to add translations to model (simply add text column for each translated attribute) – Simple to access ● Bad – Querying is not possible – Loading one translation requires loading all translations at once (serialized column)
Hstore/Jsonb (PostgreSQL) ● Similar to serialized strategy, except that querying is possible and can be fast ● Database-dependent strategy, highly coupled to persistence layer ● (Jsonb generally preferred over Hstore) Post.where(title: "foo").to_sql #=> SELECT "posts".* FROM "posts" WHERE ("posts"."title" @> '{"en":"foo"}') Post.where(title: "foo").to_sql #=> SELECT "posts".* FROM "posts" WHERE ("posts"."title" @> '{"en":"foo"}')
Other Possibilities ● Stored in external service, accessed via API (Contentful, Firebase, etc) ● File storage ● NoSQL database ● Cache storage (Redis/Memcache/etc) ● New forms of storage we cannot yet imagine
Translating with Mobility ● Existing solutions – couple implementation to storage strategy – couple implementation to ORM (ActiveRecord) and framework (Rails) – have similar but not identical interfaces ● What I wanted: – Support customizable, swappable storage solutions – No hard dependency on ActiveRecord or Rails – Interface that is consistent and unified, but flexible
A Translation Framework ● New approach to translation problem: – One translation interface, swappable storage backends – Supports all existing strategies, plus more – Supports both ActiveRecord and Sequel ORM ● Like the OmniAuth of Translation
Getting Started rails g mobility:install rails g mobility:install class Post < ApplicationRecord include Mobility translates :title, :body end class Post < ApplicationRecord include Mobility translates :title, :body end Mobility.configure do |config| config.default_backend = :key_value config.accessor_method = :translates config.query_method = :i18n end Mobility.configure do |config| config.default_backend = :key_value config.accessor_method = :translates config.query_method = :i18n end :key_value :table :column :jsonb :hstore :serialized rake db:migrate rake db:migrate
A Backend is a Ruby Class class MyBackend include Mobility::Backend # attr_reader :attribute, :model def read(locale, options = {}) # Read attribute in locale. end def write(locale, value, options = {}) # Write value to attribute in locale. end def self.configure(options) # Configure/normalize options (e.g. set any defaults) end setup do |attributes, options| # Do something with attributes and options in context of model class. end end class MyBackend include Mobility::Backend # attr_reader :attribute, :model def read(locale, options = {}) # Read attribute in locale. end def write(locale, value, options = {}) # Write value to attribute in locale. end def self.configure(options) # Configure/normalize options (e.g. set any defaults) end setup do |attributes, options| # Do something with attributes and options in context of model class. end end
Attributes as a Module class Post translates :title, backend: :key_value end class Post translates :title, backend: :key_value end class Post include Mobility::Attributes.new(:accessor, [:title], backend: :key_value) end class Post include Mobility::Attributes.new(:accessor, [:title], backend: :key_value) end It’s a Module Builder!
Elements of a Module Builder class Attributes < Module def initialize(method, *attributes) @attributes = attributes # ... end def included(klass) @attributes.each do |attribute| define_method attribute do # ... end # ... end end end class Post include Attributes.new(:accessor, [:title]) end class Attributes < Module def initialize(method, *attributes) @attributes = attributes # ... end def included(klass) @attributes.each do |attribute| define_method attribute do # ... end # ... end end end class Post include Attributes.new(:accessor, [:title]) end Subclass Module Define module methods in an included hook Include instance of Module subclass in other classes
Fallbacks as a Plugin module Mobility module Fallbacks # ... def read(locale, options = {}) fallback = options.delete(:fallback) return super if fallback == false || (fallback.nil? && fallbacks.nil?) (fallback ? [locale, *fallback] : fallbacks[locale]).detect do |fallback_locale| value = super(fallback_locale, options) break value if value.present? end end end end module Mobility module Fallbacks # ... def read(locale, options = {}) fallback = options.delete(:fallback) return super if fallback == false || (fallback.nil? && fallbacks.nil?) (fallback ? [locale, *fallback] : fallbacks[locale]).detect do |fallback_locale| value = super(fallback_locale, options) break value if value.present? end end end end ● Plugins applied to backend (fallbacks, cache, dirty) override read and/or write methods to add functionality Find first non-nil value of attribute
Recap (Sales Pitch) ● Mobility: – supports every storage strategy, each in its own class – supports multiple strategies in a single model – supports all common features, at the attribute level ● Mobility can be: – as simple as you want it to be – as complex as you need it to be
Mobility as a Translation Layer ● For application developers: – simplifies and standardizes content translation – does not lock you into a single storage strategy ● For developers of other gems: – A single Mobility integration and will support all current and future storage strategies – Example: friendly_id-mobility
shioyama
whywaita
autifyhq
unclejoe
kohbis
.jpg){kind=avatar}
izzii
linedevth
hiashisan
pingcap0315
oracle4engineer
miyashino
non97
mkwrd
lynnandtonic
chrisshort
vanstee
addyosmani
bluesmoon
marktimemedia
lauravandoore
mojombo
reverentgeek
jensimmons
akmur
zakiyama