Slide 1

Slide 1 text

Data Mapping The Rails Way Дмитрий Цепелев, Злые марсиане фото: Bryan Birdwell, https://bit.ly/2Tpy3mU

Slide 2

Slide 2 text

DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 2

Slide 3

Slide 3 text

DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 3 evilmartians.com

Slide 4

Slide 4 text

DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 4 evilmartians.com

Slide 5

Slide 5 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 5 Data mapping — процесс преобразования данных от одного формата к другому

Slide 6

Slide 6 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 6 Преобразование типов SELECT "created_at" FROM "users" LIMIT 1 ┌──────────────────────────┐ │ created_at │ ├──────────────────────────┤ │ 2019-11-22 09:29:17.5634 │ └──────────────────────────┘ User.first.created_at # "=> Fri, 22 Nov 2019 12:29:17 MSK +03:00

Slide 7

Slide 7 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 7 Построение запросов # date Order.where("created_at ">= ?", Date.today.beginning_of_day) #"=> select * from orders where created_at ">= '20-02-06' # enum Order.where(status: :complete) #"=> select * from orders where status = 3

Slide 8

Slide 8 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели 8

Slide 9

Slide 9 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 9 Атрибуты модели User.attribute_types # "=> {"id""=>#, # "created_at""=>#, # "updated_at""=>#, # ""... # }

Slide 10

Slide 10 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 10 Автоматическое определение атрибутов module ActiveRecord module ModelSchema def load_schema! @columns_hash = connection.schema_cache .columns_hash(table_name) .except(*ignored_columns) @columns_hash.each do |name, column| define_attribute( name, connection.lookup_cast_type_from_column(column) ) end end end end

Slide 11

Slide 11 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 11 ignored_columns class Post < ActiveRecord"::Base self.ignored_columns = %w(meta) end

Slide 12

Slide 12 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 12 Определение атрибутов вручную class Configuration include ActiveModel"::Attributes attribute :model, :string attribute :color, :string, default: "red" end config = Configuration.new config.model = 42 puts config.model.inspect # "=> "42" puts config.color.inspect # "=> "red"

Slide 13

Slide 13 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 13 Что будет? class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list "== [2] # "=> false

Slide 14

Slide 14 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 14 Будет ерунда class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [1, 2]

Slide 15

Slide 15 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 15 Нужно использовать lambda class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: "-> { [] } end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [2]

Slide 16

Slide 16 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 16 ActiveModel"::Type"::BigInteger ActiveModel"::Type"::Binary ActiveModel"::Type"::Boolean ActiveModel"::Type"::DateTime ActiveModel"::Type"::Date ActiveModel"::Type"::Decimal ActiveModel"::Type"::"Float ActiveModel"::Type"::ImmutableString ActiveModel"::Type"::Integer ActiveModel"::Type"::String ActiveModel"::Type"::Time Типы «из коробки» ActiveRecord"::Type"::DateTime ActiveRecord"::Type"::Date ActiveRecord"::Type"::DecimalWithoutScale ActiveRecord"::Type"::Json ActiveRecord"::Type"::Time ActiveRecord"::Type"::UnsignedInteger

Slide 17

Slide 17 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 17 class ActiveModel"::Type"::Value # DB "-> Ruby def deserialize(value) cast(value) end # User "-> Ruby def cast(value) cast_value(value) unless value.nil? end # Ruby "-> DB def serialize(value) value end end Что внутри?

Slide 18

Slide 18 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 18 Реестр типов ActiveModel::Type::Registry или наследник в ActiveRecord используется AdapterSpecificRegistry умеет находить класс типа по имени ActiveRecord"::Type.registry.lookup(:integer) # "=> #

Slide 19

Slide 19 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 19 Создание собственного типа class MoneyType < ActiveRecord"::Type"::Integer def cast(value) if !value.kind_of?(Numeric) "&& value.include?('$') price_in_dollars = value.gsub(/\$/, '').to_f super(price_in_dollars * 70) else super end end end

Slide 20

Slide 20 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 20 Использование собственного типа ActiveRecord"::Type.register( :money, MoneyType ) class User < ApplicationRecord attribute :balance, :money end User.new(balance: '$1').balance # "=> 70 class User < ApplicationRecord attribute :balance, MoneyType.new end User.new(balance: '$1').balance # "=> 70 через реестр напрямую

Slide 21

Slide 21 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 21 ⚠ Type cast происходит при чтении ⚠ class IntegerRangeType < ActiveRecord"::Type"::Integer def initialize(range) @range = range end def cast(value) raise ArgumentError unless range.cover?(value) super end end class User include ActiveModel"::Attributes attribute :age, IntegerRangeType.new(0"..100) end user = User.new user.age = 120 puts user.age # "=> ArgumentError

Slide 22

Slide 22 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Зачем разработчику знать про Attributes API 22 Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели

Slide 23

Slide 23 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 23 Виртуальные атрибуты class User < ApplicationRecord attribute :skip_confirmation, :boolean, default: false validate :confirmation, unless: :skip_confirmation? end user = User.new(skip_confirmation: '0')

Slide 24

Slide 24 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 24 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 0 0 0

Slide 25

Slide 25 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 25 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) 0 0 0 0 1 0 OR

Slide 26

Slide 26 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 26 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" 0 1 0

Slide 27

Slide 27 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 27 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 0 1 0 0 1 0 AND

Slide 28

Slide 28 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 28 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true 0 1 0 0 1 0 AND

Slide 29

Slide 29 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 29 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 0 1 0 1 0 0 AND

Slide 30

Slide 30 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 30 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false 0 1 0 1 0 0 AND

Slide 31

Slide 31 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 31 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) 0 1 0 1 0 0 OR

Slide 32

Slide 32 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 32 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) # "=> "110" settings & (1 "<< PUSH) > 0 # "=> true 1 1 0

Slide 33

Slide 33 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 33 Value Object class NotificationSettings < ActiveRecord"::Type"::Value IN_APP = 0 EMAIL = 1 PUSH = 2 … end

Slide 34

Slide 34 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 34 Value Object class NotificationSettings < ActiveRecord"::Type"::Value … def cast(value) Value.new(value) end end

Slide 35

Slide 35 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 35 Value Object class NotificationSettings < ActiveRecord"::Type"::Value … class Value attr_reader :settings def initialize(settings) @settings = settings end def subscribe(subscription) @settings "|= (1 "<< subscription) end def receives_email? @settings & (1 "<< EMAIL) > 0 end end end

Slide 36

Slide 36 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 36 Value Object class User < ActiveRecord+::Base attribute :notification_settings, NotificationSettings.new, default: 0 end user = User.new puts user.notification_settings.receives_email? # "=> false user.notification_settings.subscribe(NotificationSettings"::EMAIL) puts user.notification_settings.receives_email? # "=> true

Slide 37

Slide 37 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 37 Работа с legacy колонками class Post < ActiveRecord"::Base attribute :published_at, StringDateType.new end Post.insert_all([{ published_at: "2012-10-17 14:35" }]) puts Post.last.published_at.inspect # "=> 2012-10-17 14:35:00 +0400 Post.create(published_at: Time.now) puts Post.last.published_at # "=> 2020-02-12 20:15:00 +0300

Slide 38

Slide 38 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 38 Работа с legacy колонками class StringDateType < ActiveRecord"::Type"::Value LEGACY_FORMAT = "%Y-%m-%d %H:%M" def serialize(value) value.strftime(LEGACY_FORMAT) end def deserialize(value) Time.strptime(value, LEGACY_FORMAT) end end

Slide 39

Slide 39 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 39 Работа с JSON колонками class Post < ActiveRecord"::Base attribute :meta, PostMetaType.new end post = Post.new(meta: { published_at: 1.week.ago }) puts post.meta.published_at # "=> 2020-02-05 post.meta.published_at = Time.now post.save puts post.meta.published_at # "=> 2020-02-12

Slide 40

Slide 40 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 40 Работа с JSON колонками class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end

Slide 41

Slide 41 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 41 Вызываем конструктор 50000 раз: Бенчмарк по памяти struct dry-struct dry-initializer ActiveModel virtus 0 225000 450000 675000 900000 memsize objects strings https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda

Slide 42

Slide 42 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 42 Вызываем конструктор 50000 раз: Бенчмарк по производительности struct dry-initializer dry-struct ActiveModel virtus 0 1500000 3000000 4500000 6000000 ips https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda

Slide 43

Slide 43 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 43 Но я хочу использовать PORO!

Slide 44

Slide 44 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 44 class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end

Slide 45

Slide 45 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 45 Attributes API + struct = ♥ class PostMetaType < ActiveRecord"::Type"::Value Meta = Struct.new(:published_at) do def initialize(published_at=nil) self.published_at = if published_at.nil? "|| published_at.is_a?(Date) published_at else Date.parse(published_at) end end end def serialize(value) ActiveSupport"::JSON.encode(value) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys Meta.new(*Meta.members.map { |name| decoded[name] }) end end

Slide 46

Slide 46 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 46 Attributes API + dry-struct = ♥ class PostMetaType < ActiveRecord"::Type"::Value class Meta < Dry"::Struct attribute :published_at, Dry.Types"::JSON"::Date.optional end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys values = Meta.attribute_names.each_with_object({}) { |a, h| h[a] = decoded[a] } Meta.new(values) end end

Slide 47

Slide 47 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev JSON–колонки как модели 47 Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели

Slide 48

Slide 48 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 48 Типы для работы с JSON похожи class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end class UserSyncType < ActiveRecord"::Type"::Value class Sync include ActiveModel"::Model include ActiveModel"::Attributes attribute :last_sync_at, :datetime attribute :success, :boolean end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Sync.new(decoded) end end

Slide 49

Slide 49 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 49 Очень много скобок! product = Product.find(params[:id]) if product.configuration[:model] "== "spaceship" product.configuration[:color] = "red" end product.save

Slide 50

Slide 50 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 50 product = Product.find(params[:id]) if product.configuration.model "== "spaceship" product.configuration.color = "red" end product.save Очень много скобок!

Slide 51

Slide 51 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 51 JSON нужно валидировать product = Product.new puts product.valid? # "=> false puts product.errors.messages # "=> { configuration: ["is invalid"] } puts product.configuration.errors.messages # "=> { color: ["can't be blank"] }

Slide 52

Slide 52 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 52 store_model https:"//github.com/DmitryTsepelev/store_model class Configuration include StoreModel"::Model attribute :model, :string enum :status, %i[active archived], default: :active validates :status, presence: true end class Product < ApplicationRecord attribute :configuration, Configuration.to_type validates :configuration, store_model: true end

Slide 53

Slide 53 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 53 store_model: модель https:"//github.com/DmitryTsepelev/store_model module StoreModel module Model def self.included(base) base.include ActiveModel"::Model base.include ActiveModel"::Attributes base.include StoreModel"::NestedAttributes base.extend StoreModel"::Enum base.extend StoreModel"::TypeBuilders end end end

Slide 54

Slide 54 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 54 store_model: генерация типов https:"//github.com/DmitryTsepelev/store_model module StoreModel module TypeBuilders def to_type Types"::JsonType.new(self) end end end

Slide 55

Slide 55 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 55 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value def initialize(model_klass) @model_klass = model_klass end … end https:"//github.com/DmitryTsepelev/store_model

Slide 56

Slide 56 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 56 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value def cast_value(value) case value when String decoded = ActiveSupport"::JSON.decode(value) rescue nil @model_klass.new(decoded) unless decoded.nil? when Hash @model_klass.new(value) when @model_klass, nil value else raise_cast_error(value) end end … end https:"//github.com/DmitryTsepelev/store_model

Slide 57

Slide 57 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 57 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value … def serialize(value) case value when Hash, @model_klass then ActiveSupport"::JSON.encode(value) else super end end end https:"//github.com/DmitryTsepelev/store_model

Slide 58

Slide 58 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 58 валидации enums атрибуты–списки вложенные модели обработка необъявленных полей Что еще можно сделать? https:"//github.com/DmitryTsepelev/store_model

Slide 59

Slide 59 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 59 https:"//bit.ly/3amqMeL

Slide 60

Slide 60 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 60 nested_record https:"//github.com/marshall-lee/nested_record class User < ActiveRecord"::Base include NestedRecord"::Macro has_one_nested :profile end class Profile < NestedRecord"::Base attribute :age, :integer attribute :active, :boolean has_one_nested :contacts end class Profile"::Contacts < NestedRecord"::Base attribute :email, :string attribute :phone, :string end user.profile.age = 39 user.profile.contacts.email = '[email protected]'

Slide 61

Slide 61 text

RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 61 вложенность дает накладные расходы возможно стоит исключить JSON из выборки по умолчанию ⚠ JSON — не серебряная пуля ⚠

Slide 62

Slide 62 text

evl.ms/blog @dmitrytsepelev DmitryTsepelev @evilmartians evl.ms/telegram THANK YOU! RUBY DAY 2020 62