$30 off During Our Annual Pro Sale. View Details »

Data Mapping: the Rails Way

Data Mapping: the Rails Way

1. Data Mapping в Rails
2. Зачем разработчику знать про Attributes API
3. JSON–колонки как модели

Dmitry Tsepelev

February 29, 2020
Tweet

More Decks by Dmitry Tsepelev

Other Decks in Programming

Transcript

  1. Data Mapping The Rails Way Дмитрий Цепелев, Злые марсиане фото:

    Bryan Birdwell, https://bit.ly/2Tpy3mU
  2. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 2

  3. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 3 evilmartians.com

  4. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 4 evilmartians.com

  5. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 5 Data mapping — процесс

    преобразования данных от одного формата к другому
  6. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 6 Преобразование типов SELECT "created_at"

    FROM "users" LIMIT 1 ┌──────────────────────────┐ │ created_at │ ├──────────────────────────┤ │ 2019-11-22 09:29:17.5634 │ └──────────────────────────┘ User.first.created_at # "=> Fri, 22 Nov 2019 12:29:17 MSK +03:00
  7. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 7 Построение запросов # date

    Order.where("created_at ">= ?", Date.today.beginning_of_day) #"=> select * from orders where created_at ">= '20-02-06' # enum Order.where(status: :complete) #"=> select * from orders where status = 3
  8. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Data Mapping в Rails Зачем

    разработчику знать про Attributes API JSON–колонки как модели 8
  9. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 9 Атрибуты модели User.attribute_types #

    "=> {"id""=>#<ActiveModel"::Type"::Integer ""...>, # "created_at""=>#<ActiveRecord"::ConnectionAdapters"::PostgreSQL"::OID"::DateTime …>, # "updated_at""=>#<ActiveRecord"::ConnectionAdapters"::PostgreSQL"::OID"::DateTime …>, # ""... # }
  10. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 10 Автоматическое определение атрибутов module

    ActiveRecord module ModelSchema def load_schema! @columns_hash = connection.schema_cache .columns_hash(table_name) .except(*ignored_columns) @columns_hash.each do |name, column| define_attribute( name, connection.lookup_cast_type_from_column(column) ) end end end end
  11. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 11 ignored_columns class Post <

    ActiveRecord"::Base self.ignored_columns = %w(meta) end
  12. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 12 Определение атрибутов вручную class

    Configuration include ActiveModel"::Attributes attribute :model, :string attribute :color, :string, default: "red" end config = Configuration.new config.model = 42 puts config.model.inspect # "=> "42" puts config.color.inspect # "=> "red"
  13. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 13 Что будет? class TestModel

    include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list "== [2] # "=> false
  14. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 14 Будет ерунда class TestModel

    include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [1, 2]
  15. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 15 Нужно использовать lambda class

    TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: "-> { [] } end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [2]
  16. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 16 ActiveModel"::Type"::BigInteger ActiveModel"::Type"::Binary ActiveModel"::Type"::Boolean ActiveModel"::Type"::DateTime

    ActiveModel"::Type"::Date ActiveModel"::Type"::Decimal ActiveModel"::Type"::"Float ActiveModel"::Type"::ImmutableString ActiveModel"::Type"::Integer ActiveModel"::Type"::String ActiveModel"::Type"::Time Типы «из коробки» ActiveRecord"::Type"::DateTime ActiveRecord"::Type"::Date ActiveRecord"::Type"::DecimalWithoutScale ActiveRecord"::Type"::Json ActiveRecord"::Type"::Time ActiveRecord"::Type"::UnsignedInteger
  17. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 17 class ActiveModel"::Type"::Value # DB

    "-> Ruby def deserialize(value) cast(value) end # User "-> Ruby def cast(value) cast_value(value) unless value.nil? end # Ruby "-> DB def serialize(value) value end end Что внутри?
  18. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 18 Реестр типов ActiveModel::Type::Registry или

    наследник в ActiveRecord используется AdapterSpecificRegistry умеет находить класс типа по имени ActiveRecord"::Type.registry.lookup(:integer) # "=> #<ActiveModel"::Type"::Integer: …>
  19. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 19 Создание собственного типа class

    MoneyType < ActiveRecord"::Type"::Integer def cast(value) if !value.kind_of?(Numeric) "&& value.include?('$') price_in_dollars = value.gsub(/\$/, '').to_f super(price_in_dollars * 70) else super end end end
  20. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 20 Использование собственного типа ActiveRecord"::Type.register(

    :money, MoneyType ) class User < ApplicationRecord attribute :balance, :money end User.new(balance: '$1').balance # "=> 70 class User < ApplicationRecord attribute :balance, MoneyType.new end User.new(balance: '$1').balance # "=> 70 через реестр напрямую
  21. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 21 ⚠ Type cast происходит

    при чтении ⚠ class IntegerRangeType < ActiveRecord"::Type"::Integer def initialize(range) @range = range end def cast(value) raise ArgumentError unless range.cover?(value) super end end class User include ActiveModel"::Attributes attribute :age, IntegerRangeType.new(0"..100) end user = User.new user.age = 120 puts user.age # "=> ArgumentError
  22. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Зачем разработчику знать про Attributes

    API 22 Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели
  23. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 23 Виртуальные атрибуты class User

    < ApplicationRecord attribute :skip_confirmation, :boolean, default: false validate :confirmation, unless: :skip_confirmation? end user = User.new(skip_confirmation: '0')
  24. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 24 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 0 0 0
  25. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 25 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) 0 0 0 0 1 0 OR
  26. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 26 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" 0 1 0
  27. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 27 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 0 1 0 0 1 0 AND
  28. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 28 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true 0 1 0 0 1 0 AND
  29. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 29 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 0 1 0 1 0 0 AND
  30. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 30 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false 0 1 0 1 0 0 AND
  31. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 31 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) 0 1 0 1 0 0 OR
  32. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 32 Битовые операции IN_APP =

    0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) # "=> "110" settings & (1 "<< PUSH) > 0 # "=> true 1 1 0
  33. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 33 Value Object class NotificationSettings

    < ActiveRecord"::Type"::Value IN_APP = 0 EMAIL = 1 PUSH = 2 … end
  34. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 34 Value Object class NotificationSettings

    < ActiveRecord"::Type"::Value … def cast(value) Value.new(value) end end
  35. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 35 Value Object class NotificationSettings

    < ActiveRecord"::Type"::Value … class Value attr_reader :settings def initialize(settings) @settings = settings end def subscribe(subscription) @settings "|= (1 "<< subscription) end def receives_email? @settings & (1 "<< EMAIL) > 0 end end end
  36. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 36 Value Object class User

    < ActiveRecord+::Base attribute :notification_settings, NotificationSettings.new, default: 0 end user = User.new puts user.notification_settings.receives_email? # "=> false user.notification_settings.subscribe(NotificationSettings"::EMAIL) puts user.notification_settings.receives_email? # "=> true
  37. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 37 Работа с legacy колонками

    class Post < ActiveRecord"::Base attribute :published_at, StringDateType.new end Post.insert_all([{ published_at: "2012-10-17 14:35" }]) puts Post.last.published_at.inspect # "=> 2012-10-17 14:35:00 +0400 Post.create(published_at: Time.now) puts Post.last.published_at # "=> 2020-02-12 20:15:00 +0300
  38. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 38 Работа с legacy колонками

    class StringDateType < ActiveRecord"::Type"::Value LEGACY_FORMAT = "%Y-%m-%d %H:%M" def serialize(value) value.strftime(LEGACY_FORMAT) end def deserialize(value) Time.strptime(value, LEGACY_FORMAT) end end
  39. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 39 Работа с JSON колонками

    class Post < ActiveRecord"::Base attribute :meta, PostMetaType.new end post = Post.new(meta: { published_at: 1.week.ago }) puts post.meta.published_at # "=> 2020-02-05 post.meta.published_at = Time.now post.save puts post.meta.published_at # "=> 2020-02-12
  40. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 40 Работа с JSON колонками

    class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end
  41. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 41 Вызываем конструктор 50000 раз:

    Бенчмарк по памяти struct dry-struct dry-initializer ActiveModel virtus 0 225000 450000 675000 900000 memsize objects strings https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda
  42. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 42 Вызываем конструктор 50000 раз:

    Бенчмарк по производительности struct dry-initializer dry-struct ActiveModel virtus 0 1500000 3000000 4500000 6000000 ips https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda
  43. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 43 Но я хочу использовать

    PORO!
  44. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 44 class PostMetaType < ActiveRecord"::Type"::Value

    class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end
  45. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 45 Attributes API + struct

    = ♥ class PostMetaType < ActiveRecord"::Type"::Value Meta = Struct.new(:published_at) do def initialize(published_at=nil) self.published_at = if published_at.nil? "|| published_at.is_a?(Date) published_at else Date.parse(published_at) end end end def serialize(value) ActiveSupport"::JSON.encode(value) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys Meta.new(*Meta.members.map { |name| decoded[name] }) end end
  46. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 46 Attributes API + dry-struct

    = ♥ class PostMetaType < ActiveRecord"::Type"::Value class Meta < Dry"::Struct attribute :published_at, Dry.Types"::JSON"::Date.optional end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys values = Meta.attribute_names.each_with_object({}) { |a, h| h[a] = decoded[a] } Meta.new(values) end end
  47. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev JSON–колонки как модели 47 Data

    Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели
  48. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 48 Типы для работы с

    JSON похожи class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end class UserSyncType < ActiveRecord"::Type"::Value class Sync include ActiveModel"::Model include ActiveModel"::Attributes attribute :last_sync_at, :datetime attribute :success, :boolean end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Sync.new(decoded) end end
  49. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 49 Очень много скобок! product

    = Product.find(params[:id]) if product.configuration[:model] "== "spaceship" product.configuration[:color] = "red" end product.save
  50. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 50 product = Product.find(params[:id]) if

    product.configuration.model "== "spaceship" product.configuration.color = "red" end product.save Очень много скобок!
  51. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 51 JSON нужно валидировать product

    = Product.new puts product.valid? # "=> false puts product.errors.messages # "=> { configuration: ["is invalid"] } puts product.configuration.errors.messages # "=> { color: ["can't be blank"] }
  52. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 52 store_model https:"//github.com/DmitryTsepelev/store_model class Configuration

    include StoreModel"::Model attribute :model, :string enum :status, %i[active archived], default: :active validates :status, presence: true end class Product < ApplicationRecord attribute :configuration, Configuration.to_type validates :configuration, store_model: true end
  53. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 53 store_model: модель https:"//github.com/DmitryTsepelev/store_model module

    StoreModel module Model def self.included(base) base.include ActiveModel"::Model base.include ActiveModel"::Attributes base.include StoreModel"::NestedAttributes base.extend StoreModel"::Enum base.extend StoreModel"::TypeBuilders end end end
  54. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 54 store_model: генерация типов https:"//github.com/DmitryTsepelev/store_model

    module StoreModel module TypeBuilders def to_type Types"::JsonType.new(self) end end end
  55. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 55 store_model: тип–обертка class StoreModel"::Types"::JsonType

    < ActiveModel"::Type"::Value def initialize(model_klass) @model_klass = model_klass end … end https:"//github.com/DmitryTsepelev/store_model
  56. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 56 store_model: тип–обертка class StoreModel"::Types"::JsonType

    < ActiveModel"::Type"::Value def cast_value(value) case value when String decoded = ActiveSupport"::JSON.decode(value) rescue nil @model_klass.new(decoded) unless decoded.nil? when Hash @model_klass.new(value) when @model_klass, nil value else raise_cast_error(value) end end … end https:"//github.com/DmitryTsepelev/store_model
  57. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 57 store_model: тип–обертка class StoreModel"::Types"::JsonType

    < ActiveModel"::Type"::Value … def serialize(value) case value when Hash, @model_klass then ActiveSupport"::JSON.encode(value) else super end end end https:"//github.com/DmitryTsepelev/store_model
  58. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 58 валидации enums атрибуты–списки вложенные

    модели обработка необъявленных полей Что еще можно сделать? https:"//github.com/DmitryTsepelev/store_model
  59. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 59 https:"//bit.ly/3amqMeL

  60. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 60 nested_record https:"//github.com/marshall-lee/nested_record class User

    < ActiveRecord"::Base include NestedRecord"::Macro has_one_nested :profile end class Profile < NestedRecord"::Base attribute :age, :integer attribute :active, :boolean has_one_nested :contacts end class Profile"::Contacts < NestedRecord"::Base attribute :email, :string attribute :phone, :string end user.profile.age = 39 user.profile.contacts.email = 'john@doe.com'
  61. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 61 вложенность дает накладные расходы

    возможно стоит исключить JSON из выборки по умолчанию ⚠ JSON — не серебряная пуля ⚠
  62. evl.ms/blog @dmitrytsepelev DmitryTsepelev @evilmartians evl.ms/telegram THANK YOU! RUBY DAY 2020

    62