Data Mapping: the Rails Way

1. Data Mapping в Rails
2. Зачем разработчику знать про Attributes API
3. JSON–колонки как модели

Transcript

1. Data Mapping The Rails Way Дмитрий Цепелев, Злые марсиане фото: Bryan Birdwell, https://bit.ly/2Tpy3mU
2. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 2
3. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 3 evilmartians.com
4. DmitryTsepelev @dmitrytsepelev RUBY DAY 2020 4 evilmartians.com
5. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 5 Data mapping — процесс преобразования данных от одного формата к другому
6. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 6 Преобразование типов SELECT "created_at" FROM "users" LIMIT 1 ┌──────────────────────────┐ │ created_at │ ├──────────────────────────┤ │ 2019-11-22 09:29:17.5634 │ └──────────────────────────┘ User.first.created_at # "=> Fri, 22 Nov 2019 12:29:17 MSK +03:00
7. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 7 Построение запросов # date Order.where("created_at ">= ?", Date.today.beginning_of_day) #"=> select * from orders where created_at ">= '20-02-06' # enum Order.where(status: :complete) #"=> select * from orders where status = 3
8. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели 8
9. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 9 Атрибуты модели User.attribute_types # "=> {"id""=>#<ActiveModel"::Type"::Integer ""...>, # "created_at""=>#<ActiveRecord"::ConnectionAdapters"::PostgreSQL"::OID"::DateTime …>, # "updated_at""=>#<ActiveRecord"::ConnectionAdapters"::PostgreSQL"::OID"::DateTime …>, # ""... # }
10. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 10 Автоматическое определение атрибутов module ActiveRecord module ModelSchema def load_schema! @columns_hash = connection.schema_cache .columns_hash(table_name) .except(*ignored_columns) @columns_hash.each do |name, column| define_attribute( name, connection.lookup_cast_type_from_column(column) ) end end end end
11. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 11 ignored_columns class Post < ActiveRecord"::Base self.ignored_columns = %w(meta) end
12. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 12 Определение атрибутов вручную class Configuration include ActiveModel"::Attributes attribute :model, :string attribute :color, :string, default: "red" end config = Configuration.new config.model = 42 puts config.model.inspect # "=> "42" puts config.color.inspect # "=> "red"
13. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 13 Что будет? class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list "== [2] # "=> false
14. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 14 Будет ерунда class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: [] end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [1, 2]
15. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 15 Нужно использовать lambda class TestModel include ActiveModel"::Model include ActiveModel"::Attributes attribute :list, default: "-> { [] } end model1 = TestModel.new model1.list "<< 1 model2 = TestModel.new model2.list "<< 2 model2.list # "=> [2]
16. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 16 ActiveModel"::Type"::BigInteger ActiveModel"::Type"::Binary ActiveModel"::Type"::Boolean ActiveModel"::Type"::DateTime ActiveModel"::Type"::Date ActiveModel"::Type"::Decimal ActiveModel"::Type"::"Float ActiveModel"::Type"::ImmutableString ActiveModel"::Type"::Integer ActiveModel"::Type"::String ActiveModel"::Type"::Time Типы «из коробки» ActiveRecord"::Type"::DateTime ActiveRecord"::Type"::Date ActiveRecord"::Type"::DecimalWithoutScale ActiveRecord"::Type"::Json ActiveRecord"::Type"::Time ActiveRecord"::Type"::UnsignedInteger
17. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 17 class ActiveModel"::Type"::Value # DB "-> Ruby def deserialize(value) cast(value) end # User "-> Ruby def cast(value) cast_value(value) unless value.nil? end # Ruby "-> DB def serialize(value) value end end Что внутри?
18. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 18 Реестр типов ActiveModel::Type::Registry или наследник в ActiveRecord используется AdapterSpecificRegistry умеет находить класс типа по имени ActiveRecord"::Type.registry.lookup(:integer) # "=> #<ActiveModel"::Type"::Integer: …>
19. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 19 Создание собственного типа class MoneyType < ActiveRecord"::Type"::Integer def cast(value) if !value.kind_of?(Numeric) "&& value.include?('$') price_in_dollars = value.gsub(/\$/, '').to_f super(price_in_dollars * 70) else super end end end
20. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 20 Использование собственного типа ActiveRecord"::Type.register( :money, MoneyType ) class User < ApplicationRecord attribute :balance, :money end User.new(balance: '$1').balance # "=> 70 class User < ApplicationRecord attribute :balance, MoneyType.new end User.new(balance: '$1').balance # "=> 70 через реестр напрямую
21. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 21 ⚠ Type cast происходит при чтении ⚠ class IntegerRangeType < ActiveRecord"::Type"::Integer def initialize(range) @range = range end def cast(value) raise ArgumentError unless range.cover?(value) super end end class User include ActiveModel"::Attributes attribute :age, IntegerRangeType.new(0"..100) end user = User.new user.age = 120 puts user.age # "=> ArgumentError
22. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev Зачем разработчику знать про Attributes API 22 Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели
23. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 23 Виртуальные атрибуты class User < ApplicationRecord attribute :skip_confirmation, :boolean, default: false validate :confirmation, unless: :skip_confirmation? end user = User.new(skip_confirmation: '0')
24. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 24 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 0 0 0
25. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 25 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) 0 0 0 0 1 0 OR
26. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 26 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" 0 1 0
27. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 27 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 0 1 0 0 1 0 AND
28. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 28 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true 0 1 0 0 1 0 AND
29. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 29 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 0 1 0 1 0 0 AND
30. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 30 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false 0 1 0 1 0 0 AND
31. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 31 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) 0 1 0 1 0 0 OR
32. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 32 Битовые операции IN_APP = 0 EMAIL = 1 PUSH = 2 settings = 0 settings "|= (1 "<< EMAIL) # "=> 2 settings.to_s(2) # "=> "10" settings & (1 "<< EMAIL) > 0 # "=> true settings & (1 "<< PUSH) > 0 # "=> false settings "|= (1 "<< PUSH) # "=> 6 settings.to_s(2) # "=> "110" settings & (1 "<< PUSH) > 0 # "=> true 1 1 0
33. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 33 Value Object class NotificationSettings < ActiveRecord"::Type"::Value IN_APP = 0 EMAIL = 1 PUSH = 2 … end
34. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 34 Value Object class NotificationSettings < ActiveRecord"::Type"::Value … def cast(value) Value.new(value) end end
35. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 35 Value Object class NotificationSettings < ActiveRecord"::Type"::Value … class Value attr_reader :settings def initialize(settings) @settings = settings end def subscribe(subscription) @settings "|= (1 "<< subscription) end def receives_email? @settings & (1 "<< EMAIL) > 0 end end end
36. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 36 Value Object class User < ActiveRecord+::Base attribute :notification_settings, NotificationSettings.new, default: 0 end user = User.new puts user.notification_settings.receives_email? # "=> false user.notification_settings.subscribe(NotificationSettings"::EMAIL) puts user.notification_settings.receives_email? # "=> true
37. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 37 Работа с legacy колонками class Post < ActiveRecord"::Base attribute :published_at, StringDateType.new end Post.insert_all([{ published_at: "2012-10-17 14:35" }]) puts Post.last.published_at.inspect # "=> 2012-10-17 14:35:00 +0400 Post.create(published_at: Time.now) puts Post.last.published_at # "=> 2020-02-12 20:15:00 +0300
38. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 38 Работа с legacy колонками class StringDateType < ActiveRecord"::Type"::Value LEGACY_FORMAT = "%Y-%m-%d %H:%M" def serialize(value) value.strftime(LEGACY_FORMAT) end def deserialize(value) Time.strptime(value, LEGACY_FORMAT) end end
39. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 39 Работа с JSON колонками class Post < ActiveRecord"::Base attribute :meta, PostMetaType.new end post = Post.new(meta: { published_at: 1.week.ago }) puts post.meta.published_at # "=> 2020-02-05 post.meta.published_at = Time.now post.save puts post.meta.published_at # "=> 2020-02-12
40. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 40 Работа с JSON колонками class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end
41. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 41 Вызываем конструктор 50000 раз: Бенчмарк по памяти struct dry-struct dry-initializer ActiveModel virtus 0 225000 450000 675000 900000 memsize objects strings https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda
42. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 42 Вызываем конструктор 50000 раз: Бенчмарк по производительности struct dry-initializer dry-struct ActiveModel virtus 0 1500000 3000000 4500000 6000000 ips https:"//gist.github.com/IvanShamatov/94e78ca52f04f20c6085651345dbdfda
43. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 43 Но я хочу использовать PORO!
44. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 44 class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end
45. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 45 Attributes API + struct = ♥ class PostMetaType < ActiveRecord"::Type"::Value Meta = Struct.new(:published_at) do def initialize(published_at=nil) self.published_at = if published_at.nil? "|| published_at.is_a?(Date) published_at else Date.parse(published_at) end end end def serialize(value) ActiveSupport"::JSON.encode(value) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys Meta.new(*Meta.members.map { |name| decoded[name] }) end end
46. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 46 Attributes API + dry-struct = ♥ class PostMetaType < ActiveRecord"::Type"::Value class Meta < Dry"::Struct attribute :published_at, Dry.Types"::JSON"::Date.optional end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = (value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value).symbolize_keys values = Meta.attribute_names.each_with_object({}) { |a, h| h[a] = decoded[a] } Meta.new(values) end end
47. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev JSON–колонки как модели 47 Data Mapping в Rails Зачем разработчику знать про Attributes API JSON–колонки как модели
48. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 48 Типы для работы с JSON похожи class PostMetaType < ActiveRecord"::Type"::Value class Meta include ActiveModel"::Model include ActiveModel"::Attributes attribute :published_at, :date end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Meta.new(decoded) end end class UserSyncType < ActiveRecord"::Type"::Value class Sync include ActiveModel"::Model include ActiveModel"::Attributes attribute :last_sync_at, :datetime attribute :success, :boolean end def serialize(value) ActiveSupport"::JSON.encode(value.attributes) end def cast(value) decoded = value.is_a?(String) ? ActiveSupport"::JSON.decode(value) : value Sync.new(decoded) end end
49. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 49 Очень много скобок! product = Product.find(params[:id]) if product.configuration[:model] "== "spaceship" product.configuration[:color] = "red" end product.save
50. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 50 product = Product.find(params[:id]) if product.configuration.model "== "spaceship" product.configuration.color = "red" end product.save Очень много скобок!
51. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 51 JSON нужно валидировать product = Product.new puts product.valid? # "=> false puts product.errors.messages # "=> { configuration: ["is invalid"] } puts product.configuration.errors.messages # "=> { color: ["can't be blank"] }
52. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 52 store_model https:"//github.com/DmitryTsepelev/store_model class Configuration include StoreModel"::Model attribute :model, :string enum :status, %i[active archived], default: :active validates :status, presence: true end class Product < ApplicationRecord attribute :configuration, Configuration.to_type validates :configuration, store_model: true end
53. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 53 store_model: модель https:"//github.com/DmitryTsepelev/store_model module StoreModel module Model def self.included(base) base.include ActiveModel"::Model base.include ActiveModel"::Attributes base.include StoreModel"::NestedAttributes base.extend StoreModel"::Enum base.extend StoreModel"::TypeBuilders end end end
54. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 54 store_model: генерация типов https:"//github.com/DmitryTsepelev/store_model module StoreModel module TypeBuilders def to_type Types"::JsonType.new(self) end end end
55. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 55 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value def initialize(model_klass) @model_klass = model_klass end … end https:"//github.com/DmitryTsepelev/store_model
56. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 56 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value def cast_value(value) case value when String decoded = ActiveSupport"::JSON.decode(value) rescue nil @model_klass.new(decoded) unless decoded.nil? when Hash @model_klass.new(value) when @model_klass, nil value else raise_cast_error(value) end end … end https:"//github.com/DmitryTsepelev/store_model
57. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 57 store_model: тип–обертка class StoreModel"::Types"::JsonType < ActiveModel"::Type"::Value … def serialize(value) case value when Hash, @model_klass then ActiveSupport"::JSON.encode(value) else super end end end https:"//github.com/DmitryTsepelev/store_model
58. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 58 валидации enums атрибуты–списки вложенные модели обработка необъявленных полей Что еще можно сделать? https:"//github.com/DmitryTsepelev/store_model
59. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 59 https:"//bit.ly/3amqMeL
60. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 60 nested_record https:"//github.com/marshall-lee/nested_record class User < ActiveRecord"::Base include NestedRecord"::Macro has_one_nested :profile end class Profile < NestedRecord"::Base attribute :age, :integer attribute :active, :boolean has_one_nested :contacts end class Profile"::Contacts < NestedRecord"::Base attribute :email, :string attribute :phone, :string end user.profile.age = 39 user.profile.contacts.email = 'john@doe.com'
61. RUBY DAY 2020 DmitryTsepelev @dmitrytsepelev 61 вложенность дает накладные расходы возможно стоит исключить JSON из выборки по умолчанию ⚠ JSON — не серебряная пуля ⚠
62. evl.ms/blog @dmitrytsepelev DmitryTsepelev @evilmartians evl.ms/telegram THANK YOU! RUBY DAY 2020 62