Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rapid Data Modeling in ActiveRecord with the JSON Data Type

Rapid Data Modeling in ActiveRecord with the JSON Data Type

So you are building an app that has a ton of forms each with tons of fields. Your heart sinks as you think of writing and managing all those models, migrations and associations. PostgreSQL JSON column and ActiveRecord::Store to the rescue! This talk covers a way to wrap these Rails 4 features to simplify the building of extensive hierarchical data models. You will learn to expressively declare schema-less attributes on your model that act much like “real" columns, meaning they are typecast, validated, query able, embeddable, and behave with Rails form builders.

David Furber

April 23, 2015
Tweet

Other Decks in Technology

Transcript

  1. OR How I sped up my Rails development by fitting

    a document store into my ActiveRecord models.
  2. Speed up Lots of Little Projects • Simple Form + Slim

    • View template inheritance • Inherited Resources • Engines for common components • Don’t waste work • Read the Rails API • Be consistent!
  3. PROJECT ARAGATS Digital Catalog of Artifacts Public Research Tool Archaeology

    Armenia Ancient Human Settlements Used in Fieldwork
  4. Ontology of archaeology Bone Pottery Wood Object Stone Object Metal

    Object Ceramic Object Bone Object Butchery Shell Radiocarbon Dendrochronology
  5. It’s just a pot, right? 11 Rim Neck Core Bands

    Base Surface Treatment Pinch Handle Grip Handle Lid Shoulder Spout Leg Outpour Position Body Foot Decorations Identification Periodization Dimensions Possible Use
  6. SELECT * FROM artifacts WHERE (data->>’age’::int) >= 3000 AND (data->>’grip_handle_type’::text)

    = ‘INDT’ AND (data->>’complete’::bool) = ‘t’ 12 Postgres can search JSON?
  7. class Artifact < ActiveRecord::Base # common attributes are columns end

    class Pottery < Artifact # Pot specific attributes end class Fauna < Artifact # Bone specific attributes end 13 A Case for Single Table Inheritance?
  8. Hello store_accessor create_table :artifacts do |t| t.references :place, index: true

    t.integer :period_start t.integer :period_end t.jsonb :data end class Artifact < ActiveRecord::Base store_accessor :data, :width end 14
  9. Hello store_accessor @artifact= Artifact.new width: 20 @artifact.width # => 20

    @artifact.data['width'] # => 20 @artifact.width = ‘20’ # => ‘20’ @artifact.width # => ‘20’ @artifact.width_changed? # => BOOM! @artifact.data_changed? # => true 15
  10. Ransack for search and sort Entity.ransack(width_gteq: 30).result Entity.ransack(s: ‘width desc’).result

    = form.input :width_gteq, as: :number ransacker "width" do Arel.sql "(data->>'width')::int” end 16
  11. def self.integer_attributes(args) args.each do |key| define_method key { value =

    read_store_attribute(:data, key) value && value.to_i } define_method "#{key}=".to_sym { |value| value = value.present? ? value.to_i : nil write_store_attribute(:data, key, value) } ransacker key do Arel.sql "(data->>'#{key}')::int" end end end # Example of use: class Bone < Artifact integer_attributes :length, :width, :height end 18
  12. def self.store_attribute(attr, conversion, ransacker_type) define_method key { value = read_store_attribute(:data,

    key) value && value.send(conversion) } define_method "#{key}=".to_sym { |value| value = value.present? ? value.send(conversion) : nil write_store_attribute(:data, key, value) } ransacker key do Arel.sql "(data->>'#{key}')::#{ransacker_type}” end end 19
  13. def self.store_attributes(attrs, conversion, ransacker_type) attrs.each {|attr| store_attribute attr, conversion, ransacker_type

    } end def self.integer_attributes(*attrs) store_attributes attrs, :to_i, 'int' end def self.string_attributes(*attrs) store_attributes attrs, :to_s, 'text' end 20
  14. class CeramicObject < Artifact boolean_attributes :complete string_attributes :form, :modification, :polishing,

    :other_modification float_attributes :length, :width, :breadth, :diameter, :weight end 21
  15. class CeramicObject < Artifact attribute :complete, :boolean, default: false attribute

    :form, :string attribute :modification, :string attribute :other_modification :string attribute :polishing, :string attribute :length, :decimal attribute :width, :decimal attribute :breadth, :decimal attribute :diameter, :decimal attribute :weight, :decimal end 22
  16. The mission of the Inclusive Recreation Resource Center (IRRC) is

    to promote and sustain participation by people of all abilities in inclusive recreation activities and resources.
  17. Inclusive Recreation Resource Center at SUNY Cortland 21 If yes,

    are the spaces clearly marked with upright signs designating them as accessible? (international symbol of accessibility or clear wording) … yes … no … n/a … photo 22 Number of designated accessible spaces: (if unlined, approximate number of accessible spaces) ______ spaces 23 Width of designated accessible spaces: _______ inches (at least 96”) 24 Number of designated accessible spaces with access aisles: _______ spaces 25 Width of access aisles: _______ inches (at least 60”; 96” in NY) 26 Distance from parking to entrance: (if no designated accessible parking, measure distance from middle of parking lot/area to entrance) _______ feet 27 Is there a continuous unobstructed route of travel from parking to entrance? … yes … no … n/a … photo 28 Is the route of travel well-lit? … yes … no … n/a … photo 29 Describe the surface of the route of travel: 30 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) 31 Width of route of travel (at its narrowest point): ________ inches (at least 36”) 32 Comments/additional information about parking and route of travel to entrance: ENTER: Entrances 33 Name of the first entrance: __________________________________________________________________________________ Attach “Additional Entrance Checklists” for each entrance assessed 34 Is the international symbol of accessibility displayed by entrance? (or signage designating accessibility) … yes … no … n/a … photo 35 Are there stairs to gain entry? … yes … no … n/a … photo 36 Number of stairs: ______ stairs 37 Describe the type of railing: 38 Is a ramp provided? … yes … no … n/a … photo 39 Width of ramp (at its narrowest point): _________ inches (at least 36”) 40 Slope of ramp (at its steepest incline): _________ % (maximum 8%) 41 Depth of ramp landing: _________ inches (at least 60” depth) 42 Describe the surface of ramp: 43 Describe the type of railing on the ramp: 44 Type of door: … manual … automatic … revolving … other:_________________________________ … n/a 45 Are there doors in a series to gain entry? … yes … no … n/a … photo 46 If yes, distance between doors: _________ inches (at least 48”) 47 Clear open width of door: _________ inches (at least 32”) 48 Opening force: (enter n/a if automatic door) _________ pounds (8.5 pounds or less) 49 Handle type entering: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: 50 Handle type exiting: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: 51 Clear space on pull (handle) side of door: _________ inches (at least 18”) Inclusive Recreation Resource Center at SUNY Cortland © 2014 Inclusive Recreation Resource Center 1 Inclusivity Assessment Tool Section A: Agency Information 1 Name of agency 2 Street address (or intersection) (if remote site with no street address, provide GPS coordinates) 3 City 4 State 5 Zip code 6 County 7 Mailing address (if different from street address) 8 Telephone number 9 Agency email address 10 Website address 11 Type of agency 12 Activities offered Section B: Physical Inclusion • For Section B and the Specialty Checklists, you will need a tape measure, a clinometer, a 60” circle, and a fish scale. • Complete an “Additional Building Checklist” for each major facility at the agency. APPROACH: Transportation 13 Does transportation serve area or facility? … yes … no … n/a … photo 14 If yes, type of transportation available: … public … agency 15 If yes, does transportation accommodate a person using a wheelchair? … yes … no … n/a … photo 16 Comments/additional information about transportation: NOTE: If needed, ask about Transportation Services during the interview APPROACH: Parking 17 Name of the first parking lot/area: ______________________________________________________________________________________ Attach “Additional Parking Checklists” for each parking lot/area assessed 18 Type of parking assessed: (if both are present, assess the agency parking) … agency parking … street parking 19 Parking spaces are: … lined … unlined 20 Are designated or marked accessible parking spaces provided? … yes … no … n/a … photo TM Approach Enter Use Inclusive Recreation Resource Center at SUNY Cortland © 2014 Inclusive Recreation Resource Center 3 USE: Registration/Reception/Check-in Area 53 Is there a continuous unobstructed route of travel from entrance (if indoor facility) or parking (if outdoor facility) to registration/check-in area? … yes … no … n/a … photo 54 Is the route of travel well-lit? … yes … no … n/a … photo 55 Describe the surface of the route of travel: 56 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) 57 Width of route of travel (at its narrowest point): ________ inches (at least 36”) 58 Is the registration/reception/check-in counter easily located? … yes … no … n/a … photo 59 Height of counter: ________ inches (36” or less) 60 If counter is higher than 36”, is a fold-out shelf available? … yes … no … n/a … photo 61 Is the registration area large enough for wheelchair maneuvering? (60” circle) … yes … no … n/a … photo 62 If no, provide dimensions of free space in registration area: ________ inches by ________ inches 63 Is signage clear in registration area? (word, picture, and Braille or raised lettering) … yes … no … n/a … photo 64 Comments/additional information about reception/registration/check-in area: USE: Public Restroom/Toilet 65 Name of the first restroom: ________________________________________________________________________________ Attach “Additional Restroom Checklist” for each restroom assessed 66 Is there a continuous unobstructed route of travel from entrance (if indoor facility) or parking (if outdoor facility) to restroom? … yes … no … n/a … photo 67 Is the route of travel well-lit? … yes … no … n/a … photo 68 Describe the surface of the route of travel: 69 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) 70 Width of route of travel (at its narrowest point): ________ inches (at least 36”) 71 Is a single use restroom provided (e.g., family, companion, single room)? … yes … no … n/a … photo 72 Restroom assessed: … single use/ family … female … male 73 Is signage clear? (word, picture, and Braille or raised lettering provided for signs) … yes … no … n/a … photo 74 Clear open width of door to restroom: ________ inches (at least 32”) 75 Opening force: (enter n/a if automatic door) _________ pounds (5 pounds or less) 76 Handle type entering: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: 77 Handle type exiting: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: 78 Clear space on pull (handle) side of door: _________ inches (at least 18”) 79 Does the toilet stall door swing open to outside of stall? … yes … no … n/a … photo 80 Is the toilet area large enough for wheelchair maneuvering? (60” circle) … yes … no … n/a … photo 81 If no, provide dimensions of free floor space in stall/toilet area: ________ inches by ________ inches 82 Is the center line of the toilet positioned 16”-18” from the nearest side wall? … yes … no … n/a … photo 83 Are grab bars mounted on at least two sides of the toilet? … yes … no … n/a … photo 84 Number of grab bars: _________ bars 85 Toilet seat height: ________ inches (between 17”-19”) Inclusive Recreation Resource Center at SUNY Cortland 86 Is the sink area large enough for wheelchair maneuvering? (60” circle) … yes … no … n/a … photo 87 If no, provide dimensions of free space in sink area: ________ inches by ________ inches 88 Height of sink: ________ inches (34” or less from floor) 89 Sink is: … wall-mounted … cabinet 90 Depth of knee space under sink: (from outer edge of sink to pipes or other obstruction under sink) _________ inches (at least 8”) 91 Are faucet controls usable without grasping? … yes … no … n/a … photo 92 Location of soap dispensers: … wall-mounted … on sink 93 If wall-mounted, height of soap dispensers: ______ inches (no higher than 48” from floor) 94 Location of hand dryer/paper towels: … wall-mounted … on sink 95 If wall-mounted, height of hand dryers/paper towel dispensers: ______ inches (no higher than 48” from floor) 96 Comments/additional information about public restroom/toilet: USE: Elevator(s) 97 Is an elevator provided to access recreation areas on other floors? … yes … no … n/a … photo 98 Is there a continuous unobstructed route of travel from entrance to elevator? … yes … no … n/a … photo 99 Is the route of travel well-lit? … yes … no … n/a … photo 100 Describe the surface of the route of travel: 101 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) 102 Width of route of travel (at its narrowest point): ________ inches (at least 36”) 103 Elevator control panel height: ________ inches (48” max above the floor) 104 Methods of communication for elevator controls: (check all that are present) … audible … tactile … visual … none 105 Elevator door width: ________ inches (at least 36”) 106 Does elevator have a sensor that reopens the door or does door stay open at least 20 seconds? … yes … no … n/a … photo 107 Size (clear space) of elevator car: ________ inches by ________ inches 108 Is the elevator well-lit? … yes … no … n/a … photo 109 Comments/additional information about elevator: USE: Stairs 110 Is there a continuous unobstructed route of travel from entrance to the stairs? … yes … no … n/a … photo 111 Is the route of travel well-lit? … yes … no … n/a … photo 112 Describe the surface of the route of travel: 113 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) 114 Width of route of travel (at its narrowest point): ________ inches (at least 36”) 115 Number of stairs: ________ stairs 116 Are the stairs well-lit? … yes … no … n/a … photo 117 Are stair edges clearly marked in contrasting color? … yes … no … n/a … photo 118 Is a handrail provided? … yes … no … n/a … photo © 2014 Inclusive Recreation Resource Center Additional Entrance Checklist ENT1 Facility name: ENT2 Total number of entrances assessed*: ENT3 Name or location of the additional entrance: ENT4 Is the international symbol of accessibility displayed by entrance? (or signage designating accessibility) … yes … no … n/a … photo ENT5 Are there stairs to gain entry? … yes … no … n/a … photo ENT6 Number of stairs: ______ stairs ENT7 Describe the type of railing: ENT8 Is a ramp provided? … yes … no … n/a … photo ENT9 Width of ramp (at its narrowest point): _________ inches (at least 36”) ENT10 Slope of ramp (at its steepest incline): _________ % (maximum 8%) ENT11 Depth of ramp landing: _________ inches (at least 60” depth) ENT12 Describe the surface of ramp: ENT13 Describe the type of railing on the ramp: ENT14 Type of door: … manual … automatic … revolving … other:_________________________________ … n/a ENT15 Are there doors in a series to gain entry? … yes … no … n/a … photo ENT16 If yes, distance between doors: _________ inches (at least 48”) ENT17 Clear open width of door: _________ inches (at least 32”) ENT18 Opening force: (enter n/a if automatic door) _________ pounds (8.5 pounds or less) ENT19 Handle type entering: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: ENT20 Handle type exiting: … automatic … knob … pull … push/pull paddle/bar … entry set-top button … lever … push plate … other: ENT21 Clear space on pull (handle) side of door: _________ inches (at least 18”) ENT22 Comments/additional information about entrance: *Attach “Additional Entrance Checklists” for each additional entrance assessed Inclusive Recreation Resource Center at SUNY Cortland Boating Area Checklist BOAT1 Agency Name: BOAT2 Number of boat areas assessed: ________ boating areas BOAT3 Name/location of boat ramp/launch: BOAT4 Is there a continuous unobstructed route of travel from parking to boat launch? … yes … no … n/a … photo BOAT5 Is the route of travel well-lit? … yes … no … n/a … photo BOAT6 Describe the surface of the route of travel: BOAT7 Slope of route of travel (at its steepest incline): ________ % (enter 0% if flat) BOAT8 Width of route of travel (at its narrowest point): ________ inches (at least 36”) BOAT9 What is the distance from the parking area to the boat launch area? _______ feet (if no accessible parking, measure distance from middle of parking lot/area to launch area) ________ feet BOAT10 Describe the boat launch area (check all that apply): … Natural surface boat launch area (no paving or physical structure) … Hard surface boat launch ramps without a pier … Hard surface boat launch with pier(s) BOAT11 If applicable, is there a ramp or gangway that connects the land to the pier? (a gangway or slope that bridges the span between land and the pier) … yes … no … n/a … photo BOAT12 If yes, how long is the gangway or ramp? ________ feet BOAT13 If yes, what is the width of the gangway or ramp connecting the land to the pier? ________ inches (at least 36”) BOAT14 If yes, describe the slope of the gangway or ramp: ________ % (enter 0% if flat) BOAT15 If yes, are there handrails on the gangway or ramp to assist with transition from land to pier? … yes … no … n/a … photo BOAT16 Are there any physical indicators that there is a transition from the landside connection or gangway to the pier or dock (i.e. textured pathways, changes in color)? … yes … no … n/a … photo BOAT17 Describe the surface of the pier (wooden planks, smooth, few rises, mostly even surface, etc.): BOAT18 What is the width of the pier at its narrowest point? (60” minimum or, where there are obstacles, the route can narrow to 36” for no more than a 24” length on the pier) ________ inches BOAT19 Is there a boat slip (either a permanent mooring spot or space to temporarily tie off) where a 60” clear route runs the length of the boat slip? … yes … no … n/a … photo BOAT20 Where there are obstructions on the pier, is there a 60” wide clear space at least every 10 feet along the pier? … yes … no … n/a … photo BOAT21 Is edge protection provided at continuous openings (lips at sides of piers, docks, or gangways with a maximum of 4” high and 2” wide)? … yes … no … n/a … photo BOAT22 Comments/additional information regarding boating area:
  18. Domain Driven Design •  Assessments of physical structures •  Common

    elements such as doors, ramps, routes, parking, stairs •  Never need to ask how many comfort stations have restrooms with 60” maneuver space and handle flush •  Value objects = composition through embeddable models? 29
  19. A door is an embedded model class EmbeddedModel include ArDocStore::EmbeddableModel

    attribute :name, :string attribute :comments, :string attribute :is_present, :boolean end 30
  20. A door in code class Door < EmbeddedModel attribute :door_type,

    :enumeration, multiple: true, dictionary: :door_types attribute :open_handle, :enumeration, multiple: true, dictionary: :door_handle_types attribute :close_handle, :enumeration, multiple: true, dictionary: :door_handle_types attribute :clear_distance, as: :integer attribute :opening_force, as: :integer attribute :clear_space, as: :integer end 31
  21. A comfort station has a door class SpecialtyArea < ActiveRecord::Base

    include ArDocStore::Model embeds_one :door embeds_one :route embeds_one :ramp end class ComfortStation < SpecialtyArea embeds_many :restrooms embeds_many :showers embeds_one :parking_area end 32
  22. So do restrooms and showers class Restroom < EmbeddedModel embeds_one

    :door . . . end class Shower < EmbeddedModel embeds_one :door . . . end 33
  23. And the door form code = form.fields_for :door do |door|

    = door.input :is_present, as: :boolean, input_html: {data: {toggle: 'form', form: 'door'}}, wrapper_html: {class: 'toggler'} = door.input :clear_distance, as: :integer, input_html: { postfix: 'inches', append: '(at least 32")' } = door.input :open_handle, as: :check_boxes_other = door.input :close_handle, as: :check_boxes_other = render ‘base/doors’, form: form 34
  24. With nested attributes def create_embed_one_attributes_method model.class_eval <<-CODE, __FILE__, __LINE__ +

    1 def #{attribute}_attributes=(values={}) values.symbolize_keys! if values[:_destroy] && (values[:_destroy] == '1') self.#{attribute} = nil else #{attribute}.attributes = values end end CODE end 36
  25. Sweet Spots • STI with variant attributes • Storing value objects and

    arrays • Storing JSON API responses • Adding fields without a migration • Not a permanent commitment • Advantages of a document store
  26. Sour Spots • Disadvantages of a document store • No partial updates

    • Complex searches on JSON keys? • Nesting JSON slows queries 10x
  27. Where to go from here? Check out ArDocStore on GitHub

    at: dfurber/ar_doc_store And read the blog posts linked from there! 40