Save 37% off PRO during our Black Friday Sale! »

POLO: Working With Real World Data In Development

POLO: Working With Real World Data In Development

D3247e8f4b25f6041ab71da426f1c86e?s=128

Ivayr Farah Netto

September 25, 2015
Tweet

Transcript

  1. POLO WORKING WITH REAL WORLD DATA IN DEVELOPMENT @nettofarah 1

    / 39
  2. nettofarah github.com/nettofarah @nettofarah http://bit.ly/polo-rb 2 / 39

  3. Real World Production Data in Development 3 / 39

  4. Because Managing Data is hard Especially across Different environments 4

    / 39
  5. Data in Rails.env.dev is... Ugly, Incomplete, Weird and Biased 5

    / 39
  6. Some Consequences of Inconsistent Data 6 / 39

  7. !" !", # # misalignment, form validation, special characters $

    $ 7 / 39
  8. The Worst Consequence 8 / 39

  9. Rails.env.dev will look % 9 / 39

  10. But how do people solve these problems? 10 / 39

  11. We can test it in PRODUCTION 11 / 39

  12. Rails.env.staging ? 12 / 39

  13. Custom db/migrations for data 13 / 39

  14. rake db:seed 14 / 39

  15. CSVs and Spreadsheets 15 / 39

  16. 16 / 39

  17. Sample Database Snapshots 17 / 39

  18. Your good ol' ActiveRecord::Associations to .sql 18 / 39

  19. POLO lets you turn this... 19 / 39

  20. class Chef < ActiveRecord::Base has_many :recipes has_many :ingredients, through: :recipes

    end class Recipe < ActiveRecord::Base has_many :recipes_ingredients has_many :ingredients, through: :recipes_ingredients end class Ingredient < ActiveRecord::Base end class RecipesIngredient < ActiveRecord::Base belongs_to :recipe belongs_to :ingredient end 20 / 39
  21. Into this... 21 / 39

  22. inserts = Polo.explore(Chef, 1) # Chef -> ActiveRecord::Base object #

    1 -> Database ID. (Chef with ID 1) INSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto') 22 / 39
  23. It works with associations too 23 / 39

  24. inserts = Polo.explore(Chef, 1, :recipes) # :recipes -> # ActiveRecord::Associations::HasManyAssociation

    # # a Chef has many Recipes INSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto') INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1) INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1) 24 / 39
  25. It also works with nested associations 25 / 39

  26. inserts = Polo.explore(Chef, 1, { :recipes => :ingredients }) #

    { :recipes => :ingredients } -> # load every recipe and ingredientes ... INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1) INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1) INSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (1, 1, 1) INSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (2, 1, 2) ... INSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (1, 'Turkey', 'a lot') INSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (2, 'Cheese', '1 slice') ... 26 / 39
  27. & & Show me the magic! 27 / 39

  28. Collect all the objects lib/polo/collector.rb#L16 28 / 39

  29. asn = ActiveSupport::Notifications # Extracted so this can fit in

    a slide asn.subscribed(collector, 'sql.active_record') do # Set up ActiveRecord::Preloader base_finder = @base_class. includes(@dependency_tree) .where(id: @id) # Store SELECTS in some global storage collect_sql(@base_class, base_finder.to_sql) end 29 / 39
  30. ' ' Transform them to SQL lib/polo/sql_translator.rb#51 30 / 39

  31. attributes = record.attributes keys = attributes.keys.map do |key| "`#{key}`" end

    values = attributes.map do |key, value| column = record.column_for_attribute(key) attribute = cast_attribute(record, column, value) connection.quote(attribute) end joined_keys = keys.join(', ') joined_values = values.join(', ') table_name = record.class.table_name "INSERT INTO `#{table_name}`" + " (#{joined_keys}) VALUES (#{joined_values})" 31 / 39
  32. PROFIT! ( ( 32 / 39

  33. I need your help! ) ) 33 / 39

  34. on Github http://github.com/IFTTT/polo 34 / 39

  35. Pull Requests * * 35 / 39

  36. PostgreSQL, SQLite and Oracle support (advanced features only, basic usage

    still works) 36 / 39
  37. ActiveRecord < 3.2 support 37 / 39

  38. Check out some other really cool Open Source projects http://ifttt.github.io

    38 / 39
  39. Questions? 39 / 39