Aggrobot

This is the deck for a talk given at the RubyConf India 2015. The topic was Aggrobot

Transcript

1. gem 'aggrobot' ! @shadabansari
2. CONFESSIONS
3. Not my first talk on Aggrobot at RubyConf India
4. RubyConf India 2014 " Lightning Talks
5. RubyConf India 2014 " Lightning Talks Blah Blah Blah.....Aggrobot... blah blah.. blah.....Full talk would've been so much more awesome, sadly it wasn’t selected L Oh Man I Look so happy but I am sad .. I should definitely select his talk next time
6. INCEPTION SUCCESSFUL
7. Aggrobot Go make me an Aggregation
8. Aggregations Operations on Data groups
9. Rails Methods .sum .average .minimum .maximum .count .pluck .group
10. Example
11. Average score of Sachin Tendulkar against each country Data # Scores of Sachin Operation # Average Grouping # Country
12. $ Score.column_names ["id", "name", "venue", "score", "month", "year", "match_result", "not_out", "against", "played_in",…] Table Structure Extremely Denormalized
13. Score .where(name: 'Sachin Tendulkar') .group(:against) .average(:score) CODE
14. Score .where(name: 'Sachin Tendulkar') .group(:against) .pluck(:name, 'AVG(score)') .to_h CODE WITH PLUCK
15. Aggrobot.start(Score.where(name: 'Sachin Tendulkar')) do hash do group_by :against select avg_score: avg(:score) iterate end end AGGROBOT CODE
16. hash do # group_by :against # select avg_score: avg(:score) # iterate end AGGROBOT CODE vs OUTPUT { <against  name>  : {avg_score : <value>} }
17. Declarative vs Imperative
18. Aggrobot.start(Score.where(name: 'Sachin Tendulkar')) do list do group_by :against select avg_score: avg(:score) iterate end end AGGROBOT CODE with LIST
19. More Examples
20. Average score of Sachin Tendulkar against each country for each year
21. def year_grouping Aggrobot.block do hash do group_by :year select avg_score: avg(:score) iterate end end end AGGROBOT CODE BLOCK
22. module Aggrobot def self.block(&block) block end end AGGROBOT.BLOCK
23. def country_grouping Aggrobot.block do hash do group_by :against recurse year_grouping end end end AGGROBOT CODE BLOCK
24. Function Composition
25. Aggrobot.start( Score.where(name: 'Sachin Tendulkar'), country_grouping) AGGROBOT CODE
26. More Functional Code
27. def year_grouping(sub_grouping = nil) ... sub_grouping ? recurse(sub_grouping) : iterate ... end def country_grouping(sub_grouping = nil) ... sub_grouping ? recurse(sub_grouping) : iterate ... end AGGROBOT CODE BLOCK
28. Aggrobot.start( Score.where(name: 'Sachin Tendulkar'), country_grouping(year_grouping)) Aggrobot.start( Score.where(name: 'Sachin Tendulkar'), year_grouping(country_grouping)) AGGROBOT CODE
29. Extra Constraints
30. Average Score of Sachin Tendulkar against each country for each year* *Only Top 2 countries by total runs scored, rest should be grouped as `Others`
31. def country_grouping Aggrobot.block do hash do group_by :against other_group: 'Others', sort_by: sum(:score) select avg_score: avg(:score) recurse year_grouping end end end AGGROBOT CODE BLOCK , limit_to: 2,
32. More Constraints
33. Total Score, Average Score, Number of Wins of Sachin Tendulkar against each country bucket-ized by score* *Buckets of 0-10, 11-30, 31-50, 51-75, 76-100, 101-150, 151-200, 201-300, 301+
34. SCORE_BUCKETS = [0..10, 11..30, 31..50, 51..75, 76..100, 101..150, 151..200, 201..300, 301..Float::INFINITY] def score_grouping Aggrobot.block do hash do group_by :score, buckets: SCORE_BUCKETS select avg_score: avg(:score), total_score: sum(:score), wins: count_condition('match_result = "Won"') iterate end end end AGGROBOT CODE BLOCK
35. def country_grouping Aggrobot.block do hash do group_by :against recurse score_grouping end end end AGGROBOT CODE BLOCK
36. Let us Do something crazy
37. Let us Blow our mind
38. By Running an aggregation With 600+ queries & Just 4 Aggrobot Blocks
39. DEMO APP
40. ARCHITECTURE
41. SUGAR ARR ARR = ActiveRecord::Relation
42. PING PONG with ActiveRecord::Relation
43. Aggrobot.block group_by :a recurse Aggrobot.block group_by :b recurse Aggrobot.block group_by :c recurse Aggrobot.block group_by :d Aggrobot.start(rel, block) AGGROBOT CHAINING # rel.group(:a) # rel.where(a: 'a1').group(:b) # rel.where(a: 'a1’).where(b: ‘b1’).group(:c) # rel.where(a: 'a1’).where(b: ‘b1’) .where(c: ‘c1’).group(:d)
44. AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION AGGREGATION
45. INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION INCEPTION
46. Work In Progress
47. DOCUMENTATION
48. WHAT DOCUMENTATION ?
49. CODE
50. YOU’RE NOT SUPPOSED TO DO THAT !!
51. % github: shadabahmed/aggrobot
52. THAT’S ALL !!
53. QUESTIONS ????
54. THANK YOU