Today I want to talk about algorithms. 

Today I want to talk about algorithms. an algorithm. 

My name is Gordon Diggs. 

I work here: www.paperlesspost.com @paperlesspost / @paperlessdev 

But this talk isn't about that. 

I collect these: 

But I started getting a lot of them 

So I made this: vinyl.gordondiggs.com 

It makes these: ooooo...pretty! 

At first, I did this settings.fields.each do |field| occurrences[field] = {} Item.all.each do |item| value = item.send(field.robotize) if occurrences[field][value] occurrences[field][value] += 1 else occurrences[field][value] = 1 end end if (others = occurrences[field].select{|k,v| v==1}.size) > 1 occurrences[field].delete_if{|k,v| v==1} occurrences[field]["Other"] = others end end 

That sucks! Super slow 

That sucks! With 9000 items, it takes ~23.4457 seconds 

OK, fine. What do? Reduce the need to do it: cache the result, only recalculate when something changes 

But it's still slow! Find another way to do it. Step 1: ditch mongodb for postgres 

A new dawn awaits SQL has cool stuff, like counting and grouping (we want that, right?) 

The new solution settings.fields.each do |field| occurrences[field] = {} items = DataMapper.repository.adapter.select( "SELECT #{field.robotize} as \"col\", COUNT(*) as \"times\" FROM items GROUP BY #{field.robotize} ORDER BY \"times\" desc") items.each do |item| occurrences[field][item[:col]] = item[:times] end if (others = occurrences[field].select{|k,v| v==1}.size) > 1 occurrences[field].delete_if{|k,v| v==1} occurrences[field]["Other"] = others end end 

How fast is that? With the same 9000 items, it takes ~0.4663 seconds 

Why is that? Because SQL is way better at this than we are. 

The Benefits ● Can run it whenever I want without repercussions ● Scales way better ● Can stare at graphs all day 

The Takeaway ● There are things that ruby does really well, and things that SQL does better. 

Thank you. @gordondiggs