M A G G I E N E L S O N ( @ M A G G I E 1 0 0 0 ) Findery: an Elasticsearch case study 

R E D I S C O V E R Y O U R W O R L D What is Findery? 

No content

No content

No content

F I N D E RY T I M E L I N E 

L O N G , L O N G A G O … 

• Young startup 

• RoR application 

• Easy to prototype 

• Easy to build in technical debt 

“Hey, Maggie, can you add search?” 

S E A R C H : F I R S T I T E R AT I O N 

• using Solr • welcoming and informative community 

• Solr isn’t AWS’s target audience • lots of babysitting of machines 

• very robust, lots of features • geo features not in by default pre 4.x release - manage via plugins 

• query language has a steep learning curve • difficult to share code with new/inexperienced developers 

S E A R C H : E L A S T I C S E A R C H 

“Y’know, for search!” 

• a welcoming and informative community 

• easy to set up and monitor on AWS • chef driven setup, multiple data nodes behind a load balancer • new nodes discoverable via well-maintained plugins • additional internal monitoring using Sematext 

• easy to share with new developers (JSON ftw!) • geo features in by default 

A S T H E C O M PA N Y G R O W S … 

• Moving portions of the app from RoR to Node.js • Easy transition 

• JSON queries remain usually unchanged • some changes to indexing strategy 

No content

G E N E R A L S I T E S E A R C H • ES 101 • index objects, search objects using basic match / query_string queries 

W O R D A U T O C O M P L E T E • I <3 ngrams • tags autocomplete • username autocomplete 

H AV E R S I N E F O R M U L A D I S TA N C E B E T W E E N P O I N T S O N T H E S U R FA C E O F A S P H E R E 

A U T O C O M P L E T E W I T H A G E O C O M P O N E N T • “find venues near me” • about 60M records • ngram autocomplete for names of venues • combined with sorting by latitude and longitude • good performance when queries are pre warmed (in 10’s of ms) • even better performance if you can set a max on distance 

D ATA O N A M A P : I S S U E S 

• potentially lots of notes in a small area 

• potentially lots of notes in a small area • potentially very few notes in a big area 

• potentially lots of notes in a small area • potentially very few notes in a big area • some notes are (very subjectively) better than others 

• potentially lots of notes in a small area • potentially very few notes in a big area • some notes are (very subjectively) better than others • some notes are (very objectively) better than others 

D ATA O N A M A P : S O L U T I O N S 

“ M A N Y N O T E S I N A S M A L L A R E A ” • simple bounding box query • boost notes based on scores • prevent “rich get richer” with score half life • boost newer content where it makes sense 

“ F E W N O T E S I N A B I G A R E A ” • expand bbox until you find something? • order by distance AND score? (kind of expensive) • determine some sort of a base score? • quality vs. performance 

“ S O M E N O T E S A R E ( S U B J E C T I V E LY ) B E T T E R T H A N O T H E R S ” • people tend to like what their friends like (incorporate the friend graph into queries) • people who liked X also liked Y (generate a few Y for every X) • a little bit of editorial content (easiest tech-wise, needs lots of people) 

“ S O M E N O T E S A R E ( O B J E C T I V E LY ) B E T T E R T H A N O T H E R S ” • spam detection • note scoring per-user, per-IP, per-anything 

No content

“ P O P U L A R TA G S I N T H I S A R E A ” • fun to implement • lots of lat/long pairs • essentially a simple geo search based on object’s attributes 

C O L L E C T I O N S O F N O T E S • multiple lat/long points in a collection • what does “near me” mean? • shape queries 

“The interesting problems are the ones that are difficult.” 

Q & A 

• Photo credits: • http://www.flickr.com/photos/m2w2/1400035437/ • https://www.artsjournal.com/aestheticgrounds/bunnies-and-rabbits-and-hares-passivity-in-publicart/roas- rabbit-in-hackney-2008a/ • http://www.florentijnhofman.nl/dev/project.php?id=181 • http://exhibitioninquisition.wordpress.com/2012/02/10/four-facts-this-will-have-been/ • http://silence-design.deviantart.com/art/Rabbit-Street-Art-159681282 • http://www.flickr.com/photos/boonovista/391812573/ ! • More rabbits to visit: https://findery.com/heather/sets/rabbits-to-visit 

M A G G I E N E L S O N @ M A G G I E 1 0 0 0 T H A N K S ! ! ! ! ! ! F I N D E RY. C O M