ElasticSearch at Recurly

Slide 1

Slide 1 text

Elasticsearch at Recurly Isaac Hall & Aaron Heckmann | Nov 2013

Slide 2

Slide 2 text

Elasticsearch at Recurly | RECURLY Subscription Billing. Simplified. 2

Slide 3

Slide 3 text

Elasticsearch at Recurly | AGENDA • Our use case for Elasticsearch" • First integration: lessons learned" • Rebuilding an Elasticsearch integration 3

Slide 4

Slide 4 text

Elasticsearch at Recurly | SEARCH UI V1 - 2009-10 • Our use case for Elasticsearch" • First integration: lessons learned" • Rebuilding an Elasticsearch integration 4

Slide 5

Slide 5 text

Elasticsearch at Recurly | SEARCH UI V2.5 - 2012 • SQL based Facets" • couldn’t query + facet/filter" • poor performance (30+ sec) 5 Search Facets

Slide 6

Slide 6 text

Elasticsearch at Recurly | 6 FIRST ATTEMPT

Slide 7

Slide 7 text

Elasticsearch at Recurly | WHAT WE HAD BEFORE (OPS) • Single Elasticsearch server" • version 0.19.1 (released March 2012)" • Index grew to over 40 GB 7

Slide 8

Slide 8 text

Elasticsearch at Recurly | 8 PROBLEMS

Slide 9

Slide 9 text

Elasticsearch at Recurly | PROBLEMS • Single Point of Failure (SPOF)" – Callbacks index new data synchronously" • No ability to perform Elasticsearch maintenance" • Slow SQL facets" • Indexes tightly coupled to the model" • ES data accumulated lots of deleted records" • Index jobs from Resque created CPU load 9

Slide 10

Slide 10 text

Elasticsearch at Recurly | 10 REIMPLEMENTATION GOALS

Slide 11

Slide 11 text

Elasticsearch at Recurly | GOALS • Faster search" • Decouple indexing from ActiveRecord" • Support Elasticsearch maintenance" • Keep the index up to date 11

Slide 12

Slide 12 text

Elasticsearch at Recurly | 12 WHAT WE HAVE NOW

Slide 13

Slide 13 text

Elasticsearch at Recurly | 13

Slide 14

Slide 14 text

Elasticsearch at Recurly | WHAT WE HAVE NOW (OPS) •Cluster of 3 ES instances" •version 0.90.2" •10 GB index (compressed!) 14

Slide 15

Slide 15 text

Elasticsearch at Recurly | WHAT WE HAVE NOW (CODE) • Stretcher gem (https://github.com/PoseBiz/stretcher)" • Query builder — supports advanced queries & facets" • Less model coupling" • Asynchronously indexing records" • Bulk indexer jobs to reindex the world" • Sidekiq 15

Slide 16

Slide 16 text

Elasticsearch at Recurly | 16 RESULTS

Slide 17

Slide 17 text

Elasticsearch at Recurly | RESULTS • High Availability, no more SPOF" • improved search performance" –upwards of 20 secs —> 150ms" • no more SQL facets 17

Slide 18

Slide 18 text

Elasticsearch at Recurly | ELASTICSEARCH FACETS 18

Slide 19

Slide 19 text

Elasticsearch at Recurly | 19 LESSONS

Slide 20

Slide 20 text

Elasticsearch at Recurly | LESSONS LEARNED • Disable auto-creation of new indexes" • Flatten documents and denormalize" • Pay attention to analyzer type" • Use bulk API 20

Slide 21

Slide 21 text

Elasticsearch at Recurly | 21 ROADMAP

Slide 22

Slide 22 text

Elasticsearch at Recurly | ROADMAP • Move indexing to own service" • More advanced filters in UI" • Expanded reporting capabilities 22

Slide 23

Slide 23 text

Elasticsearch at Recurly | 23 QUESTIONS?

Slide 24

Slide 24 text

Elasticsearch at Recurly | QUESTIONS / RESOURCES • https://github.com/karmi/retire" • https://github.com/PoseBiz/stretcher" • http://www.elasticsearch.org" • https://www.recurly.com" • https://twitter.com/recurly" • We’re Hiring! - [email protected] 24

Slide 25

Slide 25 text

Elasticsearch at Recurly | Thank You [email protected]