Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast and Flexible Donor Segmentation at Red Cross Using Elasticsearch

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
October 29, 2015

Fast and Flexible Donor Segmentation at Red Cross Using Elasticsearch

Christiaan Stoffer, CTO at Nexios.com, talks about how they implemented tools for volunteers, a platform built on top of Elasticsearch, for the administration of all 30000 volunteers of the Dutch Red Cross.

Christian Stoffer | Elastic{ON} Tour Amsterdam | October 29, 2015

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

October 29, 2015
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. Christiaan Stoffer - October 29 1   Fast and flexible

    Donor Segmentation at Red Cross using Elasticsearch
  2. 2 Christiaan Stoffer CTO / Architect / Developer

  3. 3 National emergency aid First aid International Assistance

  4. Web-application: “Tools for volunteers” 4 •  Platform for all volunteer

    related activities •  Broad adoption through focus on: §  Ease of use §  Fast results §  Self-service •  Thousands of users each week
  5. What about fundraising? 5  

  6. Some facts and figures •  ~ 2 million contacts § 

    Roles like donor, member, volunteer, etc. •  ~ 25 million financial transactions §  Small donations to major donor contributions •  ~ 300 million campaign activities and responses §  Direct marketing, telemarketing, etc. 6
  7. Fundraising application landscape – planned architecture 7 CRM   Finance

     
  8. CRM   Finance   Fundraising application landscape – actual architecture

    8 POOR PERFORMANCE!
  9. 9   “It takes way too long to build our

    market segmentation”
  10. 10   “We cannot import campaign results during work hours”

  11. Solution: stick to original plan & solve performance issues! Business

    evolution not revolution •  Same business processes & principles •  Just need a very powerful engine! 11 CRM   CRM  
  12. Solution: Enter Elasticsearch! With Elasticsearch we are able to… - 

    Provide all the power needed -  Reduce TCO by almost 30% -  Be more flexible; better respond to peaks in usage 12
  13. Main goals for fast and flexible segmentation •  One system

    for all processes §  Easy to use §  Productivity •  Fast segmentation results §  # change as you adjust segments definitions §  Near real-time data for up-to-date results •  Work whenever you want! §  No noticeable performance degradation §  Even with bulk operations 13
  14. Results of proof of concept Using Elasticsearch we are able

    to: •  Get insights really fast §  Milliseconds, not minutes or even hours •  Provide an overall better user-experience §  Focus on results; not waste time on queries •  Build to “keep scaling” 14
  15. What was our key challenge? 15  

  16. Lots of relations need to be traversed or “joined” • 

    Parent – Child §  Slows down really quick: 2+ seconds and slower §  More and bigger hardware does not provide enough improvement §  CPU intensive: noticeable performance degradation •  Nested §  Response-time stays within milliseconds Even with around 20 nested filters in 1 query (!!) §  Slightly more complicated index strategy needed 16
  17. There are boundaries with this solution… Input from Elasticsearch Support:

    Take the number of nested documents into account: •  1M ES docs with ~100 nested docs results in ~100M Lucene docs •  1 shard can hold ~2B Lucene docs And be aware of other things like: •  Complicated mapping biggest performance killer •  Large documents may become a problem at index time 17
  18. Conclusion 18  

  19. Using Elasticsearch… •  We are able to consolidate all systems

    (reducing IT costs by ~30%) •  Provide excellent performance and scalability •  Really empower users with real-time business insight 19 =  
  20. Christiaan Stoffer christiaan@nexios.com Thank you! 20  

  21. www.elas6c.co   21